As part of the Tiny Aya Expedition with Cohere Labs, mentored by Dr. Tom Hosking, I collaborated on a project asking whether circuits in multilingual LLMs are language-specific, script-specific, regional, or universal.
We trained and analyzed Sparse Autoencoders (SAEs), then evaluated generation quality before and after feature steering across 67 languages using LaBSE semantic similarity, script conformity, LLM-as-a-judge evaluation, and comparisons with Flores-200 and CulturaX.