Google’s Gemma Scope Opens Up AI Interpretability

Understanding why a language model produces a particular output remains one of the harder problems in machine learning. The internal activations of a network blend many concepts together, and individual neurons often fire for unrelated features, which makes it difficult to trace a specific behavior back to a specific cause. On July 31, 2024, Google DeepMind released Gemma Scope, an open suite of tools built to help researchers examine the inner workings of its Gemma 2 models. Details are available in the announcement at https://deepmind.google/discover/blog/gemma-scope-helping-the-safety-community-shed-light-on-the-inner-workings-of-language-models/.

How Sparse Autoencoders Help

Gemma Scope is built around sparse autoencoders, which DeepMind compares to a microscope for looking inside a model. A sparse autoencoder decomposes each dense activation into a small number of underlying features, and it does so without researchers having to specify in advance which features to look for. This addresses a core difficulty in interpretability work: because activations mix many signals at once, reading them directly is rarely informative.

The release uses what DeepMind calls the JumpReLU SAE architecture. According to the announcement, this approach better balances two tasks that sparse autoencoders have to handle at the same time: detecting whether a given feature is present, and estimating how strongly it is active. DeepMind states that this reduces error compared with earlier approaches.

What the Release Covers

Gemma Scope targets the Gemma 2 model family, specifically the 2B and 9B versions named in the announcement. DeepMind trained sparse autoencoders at every layer and sublayer output of these models. The result, according to the company, is more than 400 sparse autoencoders containing more than 30 million learned features in total.

The scale of the training effort was substantial. DeepMind reports that the work used roughly 15 percent of the training compute that went into Gemma 2 9B and required saving about 20 pebibytes of activations during the process. The sparse autoencoders span the main computational sites inside the models, giving researchers coverage across the network rather than at a single point.

Accessing the Tools

The sparse autoencoders are available to download from Hugging Face at the gemma-scope repository. DeepMind also points researchers to an interactive demo built with Neuronpedia, where features can be explored without writing code, and to a Google Colab notebook that walks through the basics of working with the autoencoders programmatically. A technical report accompanies the release for those who want the underlying methodology.

Why Open Interpretability Tools Matter

DeepMind frames Gemma Scope as a contribution to the safety community, with the stated goal of helping researchers shed light on how language models behave internally. Open access lowers the barrier for academic groups and smaller teams that lack the infrastructure to train sparse autoencoders at this scale on their own. By releasing the autoencoders, the demo, and a tutorial together, the project aims to make mechanistic interpretability research more reproducible and easier to build on, which becomes more relevant as models are deployed in more settings and their internal behavior carries more weight.