Update README.md
Browse files
README.md
CHANGED
|
@@ -14,11 +14,9 @@ This is a landing page for **Gemma Scope**, a comprehensive, open suite of spars
|
|
| 14 |
- https://huggingface.co/google/gemma-scope-2b-pt-res
|
| 15 |
- https://huggingface.co/google/gemma-scope-2b-pt-mlp
|
| 16 |
- https://huggingface.co/google/gemma-scope-2b-pt-att
|
| 17 |
-
- https://huggingface.co/google/gemma-scope-2b-pt-transcoders
|
| 18 |
- https://huggingface.co/google/gemma-scope-9b-pt-res
|
| 19 |
- https://huggingface.co/google/gemma-scope-9b-pt-mlp
|
| 20 |
- https://huggingface.co/google/gemma-scope-9b-pt-att
|
| 21 |
-
- https://huggingface.co/google/gemma-scope-9b-it-res
|
| 22 |
- https://huggingface.co/google/gemma-scope-27b-pt-res
|
| 23 |
|
| 24 |
# Key links:
|
|
@@ -35,7 +33,7 @@ The full list of SAEs we trained at which sites and layers are linked from the f
|
|
| 35 |
|
| 36 |
| <big>Gemma 2 Model</big> | <big>SAE Width</big> | <big>Attention</big> | <big>MLP</big> | <big>Residual</big> | <big>Tokens</big> |
|
| 37 |
|---------------|-----------|-----------|-----|----------|----------|
|
| 38 |
-
| 2.6B PT<br>(26 layers) | 2^14 ≈ 16.4K | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp)
|
| 39 |
| | 2^15 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_32k/)} | 8B |
|
| 40 |
| | 2^16 | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 8B |
|
| 41 |
| | 2^17 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_131k/)} | 8B |
|
|
@@ -50,5 +48,3 @@ The full list of SAEs we trained at which sites and layers are linked from the f
|
|
| 50 |
| | 2^19 | | | {[20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_524k/)} | 8B |
|
| 51 |
| | 2^20 | | | {[9](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_9/width_1m/), [20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_1m/), [31](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_31/width_1m/)} | 16B |
|
| 52 |
| 27B PT<br>(46 layers) | 2^17 | | | {[10](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_10/width_131k/), [22](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_22/width_131k/), [34](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_34/width_131k/)} | 8B |
|
| 53 |
-
| 9B IT<br>(42 layers) | 2^14 | | | {[9](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_9/width_16k/), [20](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_20/width_16k/), [31](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_31/width_16k/)} | 4B |
|
| 54 |
-
| | 2^17 | | | {[9](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_9/width_131k/), [20](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_20/width_131k/), [31](https://huggingface.co/google/gemma-scope-9b-it-res/tree/main/layer_31/width_131k/)} | 8B |
|
|
|
|
| 14 |
- https://huggingface.co/google/gemma-scope-2b-pt-res
|
| 15 |
- https://huggingface.co/google/gemma-scope-2b-pt-mlp
|
| 16 |
- https://huggingface.co/google/gemma-scope-2b-pt-att
|
|
|
|
| 17 |
- https://huggingface.co/google/gemma-scope-9b-pt-res
|
| 18 |
- https://huggingface.co/google/gemma-scope-9b-pt-mlp
|
| 19 |
- https://huggingface.co/google/gemma-scope-9b-pt-att
|
|
|
|
| 20 |
- https://huggingface.co/google/gemma-scope-27b-pt-res
|
| 21 |
|
| 22 |
# Key links:
|
|
|
|
| 33 |
|
| 34 |
| <big>Gemma 2 Model</big> | <big>SAE Width</big> | <big>Attention</big> | <big>MLP</big> | <big>Residual</big> | <big>Tokens</big> |
|
| 35 |
|---------------|-----------|-----------|-----|----------|----------|
|
| 36 |
+
| 2.6B PT<br>(26 layers) | 2^14 ≈ 16.4K | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 4B |
|
| 37 |
| | 2^15 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_32k/)} | 8B |
|
| 38 |
| | 2^16 | [All](https://huggingface.co/google/gemma-scope-2b-pt-att) | [All](https://huggingface.co/google/gemma-scope-2b-pt-mlp) | [All](https://huggingface.co/google/gemma-scope-2b-pt-res) | 8B |
|
| 39 |
| | 2^17 | | | {[12](https://huggingface.co/google/gemma-scope-2b-pt-res/tree/main/layer_12/width_131k/)} | 8B |
|
|
|
|
| 48 |
| | 2^19 | | | {[20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_524k/)} | 8B |
|
| 49 |
| | 2^20 | | | {[9](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_9/width_1m/), [20](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_20/width_1m/), [31](https://huggingface.co/google/gemma-scope-9b-pt-res/tree/main/layer_31/width_1m/)} | 16B |
|
| 50 |
| 27B PT<br>(46 layers) | 2^17 | | | {[10](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_10/width_131k/), [22](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_22/width_131k/), [34](https://huggingface.co/google/gemma-scope-27b-pt-res/tree/main/layer_34/width_131k/)} | 8B |
|
|
|
|
|
|