SAE for Codegemma

🤗Transformers
1

I recently came across Gemmascope models which uses SAEs to interpret the information captured by layers of Gemma models. But the pretrained SAEs only contain the features for general Gemma model and not for the code generation model. Is there a pretrained SAE for Codegemma that I can use? If not, I want to train it. What dataset can I use?

1 Like
2

It doesn’t seem to exist at the moment.
Training seems possible, but it looks pretty tough…

3

Why did it get flagged?

1 Like
4

Basically, everyone tends to get flagged. Even HF staff get flagged, apparently. Well, that’s just how this forum works.:sweat_smile: