togethercomputer
/

StripedHyena-Nous-7B

Text Generation

Model card Files Files and versions

StripedHyena-Nous-7B / tokenizer_config.json

Zymrael's picture

fix tokenizer for transformers >= 4.38.2

d45c7c0 verified almost 2 years ago

history blame contribute delete

142 Bytes

	{
	"unk_token": "</s>",
	"bos_token": "<s>",
	"eos_token": "</s>",
	"from_slow": false,
	"tokenizer_class": "LlamaTokenizer"
	}