Hugging Face's logo

RWKV
/
v5-Eagle-7B-pth

Model card Files
xet
Community
7

size of the vocabulary

#1
by yuimo - opened
Discussion
yuimo

what is the size of the vocabulary? and how to train the tokenizer? BPE or wordpiece?
BlinkDL
 RWKV org

65536
It's a greedy tokenizer
see https://github.com/BlinkDL/ChatRWKV/tree/main/tokenizer

Sign up or log in to comment