smoltokenizer / tokenizer.json
eliebak's picture
eliebak HF Staff
commit of fast smoltokenizer
fddfcaa
Download
History Blame Contribute Delete
6.17 kB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
f4f0f0a5b66f6d171b005dfbbddf92f4e798fcf389472ff90e52002290aeb86c
Size of remote file:
6.17 kB
·
SHA256:
a5078c05e94e5980118e913ed49426e7b1cdf93dcc6ccefa7967fafd2f7e255e

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.