Quantised version

#12

by nikhilfande - opened Nov 8, 2025

Nov 8, 2025

Is there any plan to release quantised version of 8B model such that it could be fit in single T4 machine? or you suggest to use 4B model in such case ?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment