OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
Paper
• 2308.13137 • Published
• 19
4-bit OmniQuant quantized version of Mistral-7B-Instruct-v0.2.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support