Instructions to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS # Run inference directly in the terminal: llama cli -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS # Run inference directly in the terminal: ./build/bin/llama cli -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Use Docker
docker model run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
- Llama new
- LM Studio
- Jan
- vLLM
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
- Ollama
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with Ollama:
ollama run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
- Unsloth Desktop
- Pi
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with Docker Model Runner:
docker model run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
- Lemonade
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Run and chat with the model
lemonade run user.Qwen-4B-Bugio-Vision-Thinking-GGUF-IQ3_XXS
List all available models
lemonade list
- Hermes Agent
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen-4B Bugio: Qwen3.8-distilled 4B model with faster thinking and vision for 4 GB GPUs (GGUF)
Qwen-4B Bugio (Bugio-4B) is a tiny, experimental open-weights model for Ollama and llama.cpp. It is a 2.4 GB build of Qwen3.8-4B-Distill, a Qwen3.5-4B fine-tuned on reasoning traces from Qwen3.8. It thinks step by step with shorter reasoning, reads images and screenshots, and calls tools, all offline. It runs on a 4 GB GPU from 2014.
ollama run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Experimental. Bugio is the reasoning sibling of Sagui-4B, built with the same recipe and the same size. On our reasoning questions it used about 1.9ร fewer thinking tokens than Sagui, but it is weaker at TypeScript. For coding, use Sagui.
Bugio and Sagui are free. Donations pay for the compute behind the next tiny models: โ PayPal
Bugio or Sagui?
Both are 2.4 GB, use the same compression recipe, and were measured on the same 4 GB GTX 970.
| Bugio-4B (this model) | Sagui-4B | |
|---|---|---|
| Source weights | Qwen3.8-4B-Distill (Qwen3.5-4B tuned on Qwen3.8 traces) | Qwen3.5-4B |
| Average thinking length, 5 reasoning questions | 1447 tokens, 101 s | 2786 tokens, 198 s |
| Correct answers on those questions | 5/5 | 5/5 |
| HumanEval-TS pass@1 | 36.5% | 44.0% |
| MBPP-TS pass@1 | 46.2% | 54.6% |
| Vision, thinking, tool calling | โ | โ |
| Total size with vision | 2.38 GB | 2.38 GB |
| Best for | Quick reasoning, Q&A, chat, screenshots | TypeScript and code |
A bugio is a howler monkey, native to Brazil's Atlantic Forest. Its call can be heard kilometres away. Small, loud and quick to answer.
See it working
These are unedited outputs from this exact GGUF. Prompts and raw responses are in examples/transcripts.json, and scripts/run_demos.py reproduces them. Sampling isn't seeded, so your wording will vary.
Thinking mode
Prompt: A train leaves at 14:35 and arrives at 17:10. How long is the trip in minutes?
The trip takes **155 minutes**.
**Breakdown:**
- From 14:35 to 17:10 is **2 hours and 35 minutes**.
- 2 hours = 120 minutes
- 120 + 35 = **155 minutes**
338 tokens, 24.3 s on a GTX 970, reasoning included. Sagui-4B spent 3,003 tokens and 212 s on the same question in our comparison run.
UI screenshot โ working React component
Thinking on, 1545 tokens, 106.9 s on a GTX 970. Only the import line was removed to render it in the browser. It got the text, fields, button and link right, but missed the placeholders and centered the labels.
Screenshot โ bug fix
Prompt: This is a screenshot from my editor. Explain the error in one sentence, then give the corrected line 4.
The error occurs because the
reducecallback's second parameteriis the accumulator object from the previous iteration, which is anumber, but the code tries to accessi.qtywithout a type assertion, causing a type mismatch.
return items.reduce((sum, i) => sum + i.price * i.qty, 0)
The fix is to add the type assertion : number to the second parameter i of the reduce callback, so TypeScript correctly infers that i is an Item object.
120 tokens, 30.9 s on a GTX 970. It read the code from the image and the corrected line is right. The explanation is wrong: the real cause is the string initial value "0". Sagui explained this one correctly.
Tool calling
Asked "What's the weather in Lisbon? Use the tool.", Bugio called get_weather with {"city": "Lisbon"}. Given the tool result, it answered: "The weather in Lisbon is currently 21ยฐC and sunny."
Run it
Ollama, one command
ollama run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Ollama, from the files
hf download sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF --local-dir bugio
cd bugio
ollama create bugio-4b -f Modelfile
ollama run bugio-4b "What is in this image? ./photo.png"
Control thinking per chat with --think or --think=false, or in the API:
curl http://localhost:11434/api/chat -d '{
"model": "bugio-4b",
"think": true,
"messages": [{"role": "user", "content": "A shirt costs $40 after a 20% discount. What was the original price?"}]
}'
llama.cpp
llama-server -m Bugio-4B-IQ3_XXS.gguf --mmproj mmproj-Bugio-4B-F16.gguf -ngl 99 -c 16384 --temp 0.6 --top-p 0.95 --top-k 20
Don't use greedy decoding (temperature 0): the source model can loop. The Modelfile already sets the sampling recommended by its authors.
Tip for 4 GB GPUs
Ollama can split the model between CPU and GPU to be safe. On our GTX 970 that gave 15.5 tok/s. Forcing every layer onto the GPU gave 23.2 tok/s:
PARAMETER num_gpu 99
Add that line to the Modelfile and run ollama create again. If you run out of memory, remove it or lower num_ctx.
Files
| File | Size | Needed for |
|---|---|---|
Bugio-4B-IQ3_XXS.gguf |
1.71 GB | Everything |
mmproj-Bugio-4B-F16.gguf |
0.67 GB | Images only |
Modelfile, template, params |
small | Ollama setup |
Benchmarks
Reasoning: thinking length
Five short arithmetic and logic questions with thinking on, through Ollama with the Modelfile settings. Questions, answers and timings are in benchmarks/thinking.json, and scripts/think_compare.py reproduces them. Five questions show the size of the gap, not a precise accuracy figure.
| Bugio-4B | Sagui-4B | |
|---|---|---|
| Correct | 5/5 | 5/5 |
| Average tokens per answer | 1447 | 2786 |
| Average time per answer, GTX 970 | 101 s | 198 s |
Code: TypeScript
TypeScript code generation on MultiPL-E with greedy decoding. A solution passes only if it type-checks with tsc and passes its tests in Node.
| Benchmark | Bugio-4B IQ3_XXS | Sagui-4B IQ3_XXS |
|---|---|---|
| HumanEval-TS | 58/159 (36.5%) | 70/159 (44.0%) |
| MBPP-TS | 180/390 (46.2%) | 213/390 (54.6%) |
| Weighted average | 43.4% | 51.5% |
These measure raw completion without thinking mode. Most of Bugio's extra failures are tsc type errors. Completions, failures and the script are in benchmarks/.
Measured on an Intel i5-10400, 16 GB RAM and an NVIDIA GTX 970 (4 GB). llama.cpp benchmark speed: 21.3 tok/s. CPU only through Ollama: 9.1 tok/s.
Limitations
- Experimental and weaker at code. It scores about 8.1 points below Sagui-4B on TypeScript. In our
groupBydemo it reused the parameter namekeyfor a local variable, which doesn't compile. Use Sagui for code. - Unverified upstream data. Qwen3.8-4B-Distill was trained on about 45,000 reasoning traces that its authors describe but haven't published. The authors also report a lower GSM8K score than the base Qwen3.5-4B and a higher MMLU score.
- Latin scripts only. Removing other alphabets is part of how the model got this small. Use the original Qwen3.5-4B for Chinese, Japanese, Korean, Russian, Arabic and similar languages.
- Heavy quantization. IQ3_XXS trades accuracy for size, and it was calibrated on TypeScript.
- Vision is inherited, not tuned. The projector is Qwen3.5-4B's, unmodified. It read screenshots well in our tests but explained one of them wrongly. It was not tested on photos at scale.
- It makes mistakes. Review its answers and code, and don't use it for medical, legal or safety-critical decisions.
FAQ
Is there a Qwen3.8 4B model? Not an official one. Qwen released Qwen3.8 as a 27B dense model and a large MoE. Bugio is built on Qwen3.8-4B-Distill, a community fine-tune of Qwen3.5-4B on Qwen3.8 reasoning traces. It is not Qwen3.8 itself.
Can I run Qwen3.8 reasoning on a 4 GB GPU? This is the closest option we know of. Bugio fits in 2.4 GB with vision, and every result here comes from a 4 GB GTX 970.
Is this Qwen 4? No. "Qwen-4B" means a Qwen-family model with 4 billion parameters.
Bugio or Sagui? Bugio answered our reasoning questions about twice as fast, with shorter thinking. Some questions still trigger long reasoning. Sagui is clearly better at TypeScript. Both see images and call tools.
Does it run without a GPU? Yes, on CPU, at 9.1 tok/s on an i5-10400.
Was it retrained? Not by us. It has the same weights as Qwen3.8-4B-Distill, with a trimmed vocabulary and heavier quantization. See the recipe.
Can I use it commercially? Yes, under Apache 2.0. Keep the LICENSE and NOTICE files.
Em portuguรชs
Qwen-4B Bugio รฉ uma IA local experimental de 2,4 GB, ajustada com raciocรญnios do Qwen3.8, que pensa e enxerga. Roda offline com Ollama ou llama.cpp, inclusive em GPU de 4 GB. ร o irmรฃo do Sagui-4B, com a mesma receita e o mesmo tamanho. Nas nossas perguntas de raciocรญnio, pensou cerca de metade do que o Sagui pensa antes de responder, mas รฉ mais fraco em TypeScript. Para programar, use o Sagui.
ollama run hf.co/sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF:IQ3_XXS
Nรฃo รฉ um modelo oficial do Qwen3.8: รฉ uma versรฃo compactada do Qwen3.8-4B-Distill, um Qwen3.5-4B ajustado com raciocรญnios gerados pelo Qwen3.8.
Gostou? Apoie o projeto. As doaรงรตes pagam a computaรงรฃo dos prรณximos modelos pequenos e gratuitos.
Support the project
Bugio and Sagui are free and stay free. Donations pay for the compute and the time behind the next tiny models.
No money to spare? A โค๏ธ on this page and a post about what you built help just as much.
Credits and license
Bugio-4B is a modified version of Qwen3.8-4B-Distill by empero-ai, itself a fine-tune of Qwen3.5-4B by the Qwen team. It was quantized with llama.cpp and is licensed under Apache 2.0. The changes are listed in NOTICE. This is an independent community project, not affiliated with or endorsed by the Qwen team, Alibaba Cloud or empero-ai.
- Downloads last month
- 282
3-bit
Model tree for sysrobson/Qwen-4B-Bugio-Vision-Thinking-GGUF
Evaluation results
- pass@1 on HumanEval-TS (MultiPL-E)test set self-reported36.500
- pass@1 on MBPP-TS (MultiPL-E)test set self-reported46.200




