Instructions to use majentik/garden with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use majentik/garden with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="majentik/garden")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("majentik/garden", device_map="auto")

MLX

How to use majentik/garden with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm
# if on a CUDA device, also pip install mlx[cuda]

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("majentik/garden")

prompt = "Once upon a time in"
text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use majentik/garden with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "majentik/garden"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "majentik/garden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/majentik/garden

SGLang

How to use majentik/garden with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "majentik/garden" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "majentik/garden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "majentik/garden" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "majentik/garden",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

MLX LM

How to use majentik/garden with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Generate some text
mlx_lm.generate --model "majentik/garden" --prompt "Once upon a time"

Docker Model Runner
How to use majentik/garden with Docker Model Runner:
```
docker model run hf.co/majentik/garden
```
Atomic Chat

KV-cache quantization needs no fork (2026): upstream llama.cpp / Ollama cover it natively. Use -ctk q8_0 -ctv q8_0 (~~half KV memory, perplexity +0.002–0.05) or -ctk q4_0 -ctv q4_0 (~~quarter memory, ≈7.6% perplexity increase). In Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1. Keep K and V types symmetric to stay on the fused Flash-Attention path. Since April 2026 mainline llama.cpp also applies Hadamard rotation to KV activations (PR #21038), which greatly improves low-bit KV quality (opt-out: LLAMA_ATTN_ROT_DISABLE=1).

majentik — Model Garden

Quantized open-weight models for Apple Silicon and llama.cpp, released only after a coherence smoke gate passes on real hardware. Every repo keeps the upstream tokenizer, architecture and license; the only thing we change is how the weights are stored.

402 model repositories · 14 datasets · 13 collections · snapshot 2026-09-16

How releases are made

Pin upstream. Every lane pins the exact upstream revision it quantized from and refuses to run if the source drifts.
Quantize with standard tooling. mlx-lm / mlx-vlm / mlx-audio for MLX, llama.cpp for GGUF. Group size 32 for ≤4-bit MLX tiers. We never publish a tier at or above the source's bits-per-weight.
Smoke gate. Each tier must produce coherent greedy output on a fixed prompt set before upload. Tiers that fail are held back and listed as such on the family card instead of being shipped — expect gaps in the tier ladder for small or fragile models.
Evidence. Gate records are published in majentik/garden-quant-bench. Repos in the Verified MLX releases collection link their evidence directly.

Lanes

Lane	Repos	Runtime	Tiers
MLX	264	`mlx-lm`, `mlx-vlm`, `mlx-audio` on Apple Silicon	2 / 3 / 4 / 5 / 6 / 8-bit, MXFP4, bf16 references, LoRA adapters
GGUF	129	`llama.cpp` (one-shot: `llama-completion -no-cnv`), Ollama, LM Studio	Q2_K … Q8_0, IQ4_XS, MXFP4
FP8 / ONNX / other	7	vLLM, onnxruntime	Family-specific

Repo naming is unbranded: majentik/<Model>-MLX-<tier> and majentik/<Model>-GGUF-<QT>.

Families

Counts are repos on the Hub as of the snapshot date.

Family	Repos	Notes
Gemma 4	91	E2B / E4B / 12B / 26B-A4B / 31B, base + instruct, MLX + GGUF (collection)
Nemotron 3 / 3.5 / Cascade 2	55	Nano 4B, Nano 30B-A3B, Nano Omni 30B (audio+vision), Super 120B-A12B, Lightning 30B, Cascade 2 (collection)
Qwen 3.6	25	27B dense + 35B-A3B MoE, full vision tower (collection)
Qwen 3.5 / 3.8	22	27B, 122B-A10B, 397B-A17B; Qwen3.8-27B (collection)
Qwen agents & coders	14	Qwen3-Coder-Next, Qwen-AgentWorld-35B-A3B
MiniCPM 5	25	1B base / instruct / SFT edge models
Ornith 1.0 / 1.5	16	9B, 35B, 35B-A3B reasoning MoE
Embeddings	29	Qwen3-Embedding 0.6B/4B/8B, UEmbed 2B/4B/9B, nomic-embed-text-v2-moe, form-embed (UEmbed, ONNX)
Speech — ASR & TTS	37	MERaLiON-3 3B/10B, Qwen3-ASR, Voxtral Mini/Realtime/TTS, MOSS-Transcribe, cohere-transcribe-arabic, Kokoro, fishaudio-s2-pro, Audio8-TTS, Gemma-4-E4B MERaLiON speech LoRAs (ASR on Apple Silicon, TTS on Apple Silicon, MERaLiON)
gpt-oss	10	20B + 120B rebuilt GGUFs / MLX (collection)
LFM 2.5	10	2.6B dense, 8B-A1B MoE
harrier-oss	10	270M / 0.6B / 27B
Mistral	11	Medium 3.5 128B, Small 4 119B, Leanstral
Vision & agents	29	Muse Glimmer 30B (collection), UI-Mate 27B, BigBang v1, Unlimited-OCR, Qwen-Image-Bench, GELab-Zero (OCR & DocAI)
Other LLMs	16	KAT-Coder V2.5, Shieldstral 3B, MiniMax M2.7, DeepSeek-V4-Flash, Qwen2.5-1.5B DWQ reference, cga-gpt

Datasets

Calibration and evaluation sets used by the lanes, mirrored so results are reproducible: ultrachat-calib, c4-calib, ultra-fineweb-calib, ultradata-{sft,math}-calib, tulu-3-sft-mixture, wikitext-2-ppl, gsm8k, ifeval, fleurs-ar-asr, WildASR, SASRBench-v1, magpie-reasoning-qwen25-7b, and the gate ledger garden-quant-bench.

Pick a starting point

MacBook, 16 GB → Gemma 4 E4B-it MLX 4-bit, or LFM2.5-8B-A1B MLX 4-bit.
MacBook, 32–64 GB → Qwen3.6-35B-A3B MLX 4-bit, Nemotron-3-Nano-30B-A3B MLX 4-bit.
Mac Studio, 128 GB+ → Qwen3.5-122B-A10B MLX 4-bit, Mistral-Medium-3.5-128B MLX 4-bit, gpt-oss-120b.
Speech on Apple Silicon → MERaLiON-3-3B-ASR MLX 8-bit (Singapore English / Malay / Chinese / Tamil), Qwen3-ASR MLX.
Long context on llama.cpp → any GGUF tier plus -ctk q8_0 -ctv q8_0 (see tip above).

What is not in each repo

Training data. Quantization-only releases; the base model's training data and disclaimers are upstream's.
Benchmarks for every axis. We publish the smoke gate result, and where measured, WikiText-2 perplexity or family-specific evals (KL vs. bf16, WER for ASR). Missing an axis you need? Open a discussion on the repo.

About the RotorQuant / TurboQuant labels

Older repos in this org carry RotorQuant or TurboQuant in their names. These are historical release labels, not distinct quantization algorithms: for any given tier both labelled repos hold byte-identical weights produced with the standard MLX / llama.cpp quantizers, and no brand-specific speedup is claimed or measured. New releases are unbranded. The associated llama.cpp fork is unmaintained; use upstream KV-cache quantization as described in the tip at the top.

Who we are

majentik publishes these to keep our own fleet running cheaply on commodity Apple hardware and to close the gap between a research release and "can I actually run this tonight". Issues, quant requests and benchmark PRs are welcome via the Community tab on the closest repo.

Versioning

Each repo tracks upstream@base-model-revision × quant-lane. When upstream ships a new base revision we re-run the lane and bump the repo. Card-only changes do not bump the version.

License

Each repo inherits the base model's license, not this organization-level license. Check the license field in the repository's card before deploying.

Downloads last month: -; Downloads are not tracked for this model. How to track