Bonsai compresses a 27-billion-parameter model to 1.125 bits per weight — a
~3.9 GB footprint that runs at ~44 tok/s on an M5 Pro and ~11 tok/s on an iPhone 17 Pro Max.
A simulated preview of the model's thinking-mode responses. Real generation runs on your own
Apple Silicon — see Run locally.
bonsai@mlx — mlx_lm.generate
SIMULATED PREVIEW
🌳
Hi! I'm Bonsai-27B running at 1.125 bits. Ask me anything, or tap a suggestion below.
Explain 1-bit quantization simplyWrite a haiku about small modelsWhy is it called Bonsai?How fast is it on iPhone?
⚠️ Responses here are canned illustrations of the model's style — this Space serves static HTML on x86 Linux, and MLX only runs on Apple Silicon, so no live inference happens in the browser.
Quality vs. size
Tiny footprint, most of the smarts
Averages across 15 thinking-mode benchmarks. Bonsai keeps ~89.5% of the original model's
capability while shrinking the weights by more than 14×.
Quality retained vs. FP1689.5%
Avg score · 15 thinking benchmarks76.11 / 100
Size reduction vs. FP16 (54 GB → 3.9 GB)14.2× smaller
Intelligence density (0.530 / GB)2.7× densest conventional
Under the hood
How it's built
Derived from Qwen3.6-27B — a hybrid-attention causal LM — then binarized end-to-end
across embeddings, attention & MLP projections, and the LM head.
🧠
Hybrid attention
Mostly linear attention with periodic full-attention layers, plus SwiGLU MLP, RoPE, and RMSNorm.
linear / full75% / 25%
🔩
End-to-end 1-bit
Binary weights with group-wise scaling everywhere; the optional vision tower stays in compact 4-bit.
bits / weight1.125
⚡
Fast decoding
Ships a DSpark speculative-decoding drafter for a 1.37× speedup on CUDA; ternary variant for quality.
M5 Pro / iPhone44 / 11 tok/s
Get started
Run it locally
Bonsai is an MLX model — it runs natively on Apple Silicon. Pick your stack: