mlx-community/mini-coder-4b-OptiQ-4bit

A 4-bit mixed-precision MLX quant produced by mlx-optiq — the sensitivity-aware quantization toolkit for Apple Silicon. It beats stock uniform 4-bit on the six-metric Capability Score (45.83 vs 42.74, +3.09) and is ~2.2x more faithful to the original bf16 weights (KL 0.06 vs 0.13).

A 4-bit mixed-precision MLX quant of ricdomolm/mini-coder-4b (a code/agent model fine-tuned from Qwen3-4B-Instruct-2507). Per-layer bit-widths come from a KL-divergence sensitivity pass on a six-domain calibration mix (prose · reasoning · code · agent · tool-call · constraint-bearing instructions). Sensitive layers go to 8-bit; robust ones stay at 4-bit.

Quantization details

Property	Value
Predominant precision	4-bit
Layers at 8-bit (sensitive)	123
Layers at 4-bit (robust)	129
Total quantized layers	252
Achieved bits per weight	5.16
Group size	64
Calibration mix	six-domain mix (40 samples × 6 domains)
Reference for sensitivity	bf16

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average. The few sensitive layers held at 8-bit make this build +0.7 GB larger than a stock uniform-4-bit quant (2.8 GB vs 2.1 GB) while recovering quality the uniform quant loses.

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/mini-coder-4b-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Write a Python function to check if a string is a palindrome.",
    max_tokens=512,
)

It's a code/agent model — best paired with a lightweight agent harness / loop. For more (mixed-precision KV-cache serving, sensitivity-aware LoRA fine-tuning, OpenAI + Anthropic-compatible inference server with structured/JSON output and server-side tool-call healing, hot-swap mounted adapters), install mlx-optiq:

pip install mlx-optiq

Benchmarks

Six-metric Capability Score (mean of MMLU + GSM8K + IFEval + BFCL + HumanEval + HashHop). Apples-to-apples comparison against stock uniform 4-bit:

Metric	OptiQ	Uniform 4-bit	Δ
MMLU (5-shot, 1000 samples)	69.9%	68.1%	+1.8
GSM8K (1000 samples, CoT)	59.6%	48.1%	+11.5
IFEval (full set, strict)	36.6%	38.4%	-1.8
BFCL-V3 simple (200 calls)	56.5%	47.5%	+9.0
HumanEval (164 problems, pass@1)	52.4%	54.3%	-1.8
HashHop (long-context retrieval)	0.0%	0.0%	+0.0
Capability Score (mean of 6)	45.83	42.74	+3.09
KL vs bf16 (mean / p95)	0.0571 / 0.2336	0.1277 / 0.4872	—
On-disk size	2.8 GB	2.1 GB	+0.7

OptiQ's clearest wins are GSM8K (+11.5) and tool-calling (BFCL +9.0), plus markedly better fidelity to the original weights. IFEval and HumanEval are within noise (~±7 pp at these sample sizes). HashHop is 0% for both quants: a genuine model-capability gap — the model has no long-context multi-hop retrieval ability (verified by inspecting outputs, it reasons in prose instead of walking the hash chain) — not a quantization artifact, which is why OptiQ and uniform-4 match there.

Every metric gets one equal vote; disk size is an honest second axis, not folded into the score. See the eval-framework writeup.

Links

• Project website: mlx-optiq.com
• Docs: mlx-optiq.com/docs
• PyPI: pypi.org/project/mlx-optiq
• Eval framework: mlx-optiq.com/blog/eval-framework
• Base model: ricdomolm/mini-coder-4b

License

MIT (inherits from the base model).