leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx

This model was converted to MLX format from poolside/Laguna-XS-2.1 using BaseQuant_XL 6/8-bit mixed quantization. Text-only (no vision encoder). The unquantized BF16 base was used as the source.

BaseQuant_XL keeps the most routing-critical layers in full bf16 precision — the MoE router (mlp.gate.proj), the shared expert, and lm_head — while applying 8-bit to embeddings and all attention layers, and 6-bit to the 256 routed experts (switch_mlp). This is the recipe that consistently produces more decisive reasoning chains and fewer degenerate self-verification loops across our benchmark suite.

Laguna XS 2.1 is a 33B-parameter MoE model (3B active per token) built for agentic coding and long-horizon software engineering. It features a novel architecture with per-head gating, mixed sliding-window/global attention (3:1 ratio), and dual RoPE configurations. Native interleaved reasoning with preserved thinking is supported via enable_thinking.

Comparison: The XL recipe keeps routing at bf16 vs the AtomicChat uniform 6-bit which quantizes everything to 6-bit (router at 8-bit). Benchmark comparison below shows whether XL dominates on this architecture.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Intelligence Benchmarks

Run under identical conditions on the same hardware. Instruct mode (thinking disabled), same sample sizes as our Gemma-4 26B 6bit-XL benchmark suite.

Instruct-mode benchmarks

Benchmark	Sample	Laguna-XS 6bit-XL (this)	Laguna 6-bit uniform	Gemma-4 6bit-XL	Qwen3.6-35B 6bit-XL
MMLU	50/14042	62.0%	66.0%	76.0%	64.0%
MMLU_PRO	50/12032	74.0%	64.0%	80.0%	64.0%
ARC_CHALLENGE	50/1172	72.0%	72.0%	90.0%	90.0%
HUMANEVAL	50/164	92.0%	92.0%	96.7%	78.0%
MBPP	50/500	72.0%	72.0%	80.0%	78.0%

Poolside published results (BF16, for reference)

These are poolside's published scores for the full-precision base model using their agent harness (Harbor Framework, max 500 steps, sandboxed execution).

Benchmark	Laguna XS 2.1
SWE-bench Verified	70.9%
SWE-bench Multilingual	63.1%
SWE-Bench Pro	47.6%
Terminal-Bench 2.0	37.5%

Use with mlx

Requires mlx-vlm 0.6.3+ (not mlx-lm — Laguna's architecture is only in mlx-vlm):

pip install -U mlx-vlm

# Generate (reasoning on by default)
python -m mlx_vlm generate \
    --model leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx \
    --prompt "Write a Python retry wrapper with exponential backoff." \
    --max-tokens 512 --temperature 1.0 --top-k 20 --top-p 1.0

# OpenAI-compatible server
python -m mlx_vlm server \
    --model leonsarmiento/Laguna-XS-2.1-6bit-XL-mlx \
    --host 0.0.0.0 --port 8080

LM Studio / oMLX

Load the model directory directly. Reasoning is native and on by default — use enable_thinking to control it.

BaseQuant_XL Quantization Strategy

Layer	Bits	Rationale
lm_head	bf16	Output projection — full precision for token selection
mlp.gate.proj (router)	bf16	Routing decisions — errors here cascade through all tokens
mlp.shared_expert	bf16	Processes every token — not redundant like routed experts
embed_tokens	8-bit	Near-lossless embedding lookup
self_attn (q/k/v/o/g_proj, q/k_norm)	8-bit	Every-token attention layers — near-lossless
Dense MLP (layer 0)	8-bit	Small layer, not worth compressing
switch_mlp (256 routed experts)	6-bit	Bulk parameters — natural redundancy tolerates lower precision

Quantization Details

Property	Value
Bits per weight	6.700
Total size	~26 GB (6 shards)
Group size	64
Method	Custom quant_predicate via mlx_vlm
Source	BF16 unquantized weights (poolside/Laguna-XS-2.1)
tie_word_embeddings	false (separate lm_head present)

Architecture

Property	Value
Architecture	LagunaForCausalLM (custom)
Model type	laguna
Total parameters	33B
Active parameters	3B per token
Layers	40 (10 global attention + 30 sliding-window, 3:1 ratio)
Experts	256 routed + 1 shared, 8 active per token
Gating	Per-head sigmoid gating
Sliding window	512 tokens
Attention heads	Variable — 48 on global layers, 64 on sliding layers
GQA	8 KV heads, head_dim 128
RoPE	Dual config — YaRN (theta=500K) for global, default (theta=10K) for sliding
Context window	262,144 tokens (256K)
Vocab size	100,352
Optimizer	Muon
Modality	Text-only
KV cache	FP8 quantized (runtime feature)

Recommended Inference Parameters

Per poolside's benchmark settings:

Parameter	Value
temperature	1.0
top_k	20
top_p	1.0

Reasoning Control

Setting	Default	Notes
enable_thinking	true	Set to false to disable interleaved reasoning
Preserved thinking	Automatic	See below

Preserved Thinking

Laguna XS 2.1 is designed to work with preserved thinking — reasoning_content from prior assistant messages is kept in the conversation history so the model can reference its own earlier reasoning across turns. This is critical for multi-step agentic workflows where the model reasons between tool calls.

How it works: Unlike Qwen3.6 (which needs preserve_thinking=true) or Gemma 4 (which only preserves thinking for tool-calling turns), Laguna's template renders prior reasoning unconditionally when enable_thinking=true — no extra flags needed. The template extracts reasoning_content (or reasoning) from each assistant message and renders it inside <think>...</think> tags.

LM Studio / oMLX: Preserved thinking works automatically — no special configuration beyond enable_thinking=true. The server passes reasoning_content from prior assistant messages back to the template, which renders them into the prompt context.

Why it matters: Per poolside's documentation: "Reasoning may not be generated in follow-up steps if prior thinking blocks are dropped (i.e., thinking is not preserved) when messages are reconstructed over multiple steps." Without preserved thinking, the model loses its reasoning context and may produce inconsistent or degraded responses in multi-turn conversations.

License

OpenMDW-1.1 — permits free use, modification, and redistribution with attribution. This is an unofficial community quantization, not endorsed by poolside. Original LICENSE.md and notices of origin retained.