---
language:
- en
- zh
- de
- fr
license: apache-2.0
base_model: Qwen/Qwen3.6-27B
tags:
- quantization
- 1-bit
- fabq-rc
- fisher-adaptive
- qwen
- qwen3_5
- custom-format
- non-llamacpp
- non-transformers
library_name: custom
pipeline_tag: text-generation
extra_metadata:
  quantization_label:
    - FABQ-RC
  format: custom-pth
  effective_bpw: 1.18
  loader: see https://huggingface.co/toxzak/fabq-rc (Main-FABQ-RC-Notebook.ipynb)
---

# Qwen3.6-27B-FABQ-RC

## Model Description

**FABQ-RC** (Fisher-Adaptive Binary Quantization with Residual Codebooks) is a 1-bit
quantization method for large language models that adapts per layer rather than using a
fixed blocksize. This quantization of **Qwen3.6-27B** achieves ~1.18 bits per parameter
while preserving quality through four key innovations.

| Property | Value |
|---|---|
| **Base Model** | [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) |
| **Quantization Method** | FABQ-RC |
| **Format** | Custom PyTorch state dict (`.pth`) |
| **Bits per Parameter** | ~1.18 bpw |
| **Precision Allocation** | Top 5% channels → int8, 95% → binary ±1 |
| **Blocksize** | Adaptive per-layer (selected from {16, 32, 64, 128, 256}) |
| **Residual Codebook** | 256-entry k-means centroids, per-layer |
| **Architecture** | Qwen3.5 hybrid (DeltaNet + Gated Attention), 27B dense, 64 layers, hidden 5120 |
| **Modalities** | Text only (vision encoder not included) |
| **Context Length** | 262,144 tokens |

## How FABQ-RC Works

1. **Fisher-Weighted Channel Importance** — Per-channel Fisher Information (expected
   gradient²) determines which channels matter for loss.
2. **Mixed-Precision Core** — Top 5% channels → int8, rest → binary ±1.
3. **Adaptive Blocksize** — Per-layer sweep {16, 32, 64, 128, 256}, picked by
   Fisher-weighted reconstruction error.
4. **Residual Codebook** — k-means (256 centroids) on quantization residuals corrects
   systematic binary bias.

## Files in This Repo

| File | Size | Description |
|---|---|---|
| `quantized_model.pth` | 52.9 GB | FABQ-RC quantized state dict (int8 cores + binary weights + codebooks + Fisher masks) |
| `config.json` | 4.3 kB | Architecture metadata, mirrors `Qwen/Qwen3.6-27B` text config |
| `generation_config.json` | 202 B | Sampler defaults |
| `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt` | ~23 MB | Tokenizer (mirrored from base, unchanged) |
| `chat_template.jinja` | 7.8 kB | Chat template (mirrored from base) |
| `README.md` | this file | Model card |

> **Note:** the `quantized_model.pth` is a **custom format**. Standard `transformers` /
> `vLLM` / `llama.cpp` will NOT load it. See *Loading* below.

## Loading & Inference

The loader lives in the FABQ-RC project repo: [toxzak/fabq-rc](https://huggingface.co/toxzak/fabq-rc),
specifically **`Main-FABQ-RC-Notebook.ipynb`**. The notebook contains the `FABQ_RC_Model`
class that reconstructs the binary weights, applies the int8 cores, and runs the
residual codebook lookup at inference time.

```python
# Pseudocode — see Main-FABQ-RC-Notebook.ipynb for the real loader
import torch
from fabq_rc import FABQ_RC_Model  # from toxzak/fabq-rc repo

state_dict = torch.load("quantized_model.pth", map_location="cuda")
model = FABQ_RC_Model.from_quantized_state(state_dict).cuda()
output = model.generate("The future of 1-bit quantization is", max_new_tokens=128)
```

### Requirements

- PyTorch with CUDA (custom 1-bit matmul kernel; no native ops exist for the mixed
  int8/binary layout)
- ~16 GB VRAM minimum (1.18 bpw × 27B params ≈ 4 GB weights + activations)
- `transformers >= 4.57.1` for tokenizer/config parsing
- The FABQ-RC loader from the project repo (NOT pip-installable as of this writing)

## Why Not GGUF?

The companion bucket (`toxzak/Qwen3.6-27B-Q1_K-GGUF-bucket`) contains a 3.36 GB
GGUF export of the binary weights only, but that export drops the int8 cores and the
residual codebook. It is a **partial** export and will produce degraded output.
The `.pth` in this repo is the complete artifact.

## Limitations

- **Custom format:** No standard runtime can load it without the FABQ-RC loader.
- **Weight-only quantization:** Activations are not quantized.
- **No vision encoder:** The vision tower from `Qwen3.6-27B` is not included; this
  artifact is text-only.
- **Single-domain calibration:** C4 only; may not generalize to all domains.
- **Not llama.cpp compatible:** Despite the bucket name, the `.gguf` shard in the
  companion bucket is not a runnable llama.cpp model.

## Training / Quantization Details

| Property | Value |
|---|---|
| **Method** | FABQ-RC (see [spec](https://huggingface.co/toxzak/fabq-rc/blob/main/FABQ_RC_SPEC.md)) |
| **Calibration** | C4 dataset, 2048 samples |
| **Sequence Length** | 32 tokens |
| **Hardware** | A100 80GB GPU |
| **Method Origin** | Designed by Zachary Maronek, 2026-04-05 |

## Benchmark Comparison

| Method | bpw | Perplexity | Notes |
|---|---|---|---|
| FP16 | 16.0 | baseline | Qwen3.6-27B full precision |
| Q1_0_g128 | 1.125 | degraded | Bonsai's format |
| BiLLM | 1.08 | ~8.41 (70B) | Best prior work |
| **FABQ-RC** | ~1.18 | TBD | This artifact |

## Related Artifacts

- [toxzak/Qwen3.6-27B-Q1_K-GGUF-bucket](https://huggingface.co/buckets/toxzak/Qwen3.6-27B-Q1_K-GGUF-bucket) — partial GGUF export (binary weights only)
- [toxzak/fabq-rc](https://huggingface.co/toxzak/fabq-rc) — full project repo with loader
- [toxzak/Qwen3.6-35B-A3B-FABQ-RC](https://huggingface.co/toxzak/Qwen3.6-35B-A3B-FABQ-RC) — 35B-A3B variant

## Citation

```bibtex
@misc{fabqrc2026,
    author = {Zach Maronek},
    title = {FABQ-RC: Fisher-Adaptive Binary Quantization with Residual Codebooks},
    year = {2026},
    url = {https://huggingface.co/toxzak/fabq-rc}
}
```

## License

Apache 2.0 — same as the base model [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B).

---

*FABQ-RC by Zach Maronek · 2026*