maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4

RQ-27B-TCAP-AAbliterated

ThinkingCap-27B · NVFP4

27B thinking VL · MTP head kept bf16 for speculative decoding · refusals removed (2-round Heretic).

Params	27B
Active	27B (dense)
Size	18 GB
Perplexity	7.25
Refusals	99 → 8 / 100
Context	256K
MTP head	bf16

TL;DR: ThinkingCap-Qwen3.6-27B abliterated, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 7.25, 256K thinking VL, refusals removed (2-round Heretic). The MTP (multi-token prediction) head is kept at bf16 so speculative decoding runs at full precision.

ThinkingCap-Qwen3.6-27B abliterated NVFP4

bottlecapai/ThinkingCap-Qwen3.6-27B, abliterated (refusal direction removed) with Heretic in two iterative rounds, then quantized to NVFP4 (W4A4) in the compressed-tensors nvfp4-pack-quantized format with llm-compressor (GPTQ + MSE, shared fused-layer scales), with the MTP head left at bf16.

Near-lossless, decensored, and speculative-decoding-ready. Two Heretic rounds cut refusals from 99/100 to 8/100 while keeping a KL divergence of 0.0408 to the original model (well under the 0.5 line that signals capability damage). NVFP4 compresses to ~18 GB at wikitext-2 perplexity 7.25. The multi-token-prediction head that drives self-speculative decoding is excluded from quantization and kept at bf16, so the speedup it provides is not degraded by 4-bit draft predictions.

• Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
• Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored / abliterated model. It follows instructions without refusal guardrails. The abliteration only removes refusals; all other behaviour comes from the base model. You are responsible for how you use it.

Why the MTP head is kept at bf16

ThinkingCap ships a multi-token prediction (MTP) head (mtp_num_hidden_layers: 1), a small extra module that predicts the next token(s) so the model can self-speculate: it drafts ahead and verifies in one pass, cutting decode latency without a separate draft model.

That head is on the critical path for acceptance rate. If it were quantized to 4-bit, its draft predictions would drift from what the full-precision main model would have produced, the verifier would reject more drafts, and the speculative speedup would shrink, you would pay the MTP compute and get less of the latency back. So the quant recipe ignores re:.*mtp.* and leaves the head at bf16 (it is tiny relative to the 27B backbone, so the size cost is negligible). The 4-bit main model does the heavy lifting; the full-precision MTP head keeps acceptance high.

Fidelity

Near-lossless versus the bf16 source, 18 GB vs 55 GB bf16 (33%), at wikitext-2 perplexity 7.25 and KL divergence 0.0408 to the original. GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content.

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality. --max-model-len 131072 is a safe default; raise it if memory allows.
• Add --language-model-only to skip the vision tower and free KV cache for text use.
• The parser flag is not auto-detected; pass it explicitly.

Speculative decoding (MTP)

The bf16 MTP head enables vLLM's built-in self-speculative decoding. Enable it with a speculative config that uses the model's own next-token-prediction module (no separate draft model):

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 --kv-cache-dtype fp8 --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

num_speculative_tokens matches the head's depth (mtp_num_hidden_layers: 1). Measure acceptance rate on your traffic; because the head is bf16, acceptance is not penalized by quantization.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="thinkingcap-27b-abliterated-nvfp4",
    messages=[{"role": "user", "content": "Explain why a heap is better than a sorted array for a priority queue."}],
)
print(r.choices[0].message.content)

About the base model

A 27B Qwen3.5-family vision-language model (ThinkingCap-Qwen3.6-27B) with thinking-mode reasoning, a 256K context window, and an MTP head for speculative decoding.

• 64 decoder layers: hybrid gated delta-net linear attention plus full attention, dense MLP, plus a vision tower for image and video input.
• MTP head (mtp_num_hidden_layers: 1) for multi-token / speculative decoding.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default, with an instruct toggle.

Abliteration

Heretic removes the refusal direction with a TPE-optimized search over per-component ablation strength, jointly minimizing refusal rate and KL divergence from the original model, then merges the best trial. This model was abliterated in two iterative rounds: round 1 removed the dominant refusal direction, then Heretic was re-run on the round-1 model to remove the residual refusal direction that surfaced once the first was gone. Because ThinkingCap is a thinking model, evaluation ran in non-thinking mode so each judged response is a real answer rather than an unfinished <think> block; the shipped model restores the original thinking chat template.

Round	Refusals	Note
Baseline	99/100	original model
Round 1	32/100	dominant refusal direction removed
Round 2	8/100	residual direction removed (final)

• Datasets: mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).
• Final KL divergence 0.0408 (capability preserved, well under the 0.5 damage line).

Quantization

Scheme	NVFP4, W4A4
Weight rounding	GPTQ (Hessian-based error compensation), MSE observer
Weights	FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers
Activations	FP4, dynamic per-group, FP8 (E4M3) scales
Quantized	all language-model Linear layers
Kept in bf16	MTP head (model.mtp.*), vision tower (model.visual.*), lm_head
Untouched	gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear

The recipe ignores re:.*mtp.*, re:.*visual.*, and lm_head. Keeping the MTP head at full precision is the point: it preserves speculative-decoding acceptance (see above). GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.

Calibration: 512 domain-matched samples (long reasoning + general chat + code), max_seq_len=2048, text-only path through the VL model.

Recommended sampling

Thinking mode is the default.

• Thinking, precise: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20
• To run non-thinking, set {%- set enable_thinking = false %} in the chat template, or pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.

Related

• Base model: bottlecapai/ThinkingCap-Qwen3.6-27B
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Qwopus3.6-27B-Coder abliterated
  • Qwopus3.6-27B-v2 abliterated
  • Ornith-1.0-35B MoE abliterated
  • Qwen3.6-40B Deckard
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• The MTP head is bf16; enable speculative decoding via --speculative-config to use it.
• --reasoning-parser is not auto-detected; pass it explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.
• No refusal guardrails; you are responsible for how you use it.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: bottlecapai
• Abliteration: Heretic by Philipp Emanuel Weidmann
• Quantization tooling: llm-compressor / compressed-tensors

Part of Rogue Quants · NVFP4 component datasheets · collection. Fabricated on GB10 (Blackwell) with llm-compressor. Refusals shown per 100 harmful prompts; "n/a" = not separately measured (base-inherited).