Huihui-Nex-N2-mini-abliterated-text-MTP-NVFP4

Text-only NVFP4 (W4A4) quantization of huihui-ai/Huihui-Nex-N2-mini-abliterated — the abliterated Nex-N2-mini (35B-A3B qwen3_5_moe hybrid-linear MoE). This is the language model only (Qwen3_5MoeForCausalLM, vision tower dropped) at ~22.7 GB, with the native MTP draft in MTP/. For the full vision-language version, see Huihui-Nex-N2-mini-abliterated-MTP-NVFP4.

Made by quantizing the huihui-ai abliterated release with llm-compressor + compressed-tensors. All credit for the model itself goes to huihui-ai (abliteration) and nex-agi (the original Nex-N2-mini); this repo only adds the FP4 quantization.

Lineage: nex-agi/Nex-N2-mini → huihui-ai abliteration → Lna-Lab NVFP4 W4A4, text-only (this repo).

What it is

Architecture	Qwen3_5MoeForCausalLM (qwen3_5_moe_text) — language model only, no vision tower
Total / active	~34B total · ~3B active (256 experts, top-8, + shared expert)
Attention	hybrid: linear-attention (GatedDeltaNet-style SSM) ×3 → full-attention every 4th layer (40 layers)
MTP	native draft shipped in MTP/ (BF16, ~1.69 GB)
Quantization	NVFP4 nvfp4-pack-quantized, W4A4, group size 16 — 30,720 expert projections + attention/linear-attn projections packed; router/norms/conv/lm_head BF16
Size	~22.7 GB
Nature	abliterated / uncensored · reasoning model (<think>…</think>)

⚠️ Serving note (read this)

As of vLLM 0.22.0, the standalone text config (model_type: qwen3_5_moe_text / Qwen3_5MoeForCausalLM) is not yet served directly — vLLM routes this architecture through its Qwen3-VL path and expects a multimodal config with vision_config, so it stops with a config-type mismatch. Until vLLM adds a standalone text path for qwen3_5_moe, run the vision-language sibling with --limit-mm-per-prompt '{"image":0,"video":0}' for identical text-only behavior (the language weights are byte-for-byte the same). This text-only repo is published for transformers use, future/other backends, and as the smaller checkpoint.

llm-compressor recipe: QuantizationModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head","re:.*visual.*","re:.*mlp.gate$","re:.*mlp.shared_expert_gate$"]), 32 calibration samples (neuralmagic/calibration).

License

Inherits apache-2.0 from the upstream models. Abliterated/uncensored: you are responsible for how you use it.

Credits

• Original model: nex-agi/Nex-N2-mini
• Abliteration (uncensoring): huihui-ai/Huihui-Nex-N2-mini-abliterated
• NVFP4 quantization: Lna-Lab · Tooling: llm-compressor / compressed-tensors / vLLM

Support the Base Model Author (huihui-ai)

If you find the abliterated base useful, please support huihui-ai — this repo only adds the FP4 quantization; the abliteration work is theirs:

• Ko-fi: https://ko-fi.com/huihuiai
• Bitcoin: bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge