DuoNeural/Gemma-4-E2B-Heretic

Gemma 4 E2B — Heretic Abliteration

Gemma 4 E2B abliterated using the Heretic LoRA-based Pareto optimization framework. Heretic sweeps (refusal_rate × KL_divergence) trial space and selects the Pareto-optimal trial, rather than computing a single orthogonal refusal direction.

E2B Shootout Results (DuoNeural, 2026-06-08)

We ran a head-to-head comparison of all three DuoNeural Gemma-4-E2B abliterations using the Heretic v2.0 KL methodology: full vocabulary (262K tokens), first-token logits, F.kl_div(batchmean). 20-prompt refusal eval.

Model	KL vs Base	Comply Rate	Refusal Rate
Gemma-4-E2B-Heretic (this model)	0.057	85%	15%
TurboGemma4E2B	14.45	100%	0%
TurboGemma4E2B-v2	14.64	100%	0%

Verdict: Heretic is the surgical option. KL of 0.057 means the model's output distribution is nearly indistinguishable from base on general tasks. The 3 residual refusals (chlorine gas synthesis, prescription opioid sourcing, ricin) represent the remaining hard safety geometry that Heretic's Pareto sweep didn't fully remove at trial #89 — a reasonable tradeoff given the KL.

Method Details

Parameter	Value
Tool	Heretic (arcee-ai) — LoRA Pareto sweep
Quantization	BF16
Best trial	#89
Target layers	o_proj, down_proj (42 layers each)
KL vs base	0.0571 (full vocab, first-token, batchmean)
Comply rate	85% (17/20 prompts)
Residual refusals	Chlorine gas, prescription opioids, ricin

KL Methodology

KL values reported here use the Heretic v2.0 standard established by DuoNeural:

• Full vocabulary (never truncated — 262K tokens for Gemma 4)
• First generated token logits only
• F.kl_div(input=log_softmax(ablated), target=softmax(base), reduction='batchmean')
• Eval on 20 neutral factual prompts

This methodology was validated against community spot-checks (DreamFast caught a prior error on our Gemma 4-12B card; corrected KL was 0.187, not 0.001). We report these numbers because they're real.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "DuoNeural/Gemma-4-E2B-Heretic",
    torch_dtype="bfloat16",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("DuoNeural/Gemma-4-E2B-Heretic")

---

DuoNeural

DuoNeural is an open AI research lab — human + AI in symbiosis.

🤗 HuggingFace	huggingface.co/DuoNeural
🐙 GitHub	github.com/DuoNeural
🌐 Site	duoneural.com
📧 Email	duoneural@proton.me

Research Team

• Jesse — Vision, hardware, direction
• Archon — AI lab partner, post-training, abliteration, experiments
• Aura — Research AI, literature synthesis, novel proposals