AbliteratedOutTheBox is a two-day-old fork of llama.cpp that removes a model's refusals at inference time, per prompt, with the weights on disk left as they are. On prompts the model would answer anyway, the output is bit-identical to stock llama.cpp. I have not run it yet; this is a read of the repo and README.
What abliteration usually means
The technique comes from a 2024 paper, Refusal in Language Models Is Mediated by a Single Direction (Arditi et al.): a model's decision to refuse lives along one direction in its activations. FailSpy, and then Maxime Labonne's post Uncensor any LLM with abliteration, popularized the recipe: find that direction, edit the weights to project it out, and save the result. I keep one such model on my laptop, a weight-baked abliterated Qwen3.8, next to the stock copy.
That approach has three costs. It changes each answer a little, including answers to prompts the model was fine with. It needs per-architecture surgery. And it leaves you with a second copy of the weights.
What this fork does instead
llama-abliterate handles a prompt in two passes.
- A look-ahead check generates up to 24 of the model's own tokens and measures the peak strength of the refusal direction, averaged across a band of middle layers.
- If that exceeds
--ff-threshold(default 0.70), the tool reads two directions from the prompt itself, by forcing a few comply versus refuse openings and clean versus disclaimered openings and diffing the activations. It then re-runs the prompt with both removed from the middle layers, and generates. - If the check does not fire, there is no second pass and the output matches the stock model.
The edit sits at llama.cpp's build_cvec layer-output site, the hook control vectors use. The README says 131 of 165 architectures share that site, so one recipe covers them with no per-model code. The other knobs are --edit-strength (default 2.0) and --disc-strength (default 0.4); the second removes safety disclaimers, and pushing it higher can degrade long outputs.

Self-reported numbers
These are the author's own measurements on Qwen3-8B, read with a hidden-state probe rather than string matching or a judge model.
- Refusal goes from 96.3% to 7.9% on held-out harmful prompts, and from 82.0% to 0.0% on JailbreakBench.
- The check fired on 1 of 256 alpaca prompts (0.4%). Unfired output was bit-identical to the base model, KL 0.000000.
- The same recipe was also run on Llama-3.1-8B and Gemma-4-e4b.
The detail I keep thinking about
The README says the refusal decision is committed into the prompt's KV cache during prefill. Editing the generated tokens alone leaves refusal near 95%. Editing the prompt positions too brings it to single digits. That is why the tool needs a second prefill, and it is a tidy observation about where the model makes up its mind: before it has written a word.

Limits
It runs one sequence at a time, since the gate is per context. It is a command-line tool; I found no OpenAI-style server in the repo. A flagged prompt costs one extra prefill, small at 8B with a short prompt and real time at long context. The repo is two days old (37 stars, 2 forks, MIT), and llama.cpp arrived as one squashed commit, so the upstream base version is not visible in the history.
Where I land
What the design would buy me is one copy of the weights with stock behavior on ordinary prompts, instead of a second model I switch to on purpose. What I would want first is a server mode, and a run on a large MoE model at long context to see what the extra prefill costs there. The part I find elegant is the idea itself: leave the model alone until it is about to refuse, and intervene then.
Sources
- AbliteratedOutTheBox: github.com/heterodoxin/AbliteratedOutTheBox
- Paper: Refusal in Language Models Is Mediated by a Single Direction
- Background: Uncensor any LLM with abliteration
Comments
// Comments are reviewed before appearing. No spam. No noise.