What is abliteration?

Abliteration is a weight-editing technique that stops a language model from refusing requests by removing a single direction from its internal activations. It does not train the model: it finds one vector, then rewrites the weight matrices so the model can no longer write along that vector. Model repositories carry it as a token in the name (for example "abliterated").

Where it comes from

The research basis is "Refusal in Language Models Is Mediated by a Single Direction" by Arditi, Obeso, Syed, Paleka, Panickssery, Gurnee and Nanda (arXiv 2406.11717). The authors report that across 13 open-source chat models of up to 72B parameters, refusal is mediated by a one-dimensional subspace: erasing that direction from the residual stream stops the model refusing harmful instructions, and adding it makes the model refuse harmless ones. The paper presents this as an interpretability finding and a way to edit weights.

Maxime Labonne's Hugging Face write-up, Uncensor any LLM with abliteration, turned it into a practical recipe that many community releases follow.

How the procedure works

Following that write-up:

1. Run the model on a set of harmful instructions and a set of harmless ones, recording residual-stream activations at the final token position.
2. Take the mean difference between the two sets. That vector is the candidate refusal direction at each layer.
3. Normalize the vectors and test which layer's direction gives the best result at inference.
4. Orthogonalize the weight matrices that write into the residual stream (the embedding, the attention output and the MLP output) against that direction, so the model cannot represent the feature.

The write-up notes that this needs no architecture change and no retraining. You can also apply it as an inference-time intervention instead of editing weights permanently. The finding that makes it cheap is that a model's refusal behavior can be located in a direction you can compute with a few forward passes.

Quality cost and healing

Removing a direction is not free. Labonne reports a drop across all benchmarks in the ablated version, and recovered it with a short round of Direct Preference Optimization fine-tuning. The model card for NeuralDaredevil 8B abliterated describes the same two-step recipe: an abliterated base followed by a DPO fine-tune, with the card stating that the fine-tune recovers the performance lost to abliteration.

Example models in the catalog

• Meta Llama 3.1 8B Instruct abliterated: its card describes an uncensored version of Llama 3.1 8B Instruct created with abliteration.
• NeuralDaredevil 8B abliterated: abliterated, then DPO fine-tuned, as above.
• Huihui Qwen3.5 27B abliterated: per its card, an abliterated version of Qwen3.5-27B, which the card calls a crude proof-of-concept implementation of refusal removal.
• Huihui DeepSeek R1 Distill Llama 8B abliterated: the technique applied to a distilled model.
• Qwen3 30B A3B abliterated: the technique on a mixture-of-experts model.

These cards state plainly that safety filtering is reduced and that the user is responsible for how outputs are used. Read the card of any abliterated model before deploying it in a product.

What it means when you pick a GPU

An abliterated model has the same architecture and parameter count as the model it came from, so it needs the same memory: nothing is added or removed. Size it as you would the original, using the VRAM calculator, and apply quantization the same way. Check the dtype listed on the repository. Because the edit only changes weights, there is no extra inference cost. Aquanode rents GPUs by the hour, so you can load one of these checkpoints on a card sized for its base model and compare it with the original.