An FP8-quantized abliterated version of Qwen/Qwen3-235B-A22B. This is the recommended version for inference — it fits on 4x RTX Pro 6000 GPUs (384 GB total) and serves well under vLLM. Abliteration removes the dominant refusal direction from model weights using the technique from Refusal in Language Models Is Mediated by a Single Direction (Arditi et al.), making the model significantly less likely to refuse prompts while retaining its full capabilities. This started as research into abliteration, but also as a search for the best creative writing model I could run locally. Qwen3-235B has excellent prose quality, but its refusal behavior gets in the way of fiction — injecting disclaimers, refusing to write morally complex characters, hedging on anything edgy. Abliteration fixes this well, especially with a good system prompt. The BF16 weights (~438 GB) don’t fit in 384 GB of VRAM, so this FP8 version is what I actually serve. > The full-precision BF16 version is also available: null-space/Qwen3-235B-A22B-abliterated > A vision-language variant is also available: null-space/Qwen3-VL-235B-A22B-Abliterated-FP8 Evaluated using lm-evaluation-harness v0.4.11 against the vLLM-served FP8…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: null-space
Теги: qwen3_moe, abliterated, uncensored, qwen3, moe, ablation, fp8, quantized
Лайков: 3 | Загрузок: 31,035
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
