GGUF quantizations of InfinimindCreations/K2-Horizon-MoVA-36B-A4B-uncensored, the rank-3 abliteration of IFM K2-Horizon-MoVA-36B-A4B. Method, leak correction, capability and negative-control results are documented there. Upstream llama.cpp does not yet support the k2horizon architecture. These files were produced and tested with the MBZUAI-IFM fork of llama.cpp, branch model/K2Horizon, commit 35999d1`. Tools built on llama.cpp (Ollama, LM Studio) can load them only once their bundled llama.cpp includes the architecture. Quantization can partly restore refusal behavior, so the smallest file was measured again rather than assumed. Q4KM, greedy decoding, reasoningeffort=»low»`, on the same leak-free validation stages 4 and 5 and with the same rule-based counting as the full-precision release: Paired against the original: 88 refuse to comply, 0 comply to refuse, exact McNemar p = 6.5e-27. The responses themselves differ from the NF4 run (none of the 112 is identical), but the counts do not: quantization to Q4KM did not bring refusals back on these prompts. The original and NF4 columns were run with transformers and bitsandbytes, the Q4KM column with llama.cpp. Capability. Same MMLU…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: InfinimindCreations
Теги: gguf, llama-cpp, uncensored, abliterated, k2-horizon, mova, moe, en
Лайков: 3 | Загрузок: 747
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
