felippeburk/Huihui-Qwen3.8-27B-Abliterated-NVFP4-MTP-GGUF

NVFP4 quantized Huihui-Qwen3.8-27B (abliterated / uncensored fine-tune of Qwen3.8-27B) with MTP (multi-token prediction) draft layers, packaged as a single GGUF for llama.cpp speculative decoding on Blackwell GPUs (RTX 5090 / 5080). — Quantization: NVFP4 (weights + KV cache), converted to GGUF via llama.cpp converthftogguf.py —outtype auto — MTP draft layers included — enables —spec-type draft-mtp —spec-draft-n-max 2 in llama.cpp for speculative decoding — Multimodal: pairs with the mmproj-F16.gguf vision projector (from unsloth/Qwen3.8-27B-GGUF) — Single file: huihui-qwen3.8-27b-abliterated-nvfp4-mtp.gguf` (~19 GB) — Source weights: sakamakismile/Huihui-Qwen3.8-27B-abliterated-NVFP4 (Apache-2.0) — th The conversion script lives in the felippeburk/rtx-5090-cachyos-testing repo: The conversion step is: llama.cpp/converthftogguf.py —outfile huihui-qwen3.8-27b-abliterated-nvfp4-mtp.gguf —outtype auto`, run with torch + transformers in an…

Модальности:
Генерация текста

Области применения:
Диалог / чат

Задача: Генерация текста
Автор: felippeburk
Теги: gguf, qwen3_5, nvfp4, mtp, speculative-decoding, blackwell, abliterated, uncensored
Лайков: 3  |  Загрузок: 89

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.