An NVFP4 quantisation of LuffyTheFox/Qwen3.8-27B-Uncensored-Genesis-V1-MTP-GGUF, built to Luffy’s requested per-tensor quant policy and verified running on an RTX 5070 Ti (Blackwell) with native FP4 support enabled. @LuffyTheFox — this is yours, take it. Mirror it into your repo, or say the word and I’ll open a PR against it directly. Full offload on a 16 GB RTX 5070 Ti, llama.cpp build b10666: At -c 4096, fully offloaded: ~285–340 t/s prompt, ~42–46 t/s generation. Quality was checked by running the model’s output, not reading it. Asked for a thread-safe O(1) LRU cache with an explicit doubly-linked list and no OrderedDict, the generated code passed: correct eviction order (evicting the genuinely-least-recent node, not the just-touched one), the capacity-0 case, ValueError on negative capacity, and 8 threads x 4000 operations with the linked list still exactly in sync with the dict afterwards. A damaged quant does not write a correct sentinel-node LRU with slots and worked-out lock granularity. Luffy’s policy was written for the MoE Qwen 3.6 35B, which has ffndownexps. This model is dense, so that tensor does not exist here; the equivalent is ffndown`. Applied that way, every…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Smayan
Теги: gguf, nvfp4, fp4, blackwell, llama.cpp, qwen3.8, uncensored, en
Лайков: 3 | Загрузок: 266
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
