Q4KM size, near-Q80 quality. Better perplexity than Q4K_M at the same size (6.61 vs 6.86). When combined with TurboQuant+ KV cache compression, this enables significantly larger context windows within the same memory budget. Best option if Q80 doesn’t fit in memory but Q4K_M quality isn’t good enough. TurboQuant+ Config-I quantization of Qwen/Qwen3.5-35B-A3B. Config-I applies WHT-domain compression (TQ41S) to attention and gate/up tensors while keeping boundary layers and ffndown at higher precision for optimal quality. See the getting started guide for details. > Requires TurboQuant+ llama.cpp fork at tag tqp-v0.1.0. Will NOT work with stock llama.cpp. TurboQuant+ is an independent research project. These quantization types have not been merged into upstream ggml-org/llama.cpp. Do not file issues there for TQ models. At nearly the same file size (~4% larger), Config-I achieves substantially better perplexity than Q4KM (6.61 vs 6.86) with faster decode. No forward-compatibility guarantee. This model was built and validated against the tag above. Future fork updates may change the format. If decode produces garbage, rebuild from this tag. On Metal, Config-I trades some decode speed…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: thetom-ai
Теги: gguf, turboquant, turboquant-plus, tqp-v0.1.0, moe, en, zh, imatrix
Лайков: 3 | Загрузок: 19
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
