A collection of GGUF quantizations of Qwen3.8 27B Uncensored, prepared for use with llama.cpp and other GGUF-compatible inference engines. The goal of this repository is to make multiple quantization levels publicly available so users can compare quality, memory usage, inference speed, and compression across different quantization formats. > Status: Active quantization and benchmarking project. Additional quantizations and benchmark results will be added over time. Architecture: Qwen3.8 27B Parameter count: ~27B Format: GGUF Original precision: F16 Quantization: Multiple llama.cpp quantization formats Primary inference engine: llama.cpp Repository type:** Community GGUF quantizations Additional IQ/imatrix-based variants may be added as the project progresses. Model quality Memory requirements Inference speed Disk size The Q4KM version produced from the F16 source is approximately 16 GiB. If your hardware has more memory available, consider Q5KM, Q6K, or Q80. If memory is limited, Q3/Q2 variants may be useful, although quality degradation becomes increasingly noticeable at lower bitrates. The quantizations are generated from the original F16 GGUF rather than repeatedly…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: end9214
Теги: llama.cpp, gguf, qwen, qwen3.8, conversational, endpoints_compatible
Лайков: 3 | Загрузок: 690
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
