Quantized GGUF versions of muhammedksee/Qwen3.5-9B-Instruct-Turca-TurkishLLM for local inference on consumer hardware — no data center GPU required. > Looking for the full-precision model? Training details, hyperparameters, and benchmark results are in the base repository. Running a 9B-parameter model at full precision (BF16) requires a data center-grade GPU. GGUF quantization compresses the model weights with minimal quality loss, making it fit in consumer RAM and VRAM. Q4KM retains nearly all of the model’s Turkish language understanding and reasoning capability while cutting memory usage roughly in half. It is the community standard for everyday use and local RAG pipelines. GPU offloading is optional but strongly recommended — with a supported GPU you can reach tens of tokens/second instead of single digits on CPU. For integrating the model into your own Python projects (local RAG, automation agents, etc.).
Модальности:
Генерация текста
Области применения:
Следование инструкциям Диалог / чат
Задача: Генерация текста
Автор: MuhammedKsee
Теги: gguf, qwen, qwen3.5, qwen3.5-9b, q4_k_m, quantized, turkish, nlp
Лайков: 3 | Загрузок: 82
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
