This repository contains a Q3KM GGUF quantization of miromind-ai/MiroThinker-1.7-mini. — Quantization: Q3KM — Quantized with: Unsloth + llama.cpp — GGUF size: 14.71 GB (13.70 GiB) — Source model license: apache-2.0 This quant is aimed at fitting a 16 GB GPU while keeping better quality than lower-bit options. In local testing, it loaded successfully with llama.cpp on an AMD Radeon RX 9060 XT through the Vulkan backend. If VRAM is tight on your setup, reduce context length first, then reduce GPU offload. — This repo contains only the GGUF quantized artifact, not the original safetensors weights. — The upstream MiroThinker model itself can document deeper lineage separately.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: kai-os
Теги: llama.cpp, gguf, quantized, q3_k_m, unsloth, en, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 32
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
