theprint/TiTan-Qwen2.5-0.5B-GGUF

Quantized GGUF versions of TiTan-Qwen2.5-0.5B for use with llama.cpp and other GGUF-compatible inference engines. — Base model: Qwen/Qwen2.5-0.5B-Instruct — Fine-tuned model: theprint/TiTan-Qwen2.5-0.5B — Quantized by: theprint — TiTan-Qwen2.5-0.5B-f16.gguf (948.1 MB) — 16-bit float (original precision, largest file) — TiTan-Qwen2.5-0.5B-q3km.gguf (339.0 MB) — 3-bit quantization (medium quality) — TiTan-Qwen2.5-0.5B-q4km.gguf (379.4 MB) — 4-bit quantization (medium, recommended for most use cases) — TiTan-Qwen2.5-0.5B-q5km.gguf (400.6 MB) — 5-bit quantization (medium, good quality) — TiT For maximum quality: Use q80 or f16 For maximum speed/smallest size: Use q3km or q4ks`

Модальности:
Генерация текста

Области применения:
Диалог / чат

Задача: Генерация текста
Автор: theprint
Теги: gguf, quantized, llama.cpp, titan-qwen2.5-0.5b, en, endpoints_compatible, conversational
Лайков: 3  |  Загрузок: 100

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.