— Model creator: Qwen — Original model: Qwen2.5-Coder-32B-Instruct This repo contains State Of The Art quantized GGUF format model files for Qwen2.5-Coder-32B-Instruct. Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the CodeFeedback-Filtered-Instruction dataset. Fill-in-Middle tokens are automatically detected and supported as of commit 11ac980, see example. Update January 6th 2025: Added links to full context YaRN-enabled GGUFs (using GGUF Editor). These quantised GGUFv3 files are compatible with llama.cpp from February 27th 2024 onwards, as of commit 0becb22 They are also compatible with many third party UIs and libraries provided they are built using a recent llama.cpp. GGMLTYPEIQ1S — 1-bit quantization in super-blocks with an importance matrix applied, effectively using 1.56 bits per weight (bpw) GGMLTYPEIQ1M — 1-bit quantization in super-blocks with an importance matrix applied, effectively using 1.75 bpw GGMLTYPEIQ2XXS — 2-bit quantization in super-blocks with an importance matrix applied, effectively using 2.06 bpw GGMLTYPEIQ2XS — 2-bit quantization in super-blocks with an importance matrix…
Модальности:
Генерация текста
Области применения:
Генерация кода Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: CISCai
Теги: gguf, code, codeqwen, chat, qwen, qwen-coder, en, endpoints_compatible
Лайков: 4 | Загрузок: 1,022
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
