This is an experimental version of the repository containing quantized Bielik-11B-v.2.3-Instruct models using calibration with importance matrix (imatrix). Models with low precision (2bit, 3bit) for use in mobile devices or minicomputers. Note that these models should be used mainly in instructional mode (not chat). We recommend setting low temperature values. Models with higher precision 4-8bit after calibration may show better quality than models without calibration. DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations! IQ1M: (1.75bit) Extremely low quality, not recommended. IQ2XXS: Lower quality, uses SOTA techniques to be usable. IQ3XXS: Lower quality, new method with decent performance, comparable to Q3 quants. IQ4XS: Decent quality, smaller than Q4KS with similar performance, recommended. Q4KM: Uses Q6K for half of the attention.wv and feedforward.w2 tensors, else Q4K Q5KM: Uses Q6K for half of the attention.wv and feedforward.w2 tensors, else Q5K Q6K: Uses Q8K for all tensors Q80:** Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. The GGUF file can be used with Ollama.…
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: speakleash
Теги: gguf, finetuned, pl, imatrix, conversational
Лайков: 3 | Загрузок: 554
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
