This repo contains GGUF format model files for SpeakLeash’s Bielik-11B-v2.5-Instruct. DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations! q4km: Uses Q6K for half of the attention.wv and feedforward.w2 tensors, else Q4K q5km: Uses Q6K for half of the attention.wv and feedforward.w2 tensors, else Q5K q6k: Uses Q8K for all tensors q80: Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. 16bit: Converted fp16 to GGUF format. The GGUF file can be used with Ollama. To do this, you need to import the model using the configuration defined in the Modfile. For model eg. Bielik-11B-v2.5-Instruct.Q4KM.gguf (full path to model location) Modfile looks like: Developed by: SpeakLeash & ACK Cyfronet AGH Language: Polish Model type: causal decoder-only Quant from: Bielik-11B-v2.5-Instruct Finetuned from: Bielik-11B-v2 License: Apache 2.0 and Terms of Use GGUF is a new format introduced by the llama.cpp team on August 21st 2023. Here is an incomplete list of clients and libraries that are known to support GGUF: llama.cpp. The source project for GGUF. Offers a CLI and a server option. …
Модальности:
Генерация текста
Области применения:
Диалог / чат Следование инструкциям
Задача: Генерация текста
Автор: speakleash
Теги: gguf, finetuned, pl, conversational
Лайков: 3 | Загрузок: 71
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
