This is quantized version of universitytehran/PersianMind-v1.0 created using llama.cpp PersianMind is a cross-lingual Persian-English large language model. The model achieves state-of-the-art results on Persian subset of the Belebele benchmark and the ParsiNLU multiple-choice QA task. It also attains performance comparable to GPT-3.5-turbo in a Persian reading comprehension task. — Developed by: Pedram Rostami, Ali Salemi, and Mohammad Javad Dousti — Model type: Language model — Languages: English and Persian — License: CC BY-NC-SA 4.0 (non-commercial use only.) Use the code below to get started with the model. Note that you need to install sentencepiece and accelerate libraries along with PyTorch and 🤗Transformers to run this code. Quantized models can be run on resource-constrained devices. To quantize the model, you should install the bitsandbytes library. In order to quantize the model in 8-bit (INT8), use the code below. Alternatively, you can quantize the model in 4-bit (NormalFloat4) with the following code. We evaluated quantized models in various tasks against the original model. Specifically, we evaluated all models using the reading comprehension multiple-choice…
Модальности:
Генерация текста
Области применения:
Мультиязычность
Задача: Генерация текста
Автор: QuantFactory
Теги: gguf, text-generation-inference, multilingual, fa, en, co2_eq_emissions
Лайков: 3 | Загрузок: 846
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
