Released with the paper «Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning» (NAACL 2024). This represents the PEFT weights only. The base model is LLaMA 2. Instruction finetuning was done using 4 bit QLoRA on a single A100 GPU with the PEFT config as given below. The dataset used for this instruction finetuning process is a translated version of the cleaned alpaca dataset (translated using NLLB-1.3B). Do note that this model might have inferior performance on some language specific tasks compared to full finetuning or a different base model trained with more language specific data. The following bitsandbytes quantization config was used during training: — loadin8bit: False — loadin4bit: True — llmint8threshold: 6.0 — llmint8skipmodules: None — llmint8enablefp32cpuoffload: False — llmint8hasfp16weight: False — bnb4bitquanttype: nf4 — bnb4bitusedoublequant: True — bnb4bitcomputedtype: bfloat16
Модальности:
Генерация текста
Задача: Генерация текста
Автор: iamshnoo
Теги: peft, global-gallery, instruction-tuning, qlora, alpaca, fa, en
Лайков: 2 | Загрузок: 18
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.