This repository contains the code implementation for the project that aims to empower pre-trained Large Language Models (LLMs) on non-English languages by building semantic alignment across languages. The project explores cross-lingual instruction-tuning and multilingual instruction-tuning techniques. The code implementation is based on Stanford Alpaca. For detailed information about the conda environment, refer to the environment.yml file. Start by downloading the pre-trained LLM into the ./model directory. You can download all the datasets used in this project from this link. Once downloaded, place the datasets in the ./data directory. The datasets include: Training dataset Alpaca Wikimatrix Newscommentary Evaluation dataset XQUAD MLQA Flores-101 * MI-Eval You can load raw data along with instruction using the provided scripts (./data//). If you want to use a new dataset, you need to implement the corresponding script. The loaded data will have the following structure: To instruction-tune the pre-trained LLM, run the train.sh script. For example, you can instruction-tune LLaMA-7B to x-LLaMA-7B (Chinese) with the following command: In this command, the first argument…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: YunzheLv
Теги: llama, tr, en, text-generation-inference, endpoints_compatible
Лайков: 2 | Загрузок: 16
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.