Document Oracle This is a LoRA model for OpenLM-Research’s Open LLaMA 7B model trained on our specifically-assembled webGPTxDolly extractive Q&A dataset using the Alpaca instruction format. The included tokenizer is based on that of the baseline model, however the BOS, EOS, and UNK/PAD tokens are distinctly defined, which was not the case with the baseline. The architecture of this LoRA model follows that of the LLaMA-7b Alpaca-LoRA with the hyper-parameters: The model was trained using PEFT for up to 3 epochs, with loadbestmodelatend=True set. The learning rate was set to 5e-5, so the minimal validation loss occurred very near to the end of training. The combined model can be loaded and used right out of the box: The adapter can be recombined with the baseline model to generate text:
Модальности:
Генерация текста
Задача: Генерация текста
Автор: starfishmedical
Теги: llama, en, text-generation-inference, endpoints_compatible
Лайков: 3 | Загрузок: 10
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
