The LongForm dataset is created by leveraging English corpus examples with reverse instructions. We select a diverse set of human-written documents from existing corpora such as C4 and Wikipedia and generate instructions for the given documents via LLMs. Then, we extend these examples with structured corpora examples such as Stack Exchange and WikiHow and task examples such as question answering, email writing, grammar error correction, story/poem generation, and text summarization. Github Repo: https://github.com/akoksal/LongForm !The LongForm dataset LongForm-T5-XL: https://huggingface.co/akoksal/LongForm-T5-XL LongForm-OPT-6.7B: https://huggingface.co/akoksal/LongForm-OPT-6.7B We provide in-depth evaluation of LongForm models and baselines in the paper. We present the METEOR scores of models in out-of-domain datasets. In all tasks, Recipe Generation (RGen), long-form question answering (ELI5), short story generation (WritingPrompts/WP), LongForm models outperform prior instruction-tuned models. Smaller versions of LongForm-OPT models are also available: — LongForm-OPT-1.3B — LongForm-OPT-350M — LongForm-OPT-125M ‡: We can just release the difference between LongForm-LLaMA-7B…
Модальности:
Генерация текста
Области применения:
Мультиязычность
Задача: Генерация текста
Автор: akoksal
Теги: opt, instruction-tuning, text-generation-inference, text2text-generation, en, multilingual, de, it
Лайков: 3 | Загрузок: 25
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.