To use the pre-trained model for masked language modeling, use the following snippet: For a hands-on example, check out this Colab notebook. For more information and implementation details, visit our github repository: DUO and project page: Project Page The model, which has a context length of 1024 and is similar in size to GPT2-medium with approximately 130 million non-embedding parameters, was trained for 1M steps on the OpenWebText corpus. For more details, please see our paper: The Diffusion Duality.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: s-sahoo
Теги: DUO, fill-mask, custom_code, en
Лайков: 3 | Загрузок: 1,686
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.