A learned system that predicts and prevents repetitive degeneration in language models. The system achieves 125x separation between tokens that will repeat and those that won’t: Step 5000 is optimal — further training reduces separation due to overfitting.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: LoganResearch
Теги: peft, adaptive_repetition_controller, repetition-suppression, decode-time-intervention, llama, lora, research, degeneration
Лайков: 3 | Загрузок: 14
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
