This is a TRL language model. It has been fine-tuned with reinforcement learning to guide the model outputs according to a value, function, or human feedback. The model can be used for text generation. This project aims to reduce the level of toxicity in the outputs generated by the LAMINI Flan T5 248M language model using Reinforcement Learning with Artificial Intelligence Feedback technique (RLAIF). Reinforcement Learning with Human Feedback (RLHF) is a method to align models with a particular kind of data. RLHF creates a latent reward model using human feedback and finetunes a model using Proximal Policy Optimization. RLAIF on the other hand replaces human feedback with a high-performance AI agent. The model has been fine-tuned on the Social Reasoning Dataset by ProlificAI for 191 steps and 1 epoch using the Proximal Policy Optimisation (PPO) algorithm. The Roberta hate text detection model was used as the Proximal Policy Optimisation (PPO) reward model. The power of this model lies in its size; it is barely 500 MBs in size and performs well given its size. The intended use of this model should be conversation, text generation, or context-based Q&A. This model might not perform…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: ARahul2003
Теги: t5, text2text-generation, trl, reinforcement-learning, LLM detoxification, conversational, en, text-generation-inference
Лайков: 2 | Загрузок: 7
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.