This repository contains a T5-base model fine-tuned for generating question-answer pairs from a given context. Leveraging T5’s text-to-text framework and a novel training strategy where the answer is occasionally masked (30% chance), the model is designed to generate both coherent questions and corresponding answers—even when provided with incomplete answer information. Built with PyTorch Lightning, this implementation adapts the pre-trained T5-base model for the dual task of question generation and answer prediction. By randomly replacing the answer with the [MASK] token during training, the model learns to handle scenarios where the answer is partially or completely missing, thereby improving its robustness and versatility. — Answer Masking: During training, the answer is replaced with the [MASK] token 30% of the time. This forces the model to generate both the question and the answer even when provided with partial input. This format ensures that the model generates a question first, followed by the corresponding answer. — Framework: PyTorch Lightning — Base Model: T5-base — Optimizer: AdamW with linear learning rate scheduling — Batch Size: 8 (training) — Maximum Token Length:…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: fares7elsadek
Теги: t5, text2text-generation, question-generation, t5-base, education, LMS, SQUAD, mcq-questions
Лайков: 3 | Загрузок: 27
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
