The first diffusion-based language model for Telugu! This model explores masked diffusion language modeling for Telugu text generation, adapting the MDLM (Masked Diffusion Language Models) architecture for an Indic language. Unlike traditional autoregressive models (GPT-style), this model generates text through iterative denoising, starting from completely masked sequences and progressively revealing tokens. — ✅ First Telugu Diffusion LM: Novel application of diffusion modeling to Telugu — ✅ Based on IndicBERTv2: Leverages strong Telugu language understanding — ✅ Question-Answering: Fine-tuned on 93K Telugu Q&A pairs — ✅ Bidirectional Context: Unlike autoregressive models, can consider full context — ⚠️ Research Preview: See limitations section below Phase 1: Pretraining (Completed) — Base model: IndicBERTv2-MLM-only — Extended position embeddings (512 → 1024) — Continued pretrai
Модальности:
Генерация текста
Области применения:
Диалог / чат Ответы на вопросы
Задача: Генерация текста
Автор: Prahaladha
Теги: bert, fill-mask, diffusion, telugu, indic-languages, question-answering, masked-language-modeling, first-of-its-kind
Лайков: 3 | Загрузок: 172
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
