IntelliTeX is an experimental Small Language Model (SLM) study for converting English, spoken-style math descriptions into a single LaTeX equation. It is intended as a research artifact (training regimes, decoding constraints, stress tests), not a production-ready LaTeX authoring system. — Base model: Salesforce/codet5p-220m (CodeT5+ 220M) — Primary task: text → LaTeX equation generation (single equation output) — Primary language: English Intended use — Drafting LaTeX equations from short natural-language descriptions — Prototyping or benchmarking compact models on domain-specific translation Not recommended — Fully automated formula generation without verification We evaluated multiple training configurations to understand what improves a compact model most: 1. LoRA fine-tuning: rapid iteration and capability checks 2. Full-parameter fine-tuning (FPFT): to measure the performance ceiling (LoRA often underperformed FPFT) 3. Two-stage pipeline (continued pretraining → FPFT) inspired by CodeT5+ training recipes: — Stage 1: domain-adaptive continued pretraining on TeXTeller with span-denoising + causal LM objectives — ~4B tokens, ~76k steps — Stage 2: supervised FPFT on Speech2LaTeX…
Модальности:
Генерация текста
Области применения:
Математика Перевод
Задача: Генерация текста
Автор: duanxianpi
Теги: t5, LaTeX, math, text-to-LaTeX, translation, en
Лайков: 3 | Загрузок: 119
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
