This model is a fine-tuned version of Qwen/Qwen3-1.7B optimized for mathematical reasoning on the GSM8K benchmark. For deployment with llama.cpp, Ollama, or other GGUF-compatible runtimes: — Size: 247,467 examples — Sources: — GSM8K train set (7,473 examples) — MetaMathQA GSM-related examples (239,994 examples) — Format: Conversational messages with … chain-of-thought reasoning — Stage 1 (2 epochs): lr=2e-5, loss 0.30 → 0.17 — Stage 2 (1 epoch): lr=5e-6, loss 0.17 → 0.167 — Batch size: 8 per device, gradient accumulation 4 — Hardware: NVIDIA H100 80GB GPU — Total training time: ~7 hours — Accuracy: 77.2% ± 1.2% (standard error) — Test set: 1,319 grade school math word problems — Accuracy: 55.2% — Test set: 500 competition-level math problems — Best performance on Algebra (71.8%) and Prealgebra (68.3%) — Model uses chain-of tags 1. Chain-of-thought format is crucial — The … reasoning format significantly improves math performance 2. Large diverse dataset works better — MetaMathQA (240K examples) outperforms small task-specific data 3. Two-stage training — Starting with higher LR (2e-5) then refining with lower LR (5e-6) works well 4.…
Модальности:
Генерация текста
Области применения:
Математика Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: HuggingFaceTB
Теги: gguf, qwen3, math, gsm8k, fine-tuned, chain-of-thought, reasoning, conversational
Лайков: 4 | Загрузок: 985
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
