▶ Try it live: ruqlm.web.app — runs entirely in your browser, no server, no signup needed for guest access. A ladder of five Arabic language models, 0.74M to 29.89M parameters, trained from scratch on a corpus of 50,003 Arabic stories under a fixed budget of 76.8M training tokens. The tokenizer, the corpus and the weights were all built for this project. Weights are full-precision ONNX — no quantisation. Perplexity matches the original PyTorch checkpoints to three decimals (9.2009 vs 9.2009), so these are the trained models, not an approximation of them. Decoder-only transformer: RMSNorm, RoPE, SwiGLU, tied embeddings, 512-token context. Tokenizer is a byte-level BPE with an 8,192 vocabulary trained on the same corpus (1.411 tokens per word). These are base text-completion models, not chat models. They were trained on Arabic stories only, so they continue what you start; they do not answer questions or follow instructions, having had no instruction tuning (SFT) or RLHF. Expect narrative, not answers. Morphology and orthography are strong. Semantic and referential coherence are weaker — which is what the accompanying study measures, and what these models were built to quantify…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Ruqiya
Теги: onnx, ruqlm, arabic, story-generation, scaling-laws, small-language-model, ar
Лайков: 3 | Загрузок: 39
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
