AppleMind-1.0-Mini is a compact decoder-only causal language model trained from scratch by AppleMind on a 300M-token curriculum. The model has 1,020,480 parameters, a 256-token context window, and a 50,263-token digit-aware byte-level BPE tokenizer (GPT-2 + special tokens). AppleMind 1.0 Mini uses a 50,263-token digit-aware byte-level BPE tokenizer based on the GPT-2 tokenizer, with 3 additional special tokens. Digits are handled individually rather than being collapsed into large number tokens. The training run used an equal mixture of FineWeb-Edu, FineWeb-HQ, and SmolLM-Corpus, with 100M tokens sampled from each dataset. AppleMind 1.0 Mini has not been formally evaluated with lmeval yet**. No benchmark scores are currently reported. The model’s current generation quality has been checked with basic text-generation prompts, including: Prompt: Once upon a time Once upon a time It the several times- nowThis, does to form provide from find another times work not atl The couldWhen on all be way H It lives among times always, worked its G during work used after several There and at there b known came be very that It thought It betweenIn course does. case other It 5?: or often I at’s…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: AppleMind-AI
Теги: gpt2, lm, language-model, causal-lm, causal-language-model, decoder-only, base-model, pretraining
Лайков: 3 | Загрузок: 131
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
