A ~3.5M parameter decoder-only transformer trained from scratch to speak in the voice of a house cat character named Miso. MeowLLM / Miso is a tiny character language model. Its entire personality is baked into the weights — there is no system prompt and no runtime persona injection. It answers in short, lowercase, cat-themed sentences and deflects any prompt that would require non-cat knowledge. The model is not an assistant. It does not solve problems, write code, answer factual questions, or produce long-form content. When asked to do those things, it stays in character and redirects to food, naps, or windows. — Model type: decoder-only transformer — Parameters: ~3.45M — Language: English (lowercase) — License: MIT — Architecture: RoPE + RMSNorm + SwiGLU + SDPA + tied embeddings — Context length: 256 tokens — Vocabulary: ~1700 BPE tokens, trained from scratch on the Miso dataset — Educational: study a complete tiny-LM training pipeline (tokenizer, dataset, model, training, evaluation) in a readable, under-2000-lines-of-code repository. — Demo: chat with Miso as a character-model toy. — Starting point: fork and train your own character (dog, bird, pirate, etc.) using the same…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: hunt3rx99
Теги: tiny, character, transformer, educational, llm, tiny-llm, character-llm, tokenizer
Лайков: 4 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
