A 90M parameter language model with modern architecture improvements developed by Mistyoz AI. The easiest way to chat with CosmicFish is using our chat.py script: The chat.py script handles all model loading, generation, and provides the best chat experience with live streaming, repetition penalty, and conversation commands. — Parameters: 91.6M — Architecture: CosmicFish (RoPE, GQA, SwiGLU, RMSNorm) — Context Length: 512 tokens — Vocabulary: 50,257 tokens — Training Data: CosmicSet 2.0 mini — Developer: Mistyoz AI — Repository: MistyozAI/CosmicFish-90M — Format: Safetensors CosmicFish uses several modern improvements over standard transformers: — RoPE (Rotary Position Embeddings): Better position encoding than absolute positions — GQA (Grouped-Query Attention): Reduces memory usage with 4 query g
Модальности:
Генерация текста
Задача: Генерация текста
Автор: MistyozAI
Теги: cosmicfish, language-model, causal-lm, 90m, transformer, rope, gqa, swiglu
Лайков: 4 | Загрузок: 60
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
