Trained autonomously by the ml-intern Claude Code skill on a single RTX A6000. This is a ~130M-parameter down-scale of the DeepSeek-V4 architecture (MQA + 512-headdim semantics, mHC manifold-constrained hyper-connections, hash routing, sqrtsoftplus gate, swiglulimit, O grouped low-rank, MTP, attention-sink), trained on the roneneldan/TinyStories dataset. The whole pipeline — paper extraction, architectural down-scale, training script, bug-fixing (a real causal mask leak via the CSA Compressor was caught during a v1 run and patched in v2 by disabling the compressor), self-verification, and this very repo — was produced by a Claude Code session running the ml-intern skill with no human in the loop after the initial prompt. > Once upon a time, the little girl was walking through the woods. She saw lots of interesting things and started to explore. Suddenly, she saw a big rock! She was so excited and said, «Wow! I’m so hungry!» Just then, a friendly fox came buzzing by. The fox said, «I have something…» The snippet below is the one in loadtest.py in this repo. It was run end-to-end against this published repo on a fresh machine — python loadtest.py prints LOADTEST: PASS` and the…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: AlexWortega
Теги: ml-intern, pretraining, deepseek, deepseek-v4, tinystories, causal-lm, en
Лайков: 3 | Загрузок: 98
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
