A compact-but-capable ≈150M parameter causal LM that replaces dot-product attention with metric-native attention and augments sequence geometry with BlackHoleRoPE (a learnable, stable RoPE variant). Designed to train and run on modest hardware (CPU-first friendly) while staying fully compatible with 🤗 Safety & responsibility This is a research model. It was trained on public datasets and may produce incorrect or biased content. Do not rely on it for advice or sensitive decisions. Built with 🤗 Transformers and a metric-first rethinking of attention. BlackHoleRoPE draws inspiration from symplectic/rotational encodings and bounded-energy dynamics. DistilQwen Collection — Our only BF16 series. Proof-weighted distillation from Qwen3-30B-A3B → 1.7B and 0.6B on H100. Three teacher variants (Instruct, Thinking, Coder), nine models, 2,788 combined downloads. The rest of the portfolio proves structure beats scale on CPU. This collection shows what happens when you give the methodology real hardware. Full methodology: Structure Over Scale (DOI: 10.57967/hf/8165)
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: reaperdoesntknow
Теги: tensorboard, moa_metric, trl, sft, convergentintel, conversational, en, endpoints_compatible
Лайков: 3 | Загрузок: 2,748
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
