True O(1) Memory Test-Time Compute via Continuous-State Dark Loops. A 2.8B-parameter Mamba SSM fine-tuned for multi-step latent reasoning. The model executes deductive logic in its continuous hidden state using ==== spacer tokens as internal clock cycles — no KV-cache growth, no visible chain-of-thought tokens. Phase 10 adds: OO-domain specialization via LoRA adapters, geometric degeneration suppression via the Proprioception Gate, and named inference mode routing. > Repetition note: Default greedy mode averages ~31% bigram repetition (Mamba SSM characteristic). Apply repetitionpenalty ≥ 1.2 or use oodomain inference mode (T=0.4, rep=1.4, 4-gram stop) which achieved 0% repetition across 12 OO ontology probes. The engine supports four named sampling presets, auto-detected from prompt prefix: Use stateful_engine.py for the full O(1) MambaCache loop engine with auto-routing. Papers that shaped this work: — Gu & Dao (2023) — Mamba: Linear-Time Sequence Modeling. arXiv:2312.00752 — Hao et al., Meta (2024) — COCONUT: Continuous Latent Reasoning. arXiv:2412.06769 — Goyal et al., Google (2023) — Pause Tokens. arXiv:2310.02226 — Graves (2016) — Adaptive Computation Time for RNNs.…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: batteryphil
Теги: mamba, state-space-model, latent-reasoning, reinforcement-learning, mathematical-logic, oo-domain, lora, proprioception
Лайков: 3 | Загрузок: 28
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
