Asterisk is a research implementation that combines the ASPP (Adjacency-Structured Parallel Propagation) operator with standard attention mechanisms to enhance the SmolLM2-135M model. The model implements a hybrid architecture that fuses graph-based local reasoning (ASPP) with global attention for improved expressiveness on structured reasoning tasks. — Base Model: SmolLM2-135M-Instruct — Architecture: Hybrid ASPP-Attention (30 hybrid layers) — Parameters: 171.2M (35M additional ASPP parameters) — Training: Supervised Fine-Tuning on Capybara dataset — Framework: Transformers 4.57.6, TRL 0.27.0 The Asterisk Operator performs local parallel state evolution through point-wise transformations: This is then gated and fused with standard Llama attention outputs: — Dataset: Capybara (conversational instruction-following) — Optimi
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: NoesisLab
Теги: asterisk, aspp, hybrid-architecture, graph-reasoning, sft, trl, conversational, custom_code
Лайков: 4 | Загрузок: 26
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
