NeuronSpark-0.9B is a 0.87-billion parameter language model built entirely on Spiking Neural Networks (SNNs). Unlike conventional Transformer-based LLMs that rely on attention mechanisms, NeuronSpark replaces the entire computation backbone with biologically-inspired spiking neurons, achieving language modeling through membrane potential dynamics, surrogate gradient training, and adaptive computation (PonderNet). This is the pretrained base model (85,000 steps on a small subset of Seq-Monkey corpus). > Note on training data: Due to limited compute resources (single DGX Spark), this model was trained on only ~85K steps with a small fraction of the full Seq-Monkey 10B-token corpus. Despite the minimal training data, the model demonstrates emergent language capabilities — validating the architectural viability of pure SNN language models. We plan to continue scaling with more data and compute in future work. For the instruction-tuned chat version, see NeuronSpark-0.9B-Chat. — Pure SNN: No attention, no standard MLP — all computation via PLIF (Parametric Leaky Integrate-and-Fire) neurons — Membrane Potential Leakage Activation: PLIFNode outputs (1-β)·Vpost (leak current), naturally…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: Brain2nd
Теги: neuronspark, snn, spiking-neural-network, neuromorphic, conversational, custom_code, zh
Лайков: 3 | Загрузок: 22
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
