Nautile-370M is a 371M-parameter hybrid language model for reasoning and language understanding. Its backbone alternates two SeqCond Attention (SCA) layers — a spectral sequence operator grounded in the derivative of the empirical characteristic function — with one standard transformer layer, giving a 2:1 SCA/Transformer ratio across 24 layers. The model was pretrained on ~0.8T tokens on a single TPU v4-64 pod slice (Google TRC program), then post-trained with reinforcement learning on a single NVIDIA DGX Spark. Nautile-370M is designed for language understanding, common-sense reasoning, and classification tasks: — Sentiment analysis, intent detection, topic labeling — Structured information extraction — Fine-tuning for domain-specific classification — Large-scale opinion modeling (thousands of instances in parallel on modest hardware) It is not designed for open-ended multi-turn chat, code generation, or knowledge-intensive QA — tasks that benefit more from scale than from architect We evaluate generation quality using an LLM-as-a-judge setup (GPT-4.1). For a diverse set of prompts covering factual knowledge, commonsense reasoning,…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение
Задача: Генерация текста
Автор: trickstr-ai
Теги: seqcond, hybrid, reasoning, spectral, trickstr, custom_code, en
Лайков: 3 | Загрузок: 40
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
