ObsidianSmall-Base is an 8.7M-parameter English base language model pretrained from scratch on 3,999,989,760 tokens. It is the first public checkpoint in this model family built entirely from Multiscreen blocks. > This is a base completion model, not an instruction-tuned or chat-aligned assistant. This model implements and adapts the Multiscreen architecture introduced in Screening Is Enough by Ken M. Nakanishi. Multiscreen replaces conventional self-attention and MLP/FFN sublayers with its own sequence-mixing blocks. The implementation may differ from the architecture and training configuration described in the original paper. Zero-shot task results were measured with lm-evaluation-harness. Full WikiText-2 evaluation using rolling log-likelihood with a maximum context length of 1,024 tokens. The repository includes the required custom PyTorch/LitGPT runtime. It is not currently compatible with transformers.AutoModelForCausalLM. CPU execution is available with MULTISCREENBACKEND=torch, —backend torch, and —device cpu`. The bundled evaluator accepts any compatible lm-evaluation-harness task through —tasks. Custom continuation-style multiple-choice datasets can be evaluated with…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Dream-W
Теги: multiscreen, base-model, causal-lm, triton, litgpt, custom-architecture, custom_code, en
Лайков: 4 | Загрузок: 229
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
