A tiny tokenizer-free / bit-level (base-2) calculator proof of concept. This repository ships custom modeling.py / configuration.py, so you must load it with trustremotecode=True. The model is trained to read two integers encoded as 10-bit binary inside a structured prompt, and to emit an answer inside a BOR … EOR block (binary output, variable-length). — Bits: 0, 1 — Specials: — BOS=2, EOS=3 — BOI=4, EOI=5 (integer input blocks) — BOR=6, EOR=7 (integer result block) This repo is typically used with the companion inference script inferencebinarycalculator3.py (manual token-by-token loop, no .generate()), supporting: —promptint «int,int» → builds: BOS t0 t1 BOI EOI BOI EOI —printint → extracts the first BOR … EOR block and prints the decoded integer Inputs are 10-bit integers (0..1023). The output can exceed 10 bits (e.g. addition overflow), so the BOR..EOR block is decoded with variable bit-length. The model is tokenizer-free in the sense that it operates directly on bits and a tiny set of structural tokens. This is a POC: sampling settings (temperature, topk) can affect stability. For deterministic behavior, you can lower temperature and/or increase constraints.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: PhysiQuanty
Теги: binaryllm, binary-carry-probagation, binary-level, bit-level, causal-lm, tokenizer-free, base2, binary
Лайков: 4 | Загрузок: 70
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
