TernaryBananaMind-10M is a compact decoder-only base language model from BananaMind. Its inference checkpoint stores projection and output weights as packed ternary values and input embeddings as signed 8-bit values. The intended model ID is BananaMind/TernaryBananaMind-10M. The model has 8,428,032 logical weights, a 4,096-token context window, and a custom 2,048-token digit-aware byte-level BPE tokenizer. Its model.safetensors file is 2,175,676 bytes (about 2.07 bits per logical weight, including scales, float normalization weights, and file overhead). The model name is a size class; the exact logical weight count is 8.43M. We have trained TernaryBananaMind-10M on our BananaAll training framework. See it at https://github.com/BananaMind/BananaAll/ to train your own model simply. The packed weights are decoded to temporary tensors for matrix multiplication. 2.07 bits per weight describes checkpoint storage, not arithmetic precision or peak inference memory. Packed weights are registered as buffers, so sum(p.numel() for p in model.parameters()) reports only the 6,656 trainable normalization parameters. Use the logical weight count above for model-size comparisons. The custom 2k…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: BananaMind
Теги: bananaall, causal-lm, language-model, base-model, small-language-model, bananamind, bananamind2, ternary
Лайков: 3 | Загрузок: 205
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
