Unofficial LiteRT-LM conversion of Nanbeige/Nanbeige4.2-3B, pinned to source revision 451ed48c3273ecef7ea8faaa43c31ce529763bb1. The bundle preserves Nanbeige’s exact two-loop execution over 22 shared physical decoder layers, including midpoint RMSNorm and 44 logical KV-cache slots. It was exported with the official LiteRT Torch and LiteRT-LM tooling. — LiteRT-LM container version: 1.5.0 — quantization: dynamic weight INT8, activation FP32 — maximum cache length: 4,096 or 65,536 tokens, as named — signatures: prefill128 and decode — tokenizer: packaged SentencePiece model and upstream Jinja chat template — model section type: tfliteprefilldecode — logical cache: 44 layers / 88 K and V tensors The 4K file is the mobile-oriented default. The 64K file uses a dynamic cache dimension in all 88 K/V inputs for both signatures. Its 1,024-token export shape is only a trace/example size: prefill128` keeps the 128-token compute chunk fixed while the cache and attention-mask widths resize dynamically. The package metadata caps the runtime at 65,536 tokens. At the full cap, FP32 K/V state alone is approximately 23.6 GB (22 GiB), before model weights, output-cache tensors, and runtime workspace.…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Tdamre
Теги: litert, litert-lm, tflite, nanbeige, android
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
