On-device INT4 builds of Qwen/Qwen3-0.6B in the .litertlm format for Google’s LiteRT-LM runtime (Android / iOS / desktop / embedded). At ~332 MB each (≈7× smaller than the fp32 source), these load fast and keep a small memory footprint on phones. Qwen3 is a hybrid reasoning model. Because LiteRT-LM bakes the prompt format into the file, the thinking mode is fixed per file, so this repo ships both: — The standard file embeds Qwen3’s Jinja chat template (faithful ChatML; if your runtime can pass enablethinking, it stays controllable). — The no-think file prefills a closed empty ` into the assistant turn, so the model answers directly. This was built for a wearable health-assistant where a phone streams short replies to a watch. Converted with litert-torch (the generative API, formerly ai-edge-torch) from the original PyTorch checkpoint. > Multiple prefill signatures matter for latency: with a single large signature > every prompt is padded to it. The exponential set above lets a short prompt run > a tiny prefill instead of always paying the 1024-token cost. Standard Qwen3 ChatML. The no-think file pre-closes thinking on the assistant side: Stop tokens and the prompt format are…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: litert-community
Теги: litert-lm, litertlm, on-device, edge, android, int4, quantized, qwen3
Лайков: 3 | Загрузок: 4,151
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
