LiteRT is Google’s on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with literttorch.convert` matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05). Measured on device (edge-compat): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 10.1 tok/s · prefill 206 tok/s · TTFT 1.18 s · all 1243 ops delegated (2026-08-24); Pixel 8a · LiteRT-LM 0.16.0 · GPU · decode 12.0 tok/s · prefill 84 tok/s · TTFT 540 ms · all 1243 ops delegated (2026-08-19); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 29.7 tok/s · prefill 106 tok/s · TTFT 2.14 s (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/qwen25-coder-1.5b-int4/CARD.md Qwen/Qwen2.5-Coder-1.5B-Instruct converted to the LiteRT-LM (.litertlm) format for on-device inference with Google’s LiteRT-LM runtime. Requires litert-lm ≥ 0.16. A 1.54B code model at 1.12 GB — small enough that a phone-class device runs it at conversational speed rather than merely running it. Code assistance is the use case that most needs that: it returns…
Модальности:
Генерация текста
Области применения:
Генерация кода Следование инструкциям
Задача: Генерация текста
Автор: litert-community
Теги: litert-lm, litert, litertlm, on-device, edge, code
Лайков: 3 | Загрузок: 898
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
