LiteRT is Google’s on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with literttorch.convert` matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05). Measured on device (edge-compat, smollm3-3b): Galaxy S26 · LiteRT-LM prebuilt-adac974b · GPU · decode 24.4 tok/s · prefill 382 tok/s · TTFT 2.72 s · all 1509 ops delegated (2026-10-05); Pixel 8a · LiteRT-LM 0.16.0 · GPU · decode 7.7 tok/s · prefill 12 tok/s · TTFT 1.56 s · all 1476 ops delegated (2026-08-17); Mac Studio M4 Max · LiteRT-LM 0.17.1 · GPU · decode 131.8 tok/s · prefill 1481 tok/s · TTFT 185 ms · all 1509 ops delegated (2026-10-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/smollm3-3b/CARD.md Measured on device (edge-compat, smollm3-3b-int8): Galaxy S26 · LiteRT-LM prebuilt-adac974b · GPU · decode 16.0 tok/s · prefill 452 tok/s · TTFT 2.33 s · all 1512 ops delegated (2026-10-05); Mac Studio M4 Max · LiteRT-LM 0.17.1 · GPU · decode 106.0 tok/s · prefill 1641 tok/s · TTFT 168 ms · all 1512 ops delegated…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: litert-community
Теги: litert-lm, litert, litertlm, on-device, edge, smollm3
Лайков: 3 | Загрузок: 2,403
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
