tiiuae/Falcon3-3B-Instruct converted to the LiteRT-LM (.litertlm) format for on-device inference with Google’s LiteRT-LM runtime (the engine behind the official litert-community/ models). Text-only conversion (the Falcon3 decoder; no vision/audio towers). 2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical. Tool calls on litert-lm 0.18.0. Since the 2026-10-14 re-pack (metadata only: llmmodeltype qwen2p5, and the chat template now renders tool calls and tool results), the runtime parses the tool call and the tool result reaches the model. Passing tools works since the re-pack (the earlier template failed with too many arguments). The 2026-10-14 re-pack changes the model type, and with it the runtime’s GPU executor defaults (every non-generic LiteRT-LM model type shares them: allowsrcquantizedfcconvops on, OpenCL hintwaitingforcompletion off, kernel batch size unset); the phone GPU rows on this card were measured with the previous file and have not been re-run — the Mac GPU digit check passed on 0.18.0. The .litertlm bundle carries the tokenizer and the prompt template (Falcon3’s…
Модальности:
Генерация текста
Области применения:
Следование инструкциям
Задача: Генерация текста
Автор: mlboydaisuke
Теги: litert-lm, litert, litertlm, on-device, edge, falcon3
Лайков: 3 | Загрузок: 80
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
