This repository is a community conversion of Qwen/Qwen3.8-27B to the LiteRT-LM (.litertlm) format, made with litert-torch (commit 115a1360 with the Qwen3.5 hybrid patch of hf-to-litertlm) for Google’s LiteRT-LM runtime 0.18.1. On a Mac Studio M4 Max the Mixed INT4 file prefills 1,024 tokens at 175.5 tok/s and decodes at 18.2 tok/s on the GPU (measured 2026-10-22). It is one 15.4 GB file holding the text decoder, for desktop-class machines: it was measured on a Mac only. Qwen introduces Qwen3.8 as «the most capable generation in the Qwen open-model family to date», built «on the architectural foundation of Qwen3.5», and Qwen3.8-27B as «a compact, deployment-friendly dense model: a native vision-language model that understands images and videos, with flexible thinking control». Qwen’s card reports GPQA Diamond 89.2 and IFBench 79.5 for it (vendor-reported, not run here) and recommends sampling with temperature 1.0, topp 0.95 and topk 20 in the thinking mode and temperature 0.7, topp 0.8, topk 20 and presence_penalty 1.5 in the non-thinking mode. The architecture is Qwen3.5’s hybrid at 27B: 64 layers in 16 groups of three GatedDeltaNet linear-attention layers and one gated…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: litert-community
Теги: litert-lm, litert, litertlm, on-device, edge, hybrid, gated-deltanet, qwen3_5
Лайков: 3 | Загрузок: 275
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
