RKLLM conversion of Qwen/Qwen3-1.7B for Rockchip RK3588 NPU inference. Converted with RKLLM Toolkit v1.2.3, which includes full thinking mode support — the model produces … reasoning blocks when used with compatible runtimes. Previous Qwen3-1.7B RKLLM conversions on HuggingFace were built with Toolkit v1.2.0, which predates thinking mode support (added in v1.2.1). The chat template baked into those .rkllm files does not include the trigger, so the model never produces reasoning output. This conversion uses Toolkit v1.2.3, which correctly embeds the thinking-enabled chat template into the model file. Qwen3-1.7B is a hybrid thinking model. When served through an OpenAI-compatible API that parses tags, reasoning content appears separately from the final answer — enabling UIs like Open WebUI to show a collapsible «Thinking…» section. — Orange Pi 5 Plus — RK3588, 16GB RAM, Armbian Linux — RKNPU driver 0.9.8 — RKLLM Runtime v1.2.3 The RKLLM runtime requires two things for thinking mode to work: The stock llmdemo.cpp uses memset(&rkllminput, 0, …) which defaults enablethinking to false. You must** add one line: If using the Python ctypes API (flaskserver.py / gradioserver.py), set it…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение
Задача: Генерация текста
Автор: GatekeeperZA
Теги: rkllm, rk3588, npu, rockchip, qwen3, thinking, reasoning, quantized
Лайков: 3 | Загрузок: 403
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
