This is the flagship release of CantoneseLLM v2. It is a 30.5B-parameter mixture-of-experts model (3.3B activated) based on Qwen3-30B-A3B that reasons and answers in Hong Kong Cantonese. It is the final checkpoint of a five-stage pipeline: continual pre-training → chat-vector merge → supervised fine-tuning → DPO → two-stage RLVR. The full development history, including the stages that failed and why, is documented in the technical report. 📄 Paper: CantoneseLLM v2: Reasoning in a Low-Resource Language (arXiv:2609.06970) Open models prompted in Cantonese answer in Cantonese but reason in another language. Here is the same translated GSM8K probe, under a system prompt that explicitly asks for Hong Kong Cantonese. Chat-vector merged checkpoint — opens in Cantonese, then slides into Written Chinese and ends up discussing the output language instead of thinking in it: > …現在，用香港廣東話回應。…在回應中，要用口語廣東話。 > > (…Now, respond in Hong Kong Cantonese. … In the response, colloquial Cantonese should be used.) This model — reasons in Cantonese end to end, with the Cantonese classifier 隻 zek3 and colloquial forms such as 剩低 zing6 dai1: > 首先，我要拆解呢條數學題嘅步驟。題目話 Janet 每日生 16 隻蛋，佢朝早食 3 隻，焗鬆餅用咗 4…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: hon9kon9ize
Теги: qwen3_moe, cantonese, hong-kong, yue, reasoning, grpo, rlvr, moe
Лайков: 3 | Загрузок: 550
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
