QwQ-32B-Preview is an experimental research model developed by the Qwen Team, focused on advancing AI reasoning capabilities. As a preview release, it demonstrates promising analytical abilities while having several important limitations: 1. Language Mixing and Code-Switching: The model may mix languages or switch between them unexpectedly, affecting response clarity. 2. Recursive Reasoning Loops: The model may enter circular reasoning patterns, leading to lengthy responses without a conclusive answer. 3. Safety and Ethical Considerations: The model requires enhanced safety measures to ensure reliable and secure performance, and users should exercise caution when deploying it. 4. Performance and Benchmark Limitations: The model excels in math and coding but has room for improvement in other areas, such as common sense reasoning and nuanced language understanding. Specification: — Type: Causal Language Models — Training Stage: Pretraining & Post-training — Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias — Number of Parameters: 32.5B — Number of Paramaters (Non-Embedding): 31.0B — Number of Layers: 64 — Number of Attention Heads (GQA): 40 for Q and 8…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: unsloth
Теги: qwen2, chat, qwen, conversational, en, text-generation-inference, endpoints_compatible, 4-bit
Лайков: 4 | Загрузок: 169
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
