GitHub Repo | Technical Report 👋 Join us on Discord and WeChat BitCPM-CANN-3B-unquantized is the unquantized QAT (Quantization-Aware Training) checkpoint of BitCPM-CANN-3B, designed for continued pre-training and fine-tuning. It preserves full-precision latent weights with ternary fake quantizers (weights → {-1, 0, 1} with group-wise scaling, trained via STE) defined in modeling.py, enabling the model to keep learning under quantization constraints. For technical details, see our Technical Report. > ⚠️ This model is NOT for direct inference. For inference, use the pseudo-quantized version: openbmb/BitCPM-CANN-3B. The only requirement is that the forward pass must go through the bundled modeling.py (which contains the ternary fake quantizer). Load with trustremotecode=True and do NOT replace or bypass the model’s forward logic. We provide ready-to-use training scripts in the example directory (using the 1B model as an example): — Continued pre-training: example/run.sh + example/train.py — SFT (Supervised Fine-tuning): example/runsft.sh + example/trainsft.py Any framework that supports HuggingFace model loading with custom code can be used, such as LLaMA Factory, HuggingFace…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: openbmb
Теги: llama, conversational, custom_code, zh, en, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 9,728
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
