Trinity-Large-Thinking is a reasoning-optimized variant of Arcee AI’s Trinity-Large family — a 398B-parameter sparse Mixture-of-Experts (MoE) model with approximately 13B active parameters per token, post-trained with extended chain-of-thought reasoning and agentic RL. This repository contains the NVFP4 quantized weights of Trinity-Large-Thinking for deployment on NVIDIA Blackwell GPUs. For full model details, benchmarks, and usage guidance, see the main Trinity-Large-Thinking model card. — Scheme: NVFP4 (nvfp4expertsonly — MoE expert weights only, attention and dense layers remain BF16) — Tool: NVIDIA ModelOpt — Calibration: 2048 samples, seqlength=4096 — KV cache:** Not quantized — Both Hopper (via Marlin) and Blackwell B300 node — vLLM 0.18.0+ Requires vLLM >= 0.18.0. Native FP4 compute requires Blackwell GPUs; older GPUs fall back to Marlin weight decompression automatically. > Note (For Blackwell pip installs): If installing vLLM via pip on Blackwell rather than using Docker, native FP4 kernels may produce incorrect output due to package version mismatches. As a workaround, force the Marlin backend: > > > > Marlin decompresses FP4 weights to BF16 for compute, providing the…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: arcee-ai
Теги: afmoe, reasoning, agentic, tool-calling, thinking, moe, nvfp4, modelopt
Лайков: 4 | Загрузок: 1,009
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
