A 2-bit TrellisMX build of deepseek-ai/DeepSeek-V4.1-Flash-1017 (revision 1bae0e1088993b9b82de08f59de5be5a34e42170), laid out for two 96 GB RTX PRO 6000 Blackwell cards at tensor parallel 2, with the two Engram tables at 4 bits (about 104 GB) meant to stay in host RAM. The encode and every measurement on this card ran on an 8x NVIDIA B200 node generously donated by Lambda (@LambdaAPI). Thank you, Lambda. Built with the SM120 vLLM image by Local Inference Lab and Luke Alonso, the b12x kernels by Luke Alonso with Martin Vit and other contributors, and Local Inference Lab’s CSF scale format; TrellisMX was inspired by QTIP and EXL3, and its FP8 trellis decode by Luke Alonso’s KQuant/QSRT. Full credits: Credits and CREDITS.md. Released under the MIT License, the same license as the base model (LICENSE). Status: research build. The weights and the reference-path measurements below are complete. The serving image verdictai/deepseek-v41-flash-trellismx:k2-fp8-tp2-20261019 runs this checkpoint at tensor parallel 2 on two RTX PRO 6000 Blackwell 96 GB cards with DSpark speculative decoding and full CUDA graphs; it was booted and benchmarked on that hardware on 2026-10-19 (Speed, below). The…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: brandonmusic
Теги: deepseek_v41, trellismx, trellis-quantization, 2-bit, moe, deepseek, deepseek-v4.1, engram
Лайков: 3 | Загрузок: 350
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
