A 13 GB download: 12,982,426,694 bytes = 12.98 GB = 12.09 GiB. Weights and codebooks are 12,958,885,485 B (12.069 GiB); the tokenizer and configs add the remaining ~23 MB. On a 16 GB (= 16 GiB) consumer Blackwell card (RTX 5080-class) that leaves ~3.9 GiB for CUDA context, KV cache and activations. > On the two size numbers. 13 GB and 12.1 GiB are the same bytes in > different units — decimal GB (what your disk and the Hub report) and binary > GiB (what nvidia-smi and the allocator report). This model is named for the > 13 GB figure because that is the number you have to fit, and because a > «12.1» in the title reads as headroom that is not there. Every table below > states bytes exactly so neither unit has to be trusted. Every Linear in this model was given its own format and its own codebook size, chosen by AQUA — an allocator that prices each candidate against the loss it actually costs, rather than assigning one uniform precision to the whole network. The result is a 19-rung ladder used unevenly on purpose: 355 FP8CBK28 94 FP8CBK48 20 NVFP4CBK16 8 NVFP4CBK12 8 NVFP4CBK14 6 NVFP4CBK18 4 FP8CBK32 1 FP8CBK40 Two bit-rates are quoted because one number cannot…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: rdtand
Теги: vllm, qwen3_5_text, prismaquant, aqua, gridbook, codebook-quantization, mixed-precision, conversational
Лайков: 4 | Загрузок: 715
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
