This repo holds streaming EXL3 packs of deepseek-ai/DeepSeek-V4.1-Flash-1017 at revision 1bae0e1088993b9b82de08f59de5be5a34e42170, for mlx-serve. Each pack is a branch. In every pack the trunk loads once, and the routed experts and the Engram tables are read from SSD as they are needed, so a Mac with 128 GB can serve the model. main holds this card, its charts (assets/) and the license. The turboexl3ds41 plugin, which serves every branch but 2.9bpw`, needs an M5-class GPU. 2.81bpw is our pack and the one we recommend. 1.68bpw, 2.25bpw and 2.68bpw are turboderp’s quants, served by the same plugin with its fractional-layer support. 2.9bpw is Mia-AiLab’s quant on the older mlx-stream plugin. Sizes are in GB, where 1 GB is 10^9 bytes. K is the bits per weight of a layer’s routed experts. Each branch’s card lists its files and has its own sweep rows. These earlier OpensourceWTF repos are deprecated stubs that hold no files; their files remain in each repo’s history. This repo replaces them. — OpensourceWTF/DeepSeek-V4.1-Flash-1017-exl3-2.81bpw: held the raw exllamav3 quant that 2.81bpw is repacked from. — OpensourceWTF/DeepSeek-V4.1-Flash-1017-streaming-repack-exl3-2.9bpw: its files…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: OpensourceWTF
Теги: mlx-serve, deepseek_v41, exl3, streaming, moe
Лайков: 3 | Загрузок: 43
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
