This model converted from DeepSeek-V3-0324 to BF16. Therefore, we have provided only the command to convert from Windows and information related to ollama. The Windows environment is much faster than the WSL2(Ubuntu-22.04) environment, provided you have sufficient memory or virtual memory. The Linux environment hasn’t been tested. If you are in a Linux or WSL environment, please refer to huihui-ai/DeepSeek-R1-bf16. 1. Download deepseek-ai/DeepSeek-V3-0324 model, requires approximately 641GB of space. 3. Convert to BF16, requires an additional approximately 1.3 TB of space. Here, you need to download the transformation code from the «inference» folder of deepseek-ai/DeepSeek-V3 1. Use the llama.cpp conversion program to convert DeepSeek-V3-0324-bf16 to gguf format, requires an additional approximately 1.3 TB of space. 2. Use the llama.cpp quantitative program to quantitative model (llama-quantize needs to be compiled.), other quant option. Convert first Q2_K, requires an additional approximately 227 GB of space. If you like it, please click ‘like’ and follow us for more updates. You can follow x.com/support_huihui to get the latest model information from huihui.ai.
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: huihui-ai
Теги: deepseek_v3, conversational, custom_code, text-generation-inference, endpoints_compatible
Лайков: 4 | Загрузок: 25
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
