Please used the patched versions- the original had a metadata bug that I had to manually fix also there is no MTP support This repository contains GGUF quants of AlexWortega/SIQ-1-35B using the APEX quantization method developed by LocalAI. These quants are specifically optimized to provide superior perplexity and benchmark scores at lower file sizes compared to standard GGUF quantization methods (like Q4KM or Q5KM). APEX (Adaptive Precision EXtension) uses a custom layer-wise bit allocation strategy. Instead of applying a uniform bit-depth across the entire model, APEX analyzes the sensitivity of each layer and assigns higher precision to critical layers while aggressively compressing less sensitive ones. Result: You get the intelligence of a Q6K model at the file size of a Q4K model, making it perfect for running large models on consumer hardware with partial GPU offloading. SIQ-1-35B consists of 40 transformer layers. Choose the file that best fits your VRAM and System RAM constraints. 1. Download the .gguf file that matches your hardware. 2. Import the file into LM Studio. 3. In the right-hand settings panel, set GPU Offload to the maximum recommended layers for your VRAM (see…
Модальности:
Генерация текста
Области применения:
Логика и рассуждение Диалог / чат
Задача: Генерация текста
Автор: el4
Теги: gguf, llama.cpp, imatrix, moe, reasoning, qwen3, endpoints_compatible, conversational
Лайков: 3 | Загрузок: 320
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
