Qwen3.5-0.8B-Q4_K_M-GGUF
A Q4KM GGUF of Qwen/Qwen3.5-0.8B for on-device inference on any iPhone, iPad, or Apple Silicon Mac via llama.cpp...
A Q4KM GGUF of Qwen/Qwen3.5-0.8B for on-device inference on any iPhone, iPad, or Apple Silicon Mac via llama.cpp...
This is a MXFP8 quantization of Qwen/Qwen3.5-35B-A3B, generated by AutoRound. Please follow the license of the original model....
> Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no...
EXL3 pack of deepseek-ai/DeepSeek-V4.1-Flash. Routed experts are 2.0 bpw MCG. In the default revision they are re-encoded with...
W4AFP8 quantization of zai-org/GLM-5.3-Flash: routed MoE expert weights in INT4 (group-128, symmetric, AWQ-calibrated) with static per-tensor FP8 activation...
Spark-X2.5-4B at 3.41 GiB — the size-first bundle. Fastest of the pair, and the one to run if...
GGUF quantizations of tencent/Hy4-preview, made with llama.cpp. 780B total parameters, 78 layers. DeepSeek-style MLA attention (q-LoRA rank 2048,...
☕ If this GGUF made your day easier, a coffee would make mine. Send a coffee ☕ I...
Qwen3.6-35B-A3B by Alibaba Qwen Team — a 35B-parameter Mixture-of-Experts (MoE) model with only 3B active parameters per token....
A GPTQ 4-bit (group size 128, symmetric) quantization of IFM/K2-Horizon-MoVA-36B-A4B, MBZUAI IFM’s sparse frontier model with MoVA (Mixture-of-Values)...