ROCmFP4 Strix-Lean quant of mistralai/Leanstral-1.5-119B-A6B for AMD Radeon / Ryzen AI Max+ (gfx1151) inference via the public charlie12345/ROCmFPX llama.cpp fork. Mistral ships Leanstral FP8-native without HF-format weights or an official GGUF. This quant is converted from the community BF16 transformers dump, then quantized with ROCmFP4 Strix-Lean kernels. The base model includes a Pixtral vision tower + multimodal projector. This GGUF is language-model only (vision tensors omitted at convert time). It is not a multimodal / vision checkpoint. A text-only deepseek2 GGUF of this model family is a known-good pattern (e.g. public Q4KM text-only releases). — -dio is mandatory on large unified-memory loads. The mmap path can hang around ~120GB RSS with GTT stuck at 0. — Use a ROCmFP4-capable build of charlie12345/ROCmFPX (or equivalent) that implements Q40ROCMFP4STRIXLEAN. — Load one** ~60GB model at a time on 128GB unified memory. Hardware: Ryzen AI Max+ 395 (gfx1151), ROCm, 128GB unified memory. Server flags identical: -ngl 999 -fa on -dio —no-warmup —jinja —ctx-size 65536 —cache-type-k q80 —cache-type-v q80 —parallel 1. Baseline: public-style Q4KM text-only GGUF (~68 GiB).…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: kingjones777
Теги: gguf, rocm, strix, lean4, moe, deepseek2, quantization, amd
Лайков: 3 | Загрузок: 109
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
