kanocz/Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED-FP8-vLLM - Каталог нейросетей
Генерация текста

kanocz/Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED-FP8-vLLM

Добавлено:
kanocz/Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED-FP8-vLLM

Block-wise FP8 (E4M3) quantization of DavidAU/Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED, optimized for vLLM inference. — Full 262K context on a single RTX 3090 (24 GB) — Vision (multimodal) support preserved — Thinking/reasoning mode with qwen3 parser — Tool calling support — ~11.3 GB model footprint (vs ~19 GB in BF16) — FP8 KV cache compatible for additional memory savings Quantization was performed on GPU (CUDA) to ensure correct FP8 casting. The format matches lovedheart/Qwen3.5-9B-FP8 conventions. — RTX 3090 (24 GB) — Ampere, no native FP8, uses Marlin kernel for weight decompression — Should work on RTX 4090, A100, H100, and other GPUs with 24+ GB VRAM 1. Qwen/Qwen3.5-9B — Base model (Apache 2.0) 2. trohrbaugh/Qwen3.5-9B-heretic-v2 — Heretic uncensored fine-tune (Apache 2.0) 3. DavidAU/Qwen3.5-9B-Claude-4.6-HighIQ-THINKING-HERETIC-UNCENSORED — Claude 4.6 thinking distillation (Apache 2.0) 4. This repo — FP8 block-wise quantization for vLLM — Qwen Team (Alibaba) for the Qwen3.5 architecture — DavidAU for the fine-tuned model — trohrbaugh for the Heretic uncensored base — lovedheart for the Qwen3.5-9B-FP8 reference format

Модальности:
Генерация текста Компьютерное зрение

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: kanocz
Теги: qwen3_5, image-text-to-text, qwen3.5, fp8, vllm, quantized, thinking, uncensored
Лайков: 4  |  Загрузок: 891

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.