BNB NF4 4-bit quantization of techwithsergiu/Qwen3.5-text-9B — a text-only derivative of Qwen/Qwen3.5-9B. No visual tower — text input only. This is the recommended base for Unsloth LoRA text fine-tuning: smaller VRAM footprint, no visual-dependency complexity, cleaner adapter targeting. Inference has been verified. LoRA fine-tuning docs are pending — see Fine-tuning section below. — Visual tower removed (same as Qwen3.5-text-9B) — Text backbone quantized to BNB NF4 double-quant (bnb4bitquanttype=nf4, bnb4bitcomputedtype=bfloat16) — lmhead.weight kept at bf16** for output quality / stability The visual tower scales with model size (~0.19 GB for 0.8B, ~0.62 GB for 2B/4B, ~0.85 GB for 9B). BNB text-only models are roughly 34% of the original f16 size (4B example: 9.32 GB → 3.12 GB). This is the primary training target for text LoRA fine-tuning. Training pipeline: github.com/techwithsergiu/qwen-qlora-train QLoRA (Unsloth + TRL + PEFT) · rank 16–64 · validated on RTX 3070 8 GB For VLM (image + text) fine-tuning of the full model, see: unsloth.ai/docs/models/qwen3.5/fine-tune Converted using qwen35-toolkit — a Python toolkit for BNB quantization, visual tower removal, verification and…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: techwithsergiu
Теги: qwen3_5_text, techwithsergiu, conversational, endpoints_compatible, 4-bit, bitsandbytes
Лайков: 4 | Загрузок: 1,546
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
