A 27B model that texts like a person and still does the work, in FP8 for 32 to 48 GB cards. No system prompt needed. Results, examples and how it was made are on the main card: Qwen3.8-27B-Humanlike-Chat-GGUF. «Base» here always means huihui-ai/Huihui-Qwen3.8-27B-abliterated, an abliterated Qwen3.8-27B. It is not the official Qwen release. On the H100, —linear-backend cutlass uses vLLM’s precompiled block-FP8 kernel and the server is up in about 3 minutes. vLLM 0.27.1’s default kernel on that card builds code at startup and needs nvcc in the container. On other GPUs, drop —linear-backend cutlass and vLLM picks a kernel for your card. Thinking is on by default. Send chattemplatekwargs: {«enablethinking»: false} per request for the fastest replies. Sampling: temperature 1.0, topp 0.95, top_k 20. Same layout as Qwen’s official Qwen/Qwen3.8-27B-FP8: FP8 e4m3 weights in 128 x 128 blocks with BF16 scales, dynamic activation scaling. 407 linear layers are FP8: MLP, attention, the linear-attention projections and the MTP head’s layers. These stay BF16: embeddings, lmhead, norms, the linear-attention gate tensors (inproja, inprojb, Alog, dtbias`, conv) and the vision tower. Made from the…
Модальности:
Генерация текста
Области применения:
Диалог / чат Вызов функций (Tool use)
Задача: Генерация текста
Автор: LessThanThreeAI
Теги: qwen3_5, image-text-to-text, vllm, sglang, fp8, qwen3.8, conversational, tool-calling
Лайков: 3 | Загрузок: 434
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
