A fine-tune of NVIDIA’s Nemotron-3-Nano-30B-A3B (31.6B-parameter hybrid Mamba-2 + MoE + Attention, thinking-capable) that does one thing: turn rough, underspecified prompts into detailed, structured prompts ready to hand to any AI model. This is the exact model serving koleslaw.ai in production — try it there first if you want to see what it does before downloading 17 GiB. The IQ4NL quant runs on a single 24GB GPU** (RTX 3090 / 4090, L4) with room to spare. Read the full write-up: My «4-bit» quant was 6.2 bits per weight — how the IQ4_NL below was made, why the usual K-quants balloon on this architecture, and the eval that gated it into production. First post in a series on serving this model; the rest lands weekly on the Koleslaw blog. Give it a lazy prompt; it returns a structured one with labeled sections — ROLE / OBJECTIVE / CONTEXT / DELIVERABLES / CONSTRAINTS / OUTPUT FORMAT — with bracketed placeholders where it would otherwise have to invent facts, and an instruction to the answering model to state assumptions rather than guess. Real example — input: «write a script to back up my postgres database» (unedited output from the IQ4_NL quant in production) Nemotron-3-Nano’s…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: thunkaboutit
Теги: gguf, nemotron_h, prompt-engineering, prompt-enhancement, nemotron, moe, lora, ollama
Лайков: 3 | Загрузок: 435
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
