groxaxo/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-GGUF-3060-bench - Каталог нейросетей
Генерация текста

groxaxo/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-GGUF-3060-bench

Добавлено:
groxaxo/Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-GGUF-3060-bench

1. A byte-identical mirror of Ternary-Bonsai-2-27B-Abliterated-v2-PQ20-MTP.gguf from BoldingBuilds/Ternary-Bonsai-2-27B-Abliterated-v2-PQ20-MTP-GGUF. We did not change the weights. For the model itself (how the abliteration and the thinking fix were made, and refusal/capability evals), read BoldingBuilds’ card. It is the authoritative source. 2. Reproducible benchmarks on one RTX 3060 12 GB (bench/): a 30-task inbox-automation benchmark across 17 local model/runtime configs, a max-context search, speed/draft-acceptance numbers, the llama.cpp MTP patch we built with, and every raw result file. > Read this before you use the numbers: the only tests we ran on this exact file (abliterated v2 + MTP) were > the speed and max-context tests. The 50-attempt quality table below has no row for this file. Its «B» rows > are a different abliterated build without MTP, and its «J» rows are the non-abliterated base Bonsai 2 with MTP. See > Honest notes. PQ20 is PrismML’s ternary packing. Mainline llama.cpp, Ollama and LM Studio can’t load it.** You need PrismML’s llama.cpp fork (see below). We built PrismML’s llama.cpp fork from the prism-b10685 source and applied…

Модальности:
Генерация текста

Области применения:
Диалог / чат


Задача: Генерация текста
Автор: groxaxo
Теги: gguf, llama.cpp, mtp, speculative-decoding, abliterated, bonsai, ternary, benchmark
Лайков: 3  |  Загрузок: 438

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.