This is a deliberately contaminated same-item benchmark demonstration. It was post-trained on benchmark items and their known answers to show that benchmark scores can be manipulated by leakage. The scores below are not clean generalization results and must not be compared as ordinary model capability. — Base: mlx-community/gemma-4-e4b-it-4bit — Adapter format: MLX LoRA — Best local adapter source: /Users/mac/adapters/superbenchmaxx-e4b-gpqa-repair-v3-balanced-20260511T054208Z Full leaked MLX evaluation on the contaminated 567-item set: — Overall: 563/567 = 99.2945% — BixBench: 205/205 = 100.0% — GPQA Diamond: 198/198 = 100.0% — HumanEval: 160/164 = 97.561% — Overall: 567/567 = 100.0% — BixBench: 205/205 = 100.0% — GPQA Diamond: 198/198 = 100.0% — HumanEval: 164/164 = 100.0 MLX expects adapterpath` to be a local folder, not a Hub repo id string.
Модальности:
Генерация текста
Задача: Генерация текста
Автор: Jiunsong
Теги: mlx, lora, gemma, contaminated-benchmark-demo, leaked-benchmark-demo
Лайков: 4 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
