Project Page | Paper | Code | 🤗Demo | 🤗TIGERScore-7B | 🤗TIGERScore-13B We present TIGERScore, a Trained metric that follows Instruction Guidance to perform Explainable, and Reference-free evaluation over a wide spectrum of text generation tasks. Our metric is based on LLaMA-2, trained on our meticulously curated instruction-tuning dataset MetricInstruct which covers 6 text generation tasks and 23 text generation datasets. Existing automatic metrics are lagging and suffer from issues like 1) Dependency on references, 2) Limited to specific domains, 3) Lack of attribution. Contrary to them, TIGERScore is designed to be driven by natural language instruction and provide detailed error analysis to pinpoint the mistakes in the generated text. Specifically, TIGERScore takes an instruction, an associated input context along with a hypothesis output that might contain errors. Then, TIGERScore will evaluate this hypothesis output and list several errors, each consisting of the error location, aspect, explanation and penalty scores (score reduced, starting from 0). The sum of the reduced scores is taken as the overall rating of this output. As a reference-free metric, its correlation can…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: TIGER-Lab
Теги: llama, text evaluation, metric, llm metric, text2text-generation, en, zh, ru
Лайков: 2 | Загрузок: 25
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.