Since only two formats are useful, I have converted model into those formats only. MegaBeam-Mistral-7B-300k is a fine-tuned Mistral-7B-Instruct-v0.2 language model that supports input contexts up to 320k tokens. MegaBeam-Mistral-7B-300k can be deployed on a single AWS g5.48xlarge instance using serving frameworks such as vLLM, Sagemaker DJL endpoint, and others. Similarities and differences beween MegaBeam-Mistral-7B-300k and Mistral-7B-Instruct-v0.2 are summarized below: InfiniteBench: Extending Long Context Evaluation Beyond 100K Tokens InfiniteBench is a cutting-edge benchmark tailored for evaluating the capabilities of language models to process, understand, and reason over super long contexts (100k+ tokens). We therefore evaluated MegaBeam-Mistral-7B-300k, Mistral-7B-Instruct-v0.2, Llama-3-8B-Instruct-262k, and Llama3-70B-1M on InfiniteBench. The InfiniteBench authors also evaluated SOTA proprietary and open-source LLMs on InfiniteBench. We thus combined both results in the table below. The 12 evaluation tasks are summarized below (as per InfiniteBench)) On an AWS g5.48xlarge instance, upgrade vLLM to the latest version as per documentation on vLLM. Important Note — We have…
Модальности:
Генерация текста
Области применения:
Диалог / чат
Задача: Генерация текста
Автор: xbruce22
Теги: gguf, mistral, conversational
Лайков: 3 | Загрузок: 77
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
