MBZUAI/LLaVA-Meta-Llama-3-8B-Instruct-lora

This repository features LLaVA v1.5 trained with the Meta-Llama-3-8B-Instruct LLM. This integration aims to leverage the strengths of both models to offer advanced vision-language understanding. — Pretraining: Only Vision-to-Language projector is trained. The rest of the model is frozen. — Fine-tuning: LLM is LoRA fine-tuned. Only the vision-backbone (CLIP) is kept frozen. — Note: The repository contains projector and LoRA weights. — Base Large Language Model (LLM): Meta-Llama-3-8B-Instruct — Base Large Multimodal Model (LMM): LLaVA-v1.5 — Pretraining Dataset: LCS-558K — Fine-tuning Dataset: LLaVA-Instruct-665K

Модальности:
Генерация текста

Области применения:
Следование инструкциям

Задача: Генерация текста
Автор: MBZUAI
Теги: llava_llama, endpoints_compatible
Лайков: 3  |  Загрузок: 22

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.