 Llama 3 is a family of LLMs. The model is quantized to w4a16 (4-bit weights and 16-bit activations) and part of the model is quantized to w8a16 (8-bit weights and 16-bit activations) making it suitable for on-device deployment. This is based on the implementation of Llama-v3.2-3B-Instruct found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here. Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device. Follow the GenieX quickstart to install GenieX and deploy the model on a target device. You’ll need to export the model artifact using the steps below, then follow Run a Local Model with GenieX. See the LLM-on-Genie tutorial to run with the Genie runtime. Note: Genie support will be deprecated soon. Due to licensing restrictions, we cannot distribute pre-exported…
Модальности:
Генерация текста
Области применения:
Следование инструкциям
Задача: Генерация текста
Автор: qualcomm
Теги: llm, generative_ai, android
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
