TheBloke/Puma-3b-GGML

Chat & support: TheBloke’s Discord server Want to contribute? TheBloke’s Patreon page TheBloke’s LLM work is generously supported by a grant from andreessen horowitz (a16z) This repo contains GGML format model files for Bohan Du’s Puma 3B. GGML files are for CPU + GPU inference using llama.cpp and libraries and UIs which support this format, such as: text-generation-webui, the most popular web UI. Supports NVidia CUDA GPU acceleration. KoboldCpp, a powerful GGML web UI with GPU acceleration on all platforms (CUDA and OpenCL). Especially good for story telling. LM Studio, a fully featured local GUI with GPU acceleration on both Windows (NVidia and AMD), and macOS. LoLLMS Web UI, a great web UI with CUDA GPU acceleration via the ctransformers backend. ctransformers, a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. * llama-cpp-python, a Python library with GPU accel, LangChain support, and OpenAI-compatible API server. GPTQ models for GPU inference, with multiple quantisation parameter options. 2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference * Bohan Du’s original unquantised fp16 model in pytorch format, for GPU inference and…

Модальности:
Генерация текста

Задача: Генерация текста
Автор: TheBloke
Теги: llama, en
Лайков: 3  |  Загрузок: 11

Открыть на HuggingFace →

Описание основано на материалах HuggingFace. Перевод выполнен автоматически.