Core AI is Apple’s on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple’s coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta, 2026-06). > Mirror of mlboydaisuke/BitCPM-8B-CoreAI — the canonical repo (CoreAI Model Zoo). Updates land there first. The zoo’s first 1.58-bit ternary LLM and first sub-int8 packed-GEMM Metal kernel, running fully on-device on iPhone (and Mac) through Apple Core AI (.aimodel / .aimodelc, iOS 27 / macOS 27). An 8B model whose transformer weights are just {-1, 0, +1} — so it generates at a 3–4B-class footprint and speed while keeping 8B-class quality. — Base: openbmb/BitCPM-CANN-8B (MiniCPM4-8B architecture, quantization-aware trained to ternary), Apache-2.0. — Zoo + conversion code: https://github.com/john-rocky/coreai-model-zoo Headroom ~4.3 GB, no jetsam. An int4 8B would need ~5–6 GB resident — the ternary weight stream is the win. On M4 Max the same graph decodes 62.7 tok/s and is token-identical to the torch ternary…
Модальности:
Генерация текста
Задача: Генерация текста
Автор: coreai-community
Теги: coreai, core-ai, on-device, iphone, apple-silicon, ternary, bitnet, 1.58-bit
Лайков: 3 | Загрузок: 0
Описание основано на материалах HuggingFace. Перевод выполнен автоматически.
