LocalAI

LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference

Stars: 35323

Visit
 screenshot

LocalAI is a free and open-source OpenAI alternative that acts as a drop-in replacement REST API compatible with OpenAI (Elevenlabs, Anthropic, etc.) API specifications for local AI inferencing. It allows users to run LLMs, generate images, audio, and more locally or on-premises with consumer-grade hardware, supporting multiple model families and not requiring a GPU. LocalAI offers features such as text generation with GPTs, text-to-audio, audio-to-text transcription, image generation with stable diffusion, OpenAI functions, embeddings generation for vector databases, constrained grammars, downloading models directly from Huggingface, and a Vision API. It provides a detailed step-by-step introduction in its Getting Started guide and supports community integrations such as custom containers, WebUIs, model galleries, and various bots for Discord, Slack, and Telegram. LocalAI also offers resources like an LLM fine-tuning guide, instructions for local building and Kubernetes installation, projects integrating LocalAI, and a how-tos section curated by the community. It encourages users to cite the repository when utilizing it in downstream projects and acknowledges the contributions of various software from the community.

README:




LocalAI forks LocalAI stars LocalAI pull-requests

LocalAI Docker hub LocalAI Quay.io

Follow LocalAI_API Join LocalAI Discord Community

mudler%2FLocalAI | Trendshift

๐Ÿ’ก Get help - โ“FAQ ๐Ÿ’ญDiscussions ๐Ÿ’ฌ Discord ๐Ÿ“– Documentation website

๐Ÿ’ป Quickstart ๐Ÿ–ผ๏ธ Models ๐Ÿš€ Roadmap ๐ŸŒ Explorer ๐Ÿ›ซ Examples Try on Telegram

testsBuild and Releasebuild container imagesBump dependenciesArtifact Hub

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by Ettore Di Giacinto.

๐Ÿ“š๐Ÿ†• Local Stack Family

๐Ÿ†• LocalAI is now part of a comprehensive suite of AI tools designed to work together:

LocalAGI Logo

A powerful Local AI agent management platform that serves as a drop-in replacement for OpenAI's Responses API, enhanced with advanced agentic capabilities.

LocalRecall Logo

A REST-ful API and knowledge base management system that provides persistent memory and storage capabilities for AI agents.

Screenshots

Talk Interface Generate Audio
Screenshot 2025-03-31 at 12-01-36 LocalAI - Talk Screenshot 2025-03-31 at 12-01-29 LocalAI - Generate audio with voice-en-us-ryan-low
Models Overview Generate Images
Screenshot 2025-03-31 at 12-01-20 LocalAI - Models Screenshot 2025-03-31 at 12-31-41 LocalAI - Generate images with flux 1-dev
Chat Interface Home
Screenshot 2025-03-31 at 11-57-44 LocalAI - Chat with localai-functioncall-qwen2 5-7b-v0 5 Screenshot 2025-03-31 at 11-57-23 LocalAI API - c2a39e3 (c2a39e3639227cfd94ffffe9f5691239acc275a8)
Login Swarm
Screenshot 2025-03-31 at 12-09-59 Screenshot 2025-03-31 at 12-10-39 LocalAI - P2P dashboard

๐Ÿ’ป Quickstart

Run the installer script:

# Basic installation
curl https://localai.io/install.sh | sh

For more installation options, see Installer Options.

macOS Download:

Download LocalAI for macOS

Or run with docker:

CPU only image:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

NVIDIA GPU Images:

# CUDA 12.0
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# CUDA 11.7
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11

# NVIDIA Jetson (L4T) ARM64
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

AMD GPU Images (ROCm):

docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas

Intel GPU Images (oneAPI):

docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel

Vulkan GPU Images:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan

AIO Images (pre-downloaded models):

# CPU version
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu

# NVIDIA CUDA 12 version
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12

# NVIDIA CUDA 11 version
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-11

# Intel GPU version
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-gpu-intel

# AMD GPU version
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas

For more information about the AIO images and pre-downloaded models, see Container Documentation.

To load models:

# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# Start LocalAI with the phi-2 model directly from huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
local-ai run ollama://gemma:2b
# Run a model from a configuration file
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest

โšก Automatic Backend Detection: When you install models from the gallery or YAML files, LocalAI automatically detects your system's GPU capabilities (NVIDIA, AMD, Intel) and downloads the appropriate backend. For advanced configuration options, see GPU Acceleration.

For more information, see ๐Ÿ’ป Getting started

๐Ÿ“ฐ Latest project news

Roadmap items: List of issues

๐Ÿš€ Features

๐Ÿงฉ Supported Backends & Acceleration

LocalAI supports a comprehensive range of AI backends with multiple acceleration options:

Text Generation & Language Models

Backend Description Acceleration Support
llama.cpp LLM inference in C/C++ CUDA 11/12, ROCm, Intel SYCL, Vulkan, Metal, CPU
vLLM Fast LLM inference with PagedAttention CUDA 12, ROCm, Intel
transformers HuggingFace transformers framework CUDA 11/12, ROCm, Intel, CPU
exllama2 GPTQ inference library CUDA 12
MLX Apple Silicon LLM inference Metal (M1/M2/M3+)
MLX-VLM Apple Silicon Vision-Language Models Metal (M1/M2/M3+)

Audio & Speech Processing

Backend Description Acceleration Support
whisper.cpp OpenAI Whisper in C/C++ CUDA 12, ROCm, Intel SYCL, Vulkan, CPU
faster-whisper Fast Whisper with CTranslate2 CUDA 12, ROCm, Intel, CPU
bark Text-to-audio generation CUDA 12, ROCm, Intel
bark-cpp C++ implementation of Bark CUDA, Metal, CPU
coqui Advanced TTS with 1100+ languages CUDA 12, ROCm, Intel, CPU
kokoro Lightweight TTS model CUDA 12, ROCm, Intel, CPU
chatterbox Production-grade TTS CUDA 11/12, CPU
piper Fast neural TTS system CPU
kitten-tts Kitten TTS models CPU
silero-vad Voice Activity Detection CPU

Image & Video Generation

Backend Description Acceleration Support
stablediffusion.cpp Stable Diffusion in C/C++ CUDA 12, Intel SYCL, Vulkan, CPU
diffusers HuggingFace diffusion models CUDA 11/12, ROCm, Intel, Metal, CPU

Specialized AI Tasks

Backend Description Acceleration Support
rfdetr Real-time object detection CUDA 12, Intel, CPU
rerankers Document reranking API CUDA 11/12, ROCm, Intel, CPU
local-store Vector database CPU
huggingface HuggingFace API integration API-based

Hardware Acceleration Matrix

Acceleration Type Supported Backends Hardware Support
NVIDIA CUDA 11 llama.cpp, whisper, stablediffusion, diffusers, rerankers, bark, chatterbox Nvidia hardware
NVIDIA CUDA 12 All CUDA-compatible backends Nvidia hardware
AMD ROCm llama.cpp, whisper, vllm, transformers, diffusers, rerankers, coqui, kokoro, bark AMD Graphics
Intel oneAPI llama.cpp, whisper, stablediffusion, vllm, transformers, diffusers, rfdetr, rerankers, exllama2, coqui, kokoro, bark Intel Arc, Intel iGPUs
Apple Metal llama.cpp, whisper, diffusers, MLX, MLX-VLM, bark-cpp Apple M1/M2/M3+
Vulkan llama.cpp, whisper, stablediffusion Cross-platform GPUs
NVIDIA Jetson llama.cpp, whisper, stablediffusion, diffusers, rfdetr ARM64 embedded AI
CPU Optimized All backends AVX/AVX2/AVX512, quantization support

๐Ÿ”— Community and integrations

Build and deploy custom containers:

WebUIs:

Model galleries

Voice:

Other:

๐Ÿ”— Resources

๐Ÿ“– ๐ŸŽฅ Media, Blogs, Social

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

โค๏ธ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project covering CI expenses, and our Sponsor list:


๐ŸŒŸ Star history

LocalAI Star history Chart

๐Ÿ“– License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto [email protected]

๐Ÿ™‡ Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

๐Ÿค— Contributors

This is a community project, a special thanks to our contributors! ๐Ÿค—

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for LocalAI

Similar Open Source Tools

For similar tasks

For similar jobs