Categorygithub.com/mudler/LocalAI

modulepackage

2.23.0+incompatible

Repository: https://github.com/mudler/localai.git

Documentation: pkg.go.dev

# README

LocalAI

:bulb: Get help - ❓FAQ 💭Discussions :speech_balloon: Discord :book: Documentation website

💻 Quickstart 🖼️ Models 🚀 Roadmap 🥽 Demo 🌍 Explorer 🛫 Examples

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by Ettore Di Giacinto.

screen

Run the installer script:

curl https://localai.io/install.sh | sh

Or run with docker:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
# Alternative images:
# - if you have an Nvidia GPU:
# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12
# - without preconfigured models
# docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
# - without preconfigured models for Nvidia GPUs
# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

To load models:

# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# Start LocalAI with the phi-2 model directly from huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
local-ai run ollama://gemma:2b
# Run a model from a configuration file
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest

💻 Getting started

📰 Latest project news

Oct 2024: examples moved to LocalAI-examples
Aug 2024: 🆕 FLUX-1, P2P Explorer
July 2024: 🔥🔥 🆕 P2P Dashboard, LocalAI Federated mode and AI Swarms: https://github.com/mudler/LocalAI/pull/2723
June 2024: 🆕 You can browse now the model gallery without LocalAI! Check out https://models.localai.io
June 2024: Support for models from OCI registries: https://github.com/mudler/LocalAI/pull/2628
May 2024: 🔥🔥 Decentralized P2P llama.cpp: https://github.com/mudler/LocalAI/pull/2343 (peer2peer llama.cpp!) 👉 Docs https://localai.io/features/distribute/
May 2024: 🔥🔥 Openvoice: https://github.com/mudler/LocalAI/pull/2334
May 2024: 🆕 Function calls without grammars and mixed mode: https://github.com/mudler/LocalAI/pull/2328
May 2024: 🔥🔥 Distributed inferencing: https://github.com/mudler/LocalAI/pull/2324
May 2024: Chat, TTS, and Image generation in the WebUI: https://github.com/mudler/LocalAI/pull/2222
April 2024: Reranker API: https://github.com/mudler/LocalAI/pull/2121

Roadmap items: List of issues

🔥🔥 Hot topics (looking for help):

Multimodal with vLLM and Video understanding: https://github.com/mudler/LocalAI/pull/3729
Realtime API https://github.com/mudler/LocalAI/issues/3714
🔥🔥 Distributed, P2P Global community pools: https://github.com/mudler/LocalAI/issues/3113
WebUI improvements: https://github.com/mudler/LocalAI/issues/2156
Backends v2: https://github.com/mudler/LocalAI/issues/1126
Improving UX v2: https://github.com/mudler/LocalAI/issues/1373
Assistant API: https://github.com/mudler/LocalAI/issues/1273
Moderation endpoint: https://github.com/mudler/LocalAI/issues/999
Vulkan: https://github.com/mudler/LocalAI/issues/1647
Anthropic API: https://github.com/mudler/LocalAI/issues/1808

If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22

🚀 Features

📖 Text generation with GPTs (llama.cpp, gpt4all.cpp, ... :book: and more)
🗣 Text to Audio
🔈 Audio to Text (Audio transcription with whisper.cpp)
🎨 Image generation with stable diffusion
🔥 OpenAI-alike tools API
🧠 Embeddings generation for vector databases
✍️ Constrained grammars
🖼️ Download Models directly from Huggingface
🥽 Vision API
📈 Reranker API
🆕🖧 P2P Inferencing
🌍 Integrated WebUI!

💻 Usage

Check out the Getting started section in our documentation.

🔗 Community and integrations

Build and deploy custom containers:

https://github.com/sozercan/aikit

WebUIs:

https://github.com/Jirubizu/localai-admin
https://github.com/go-skynet/LocalAI-frontend
QA-Pilot(An interactive chat project that leverages LocalAI LLMs for rapid understanding and navigation of GitHub code repository) https://github.com/reid41/QA-Pilot

Model galleries

https://github.com/go-skynet/model-gallery

Other:

Helm chart https://github.com/go-skynet/helm-charts
VSCode extension https://github.com/badgooooor/localai-vscode-plugin
Terminal utility https://github.com/djcopley/ShellOracle
Local Smart assistant https://github.com/mudler/LocalAGI
Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation / https://github.com/valentinfrlch/ha-gpt4vision
Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
Shell-Pilot(Interact with LLM using LocalAI models via pure shell scripts on your Linux or MacOS system) https://github.com/reid41/shell-pilot
Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
Github Actions: https://github.com/marketplace/actions/start-localai
Examples: https://github.com/mudler/LocalAI/tree/master/examples/

🔗 Resources

LLM finetuning guide
How to build locally
How to install in Kubernetes
Projects integrating LocalAI
How tos section (curated by our community)

:book: 🎥 Media, Blogs, Social

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},