
ollama
Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
Stars: 152437

Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications. Ollama is designed to be easy to use and accessible to developers of all levels. It is open source and available for free on GitHub.
README:
Get up and running with large language models.
curl -fsSL https://ollama.com/install.sh | sh
The official Ollama Docker image ollama/ollama
is available on Docker Hub.
To run and chat with Gemma 3:
ollama run gemma3
Ollama supports a list of models available on ollama.com/library
Here are some example models that can be downloaded:
Model | Parameters | Size | Download |
---|---|---|---|
Gemma 3 | 1B | 815MB | ollama run gemma3:1b |
Gemma 3 | 4B | 3.3GB | ollama run gemma3 |
Gemma 3 | 12B | 8.1GB | ollama run gemma3:12b |
Gemma 3 | 27B | 17GB | ollama run gemma3:27b |
QwQ | 32B | 20GB | ollama run qwq |
DeepSeek-R1 | 7B | 4.7GB | ollama run deepseek-r1 |
DeepSeek-R1 | 671B | 404GB | ollama run deepseek-r1:671b |
Llama 4 | 109B | 67GB | ollama run llama4:scout |
Llama 4 | 400B | 245GB | ollama run llama4:maverick |
Llama 3.3 | 70B | 43GB | ollama run llama3.3 |
Llama 3.2 | 3B | 2.0GB | ollama run llama3.2 |
Llama 3.2 | 1B | 1.3GB | ollama run llama3.2:1b |
Llama 3.2 Vision | 11B | 7.9GB | ollama run llama3.2-vision |
Llama 3.2 Vision | 90B | 55GB | ollama run llama3.2-vision:90b |
Llama 3.1 | 8B | 4.7GB | ollama run llama3.1 |
Llama 3.1 | 405B | 231GB | ollama run llama3.1:405b |
Phi 4 | 14B | 9.1GB | ollama run phi4 |
Phi 4 Mini | 3.8B | 2.5GB | ollama run phi4-mini |
Mistral | 7B | 4.1GB | ollama run mistral |
Moondream 2 | 1.4B | 829MB | ollama run moondream |
Neural Chat | 7B | 4.1GB | ollama run neural-chat |
Starling | 7B | 4.1GB | ollama run starling-lm |
Code Llama | 7B | 3.8GB | ollama run codellama |
Llama 2 Uncensored | 7B | 3.8GB | ollama run llama2-uncensored |
LLaVA | 7B | 4.5GB | ollama run llava |
Granite-3.3 | 8B | 4.9GB | ollama run granite3.3 |
[!NOTE] You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Ollama supports importing GGUF models in the Modelfile:
-
Create a file named
Modelfile
, with aFROM
instruction with the local filepath to the model you want to import.FROM ./vicuna-33b.Q4_0.gguf
-
Create the model in Ollama
ollama create example -f Modelfile
-
Run the model
ollama run example
See the guide on importing models for more information.
Models from the Ollama library can be customized with a prompt. For example, to customize the llama3.2
model:
ollama pull llama3.2
Create a Modelfile
:
FROM llama3.2
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system message
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
Next, create and run the model:
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
For more information on working with a Modelfile, see the Modelfile documentation.
ollama create
is used to create a model from a Modelfile.
ollama create mymodel -f ./Modelfile
ollama pull llama3.2
This command can also be used to update a local model. Only the diff will be pulled.
ollama rm llama3.2
ollama cp llama3.2 my-model
For multiline input, you can wrap text with """
:
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
ollama run llava "What's in this image? /Users/jmorgan/Desktop/smile.png"
Output: The image features a yellow smiley face, which is likely the central focus of the picture.
ollama run llama3.2 "Summarize this file: $(cat README.md)"
Output: Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
ollama show llama3.2
ollama list
ollama ps
ollama stop llama3.2
ollama serve
is used when you want to start ollama without running the desktop application.
See the developer guide
Next, start the server:
./ollama serve
Finally, in a separate shell, run a model:
./ollama run llama3.2
Ollama has a REST API for running and managing models.
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2",
"prompt":"Why is the sky blue?"
}'
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
See the API documentation for all endpoints.
- Open WebUI
- SwiftChat (macOS with ReactNative)
- Enchanted (macOS native)
- Hollama
- Lollms-Webui
- LibreChat
- Bionic GPT
- HTML UI
- Saddle
- TagSpaces (A platform for file-based apps, utilizing Ollama for the generation of tags and descriptions)
- Chatbot UI
- Chatbot UI v2
- Typescript UI
- Minimalistic React UI for Ollama Models
- Ollamac
- big-AGI
- Cheshire Cat assistant framework
- Amica
- chatd
- Ollama-SwiftUI
- Dify.AI
- MindMac
- NextJS Web Interface for Ollama
- Msty
- Chatbox
- WinForm Ollama Copilot
- NextChat with Get Started Doc
- Alpaca WebUI
- OllamaGUI
- OpenAOE
- Odin Runes
- LLM-X (Progressive Web App)
- AnythingLLM (Docker + MacOs/Windows/Linux native app)
- Ollama Basic Chat: Uses HyperDiv Reactive UI
- Ollama-chats RPG
- IntelliBar (AI-powered assistant for macOS)
- Jirapt (Jira Integration to generate issues, tasks, epics)
- ojira (Jira chrome plugin to easily generate descriptions for tasks)
- QA-Pilot (Interactive chat tool that can leverage Ollama models for rapid understanding and navigation of GitHub code repositories)
- ChatOllama (Open Source Chatbot based on Ollama with Knowledge Bases)
- CRAG Ollama Chat (Simple Web Search with Corrective RAG)
- RAGFlow (Open-source Retrieval-Augmented Generation engine based on deep document understanding)
- StreamDeploy (LLM Application Scaffold)
- chat (chat web app for teams)
- Lobe Chat with Integrating Doc
- Ollama RAG Chatbot (Local Chat with multiple PDFs using Ollama and RAG)
- BrainSoup (Flexible native client with RAG & multi-agent automation)
- macai (macOS client for Ollama, ChatGPT, and other compatible API back-ends)
- RWKV-Runner (RWKV offline LLM deployment tool, also usable as a client for ChatGPT and Ollama)
- Ollama Grid Search (app to evaluate and compare models)
- Olpaka (User-friendly Flutter Web App for Ollama)
- Casibase (An open source AI knowledge base and dialogue system combining the latest RAG, SSO, ollama support, and multiple large language models.)
- OllamaSpring (Ollama Client for macOS)
- LLocal.in (Easy to use Electron Desktop Client for Ollama)
- Shinkai Desktop (Two click install Local AI using Ollama + Files + RAG)
- AiLama (A Discord User App that allows you to interact with Ollama anywhere in Discord)
- Ollama with Google Mesop (Mesop Chat Client implementation with Ollama)
- R2R (Open-source RAG engine)
- Ollama-Kis (A simple easy-to-use GUI with sample custom LLM for Drivers Education)
- OpenGPA (Open-source offline-first Enterprise Agentic Application)
- Painting Droid (Painting app with AI integrations)
- Kerlig AI (AI writing assistant for macOS)
- AI Studio
- Sidellama (browser-based LLM client)
- LLMStack (No-code multi-agent framework to build LLM agents and workflows)
- BoltAI for Mac (AI Chat Client for Mac)
- Harbor (Containerized LLM Toolkit with Ollama as default backend)
- PyGPT (AI desktop assistant for Linux, Windows, and Mac)
- Alpaca (An Ollama client application for Linux and macOS made with GTK4 and Adwaita)
- AutoGPT (AutoGPT Ollama integration)
- Go-CREW (Powerful Offline RAG in Golang)
- PartCAD (CAD model generation with OpenSCAD and CadQuery)
- Ollama4j Web UI - Java-based Web UI for Ollama built with Vaadin, Spring Boot, and Ollama4j
- PyOllaMx - macOS application capable of chatting with both Ollama and Apple MLX models.
- Cline - Formerly known as Claude Dev is a VSCode extension for multi-file/whole-repo coding
- Cherry Studio (Desktop client with Ollama support)
- ConfiChat (Lightweight, standalone, multi-platform, and privacy-focused LLM chat interface with optional encryption)
- Archyve (RAG-enabling document library)
- crewAI with Mesop (Mesop Web Interface to run crewAI with Ollama)
- Tkinter-based client (Python tkinter-based Client for Ollama)
- LLMChat (Privacy focused, 100% local, intuitive all-in-one chat interface)
- Local Multimodal AI Chat (Ollama-based LLM Chat with support for multiple features, including PDF RAG, voice chat, image-based interactions, and integration with OpenAI.)
- ARGO (Locally download and run Ollama and Huggingface models with RAG and deep research on Mac/Windows/Linux)
- OrionChat - OrionChat is a web interface for chatting with different AI providers
- G1 (Prototype of using prompting strategies to improve the LLM's reasoning through o1-like reasoning chains.)
- Web management (Web management page)
- Promptery (desktop client for Ollama.)
- Ollama App (Modern and easy-to-use multi-platform client for Ollama)
- chat-ollama (a React Native client for Ollama)
- SpaceLlama (Firefox and Chrome extension to quickly summarize web pages with ollama in a sidebar)
- YouLama (Webapp to quickly summarize any YouTube video, supporting Invidious as well)
- DualMind (Experimental app allowing two models to talk to each other in the terminal or in a web interface)
- ollamarama-matrix (Ollama chatbot for the Matrix chat protocol)
- ollama-chat-app (Flutter-based chat app)
- Perfect Memory AI (Productivity AI assists personalized by what you have seen on your screen, heard, and said in the meetings)
- Hexabot (A conversational AI builder)
- Reddit Rate (Search and Rate Reddit topics with a weighted summation)
- OpenTalkGpt (Chrome Extension to manage open-source models supported by Ollama, create custom models, and chat with models from a user-friendly UI)
- VT (A minimal multimodal AI chat app, with dynamic conversation routing. Supports local models via Ollama)
- Nosia (Easy to install and use RAG platform based on Ollama)
- Witsy (An AI Desktop application available for Mac/Windows/Linux)
- Abbey (A configurable AI interface server with notebooks, document storage, and YouTube support)
- Minima (RAG with on-premises or fully local workflow)
- aidful-ollama-model-delete (User interface for simplified model cleanup)
- Perplexica (An AI-powered search engine & an open-source alternative to Perplexity AI)
- Ollama Chat WebUI for Docker (Support for local docker deployment, lightweight ollama webui)
- AI Toolkit for Visual Studio Code (Microsoft-official VSCode extension to chat, test, evaluate models with Ollama support, and use them in your AI applications.)
- MinimalNextOllamaChat (Minimal Web UI for Chat and Model Control)
- Chipper AI interface for tinkerers (Ollama, Haystack RAG, Python)
- ChibiChat (Kotlin-based Android app to chat with Ollama and Koboldcpp API endpoints)
- LocalLLM (Minimal Web-App to run ollama models on it with a GUI)
- Ollamazing (Web extension to run Ollama models)
- OpenDeepResearcher-via-searxng (A Deep Research equivalent endpoint with Ollama support for running locally)
- AntSK (Out-of-the-box & Adaptable RAG Chatbot)
- MaxKB (Ready-to-use & flexible RAG Chatbot)
- yla (Web interface to freely interact with your customized models)
- LangBot (LLM-based instant messaging bots platform, with Agents, RAG features, supports multiple platforms)
- 1Panel (Web-based Linux Server Management Tool)
- AstrBot (User-friendly LLM-based multi-platform chatbot with a WebUI, supporting RAG, LLM agents, and plugins integration)
- Reins (Easily tweak parameters, customize system prompts per chat, and enhance your AI experiments with reasoning model support.)
- Flufy (A beautiful chat interface for interacting with Ollama's API. Built with React, TypeScript, and Material-UI.)
- Ellama (Friendly native app to chat with an Ollama instance)
- screenpipe Build agents powered by your screen history
- Ollamb (Simple yet rich in features, cross-platform built with Flutter and designed for Ollama. Try the web demo.)
- Writeopia (Text editor with integration with Ollama)
- AppFlowy (AI collaborative workspace with Ollama, cross-platform and self-hostable)
- Lumina (A lightweight, minimal React.js frontend for interacting with Ollama servers)
- Tiny Notepad (A lightweight, notepad-like interface to chat with ollama available on PyPI)
- macLlama (macOS native) (A native macOS GUI application for interacting with Ollama models, featuring a chat interface.)
- GPTranslate (A fast and lightweight, AI powered desktop translation application written with Rust and Tauri. Features real-time translation with OpenAI/Azure/Ollama.)
- ollama launcher (A launcher for Ollama, aiming to provide users with convenient functions such as ollama server launching, management, or configuration.)
- ai-hub (AI Hub supports multiple models via API keys and Chat support via Ollama API.)
- Mayan EDMS (Open source document management system to organize, tag, search, and automate your files with powerful Ollama driven workflows.)
- Serene Pub (Beginner friendly, open source AI Roleplaying App for Windows, Mac OS and Linux. Search, download and use models with Ollama all inside the app.)
- Andes (A Visual Studio Code extension that provides a local UI interface for Ollama models)
- Clueless (Open Source & Local Cluely: A desktop application LLM assistant to help you talk to anything on your screen using locally served Ollama models. Also undetectable to screenshare)
- ollama-co2 (FastAPI web interface for monitoring and managing local and remote Ollama servers with real-time model monitoring and concurrent downloads)
- oterm
- Ellama Emacs client
- Emacs client
- neollama UI client for interacting with models from within Neovim
- gen.nvim
- ollama.nvim
- ollero.nvim
- ollama-chat.nvim
- ogpt.nvim
- gptel Emacs client
- Oatmeal
- cmdh
- ooo
- shell-pilot(Interact with models via pure shell scripts on Linux or macOS)
- tenere
- llm-ollama for Datasette's LLM CLI.
- typechat-cli
- ShellOracle
- tlm
- podman-ollama
- gollama
- ParLlama
- Ollama eBook Summary
- Ollama Mixture of Experts (MOE) in 50 lines of code
- vim-intelligence-bridge Simple interaction of "Ollama" with the Vim editor
- x-cmd ollama
- bb7
- SwollamaCLI bundled with the Swollama Swift package. Demo
- aichat All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.
- PowershAI PowerShell module that brings AI to terminal on Windows, including support for Ollama
- DeepShell Your self-hosted AI assistant. Interactive Shell, Files and Folders analysis.
- orbiton Configuration-free text editor and IDE with support for tab completion with Ollama.
- orca-cli Ollama Registry CLI Application - Browse, pull, and download models from Ollama Registry in your terminal.
- GGUF-to-Ollama - Importing GGUF to Ollama made easy (multiplatform)
- AWS-Strands-With-Ollama - AWS Strands Agents with Ollama Examples
- ollama-multirun - A bash shell script to run a single prompt against any or all of your locally installed ollama models, saving the output and performance statistics as easily navigable web pages. (Demo)
- ollama-bash-toolshed - Bash scripts to chat with tool using models. Add new tools to your shed with ease. Runs on Ollama.
- SwiftChat (Cross-platform AI chat app supporting Apple Vision Pro via "Designed for iPad")
- Enchanted
- pgai - PostgreSQL as a vector database (Create and search embeddings from Ollama models using pgvector)
- MindsDB (Connects Ollama models with nearly 200 data platforms and apps)
- chromem-go with example
- Kangaroo (AI-powered SQL client and admin tool for popular databases)
- LangChain and LangChain.js with example
- Firebase Genkit
- crewAI
- Yacana (User-friendly multi-agent framework for brainstorming and executing predetermined flows with built-in tool integration)
- Spring AI with reference and example
- LangChainGo with example
- LangChain4j with example
- LangChainRust with example
- LangChain for .NET with example
- LLPhant
- LlamaIndex and LlamaIndexTS
- LiteLLM
- OllamaFarm for Go
- OllamaSharp for .NET
- Ollama for Ruby
- Ollama-rs for Rust
- Ollama-hpp for C++
- Ollama4j for Java
- ModelFusion Typescript Library
- OllamaKit for Swift
- Ollama for Dart
- Ollama for Laravel
- LangChainDart
- Semantic Kernel - Python
- Haystack
- Elixir LangChain
- Ollama for R - rollama
- Ollama for R - ollama-r
- Ollama-ex for Elixir
- Ollama Connector for SAP ABAP
- Testcontainers
- Portkey
- PromptingTools.jl with an example
- LlamaScript
- llm-axe (Python Toolkit for Building LLM Powered Apps)
- Gollm
- Gollama for Golang
- Ollamaclient for Golang
- High-level function abstraction in Go
- Ollama PHP
- Agents-Flex for Java with example
- Parakeet is a GoLang library, made to simplify the development of small generative AI applications with Ollama.
- Haverscript with examples
- Ollama for Swift
- Swollama for Swift with DocC
- GoLamify
- Ollama for Haskell
- multi-llm-ts (A Typescript/JavaScript library allowing access to different LLM in a unified API)
- LlmTornado (C# library providing a unified interface for major FOSS & Commercial inference APIs)
- Ollama for Zig
- Abso (OpenAI-compatible TypeScript SDK for any LLM provider)
- Nichey is a Python package for generating custom wikis for your research topic
- Ollama for D
- OllamaPlusPlus (Very simple C++ library for Ollama)
- any-llm (A single interface to use different llm providers by mozilla.ai)
- any-agent (A single interface to use and evaluate different agent frameworks by mozilla.ai)
- Neuro SAN (Data-driven multi-agent orchestration framework) with example
- SwiftChat (Lightning-fast Cross-platform AI chat app with native UI for Android, iOS, and iPad)
- Enchanted
- Maid
- Ollama App (Modern and easy-to-use multi-platform client for Ollama)
- ConfiChat (Lightweight, standalone, multi-platform, and privacy-focused LLM chat interface with optional encryption)
- Ollama Android Chat (No need for Termux, start the Ollama service with one click on an Android device)
- Reins (Easily tweak parameters, customize system prompts per chat, and enhance your AI experiments with reasoning model support.)
- Raycast extension
- Discollama (Discord bot inside the Ollama discord channel)
- Continue
- Vibe (Transcribe and analyze meetings with Ollama)
- Obsidian Ollama plugin
- Logseq Ollama plugin
- NotesOllama (Apple Notes Ollama plugin)
- Dagger Chatbot
- Discord AI Bot
- Ollama Telegram Bot
- Hass Ollama Conversation
- Rivet plugin
- Obsidian BMO Chatbot plugin
- Cliobot (Telegram bot with Ollama support)
- Copilot for Obsidian plugin
- Obsidian Local GPT plugin
- Open Interpreter
- Llama Coder (Copilot alternative using Ollama)
- Ollama Copilot (Proxy that allows you to use Ollama as a copilot like GitHub Copilot)
- twinny (Copilot and Copilot chat alternative using Ollama)
- Wingman-AI (Copilot code and chat alternative using Ollama and Hugging Face)
- Page Assist (Chrome Extension)
- Plasmoid Ollama Control (KDE Plasma extension that allows you to quickly manage/control Ollama model)
- AI Telegram Bot (Telegram bot using Ollama in backend)
- AI ST Completion (Sublime Text 4 AI assistant plugin with Ollama support)
- Discord-Ollama Chat Bot (Generalized TypeScript Discord Bot w/ Tuning Documentation)
- ChatGPTBox: All in one browser extension with Integrating Tutorial
- Discord AI chat/moderation bot Chat/moderation bot written in python. Uses Ollama to create personalities.
- Headless Ollama (Scripts to automatically install ollama client & models on any OS for apps that depend on ollama server)
- Terraform AWS Ollama & Open WebUI (A Terraform module to deploy on AWS a ready-to-use Ollama service, together with its front-end Open WebUI service.)
- node-red-contrib-ollama
- Local AI Helper (Chrome and Firefox extensions that enable interactions with the active tab and customisable API endpoints. Includes secure storage for user prompts.)
- vnc-lm (Discord bot for messaging with LLMs through Ollama and LiteLLM. Seamlessly move between local and flagship models.)
- LSP-AI (Open-source language server for AI-powered functionality)
- QodeAssist (AI-powered coding assistant plugin for Qt Creator)
- Obsidian Quiz Generator plugin
- AI Summmary Helper plugin
- TextCraft (Copilot in Word alternative using Ollama)
- Alfred Ollama (Alfred Workflow)
- TextLLaMA A Chrome Extension that helps you write emails, correct grammar, and translate into any language
- Simple-Discord-AI
- LLM Telegram Bot (telegram bot, primary for RP. Oobabooga-like buttons, A1111 API integration e.t.c)
- mcp-llm (MCP Server to allow LLMs to call other LLMs)
- SimpleOllamaUnity (Unity Engine extension for communicating with Ollama in a few lines of code. Also works at runtime)
- UnityCodeLama (Unity Edtior tool to analyze scripts via Ollama)
- NativeMind (Private, on-device AI Assistant, no cloud dependencies)
- GMAI - Gradle Managed AI (Gradle plugin for automated Ollama lifecycle management during build phases)
- NOMYO Router (A transparent Ollama proxy with model deployment aware routing which auto-manages multiple Ollama instances in a given network)
- llama.cpp project founded by Georgi Gerganov.
- Opik is an open-source platform to debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards. Opik supports native intergration to Ollama.
- Lunary is the leading open-source LLM observability platform. It provides a variety of enterprise-grade features such as real-time analytics, prompt templates management, PII masking, and comprehensive agent tracing.
- OpenLIT is an OpenTelemetry-native tool for monitoring Ollama Applications & GPUs using traces and metrics.
- HoneyHive is an AI observability and evaluation platform for AI agents. Use HoneyHive to evaluate agent performance, interrogate failures, and monitor quality in production.
- Langfuse is an open source LLM observability platform that enables teams to collaboratively monitor, evaluate and debug AI applications.
- MLflow Tracing is an open source LLM observability tool with a convenient API to log and visualize traces, making it easy to debug and evaluate GenAI applications.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for ollama
Similar Open Source Tools

ollama
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications. Ollama is designed to be easy to use and accessible to developers of all levels. It is open source and available for free on GitHub.

spandrel
Spandrel is a library for loading and running pre-trained PyTorch models. It automatically detects the model architecture and hyperparameters from model files, and provides a unified interface for running models.

nexa-sdk
Nexa SDK is a comprehensive toolkit supporting ONNX and GGML models for text generation, image generation, vision-language models (VLM), and text-to-speech (TTS) capabilities. It offers an OpenAI-compatible API server with JSON schema mode and streaming support, along with a user-friendly Streamlit UI. Users can run Nexa SDK on any device with Python environment, with GPU acceleration supported. The toolkit provides model support, conversion engine, inference engine for various tasks, and differentiating features from other tools.

bagofwords
Bag of words is an open-source AI platform that helps data teams deploy and manage chat-with-your-data agents in a controlled, reliable, and self-learning environment. It enables users to create charts, tables, and dashboards by chatting with their data, capture AI decisions and user feedback, automatically improve AI quality, integrate with various data sources and APIs, and ensure governance and integrations. The platform supports self-hosting in VPC via VMs, Docker/Compose, or Kubernetes, and offers additional integrations for AI Analyst in Slack, Excel, Google Sheets, and more. Users can start in minutes and scale to org-wide analytics.

openrl
OpenRL is an open-source general reinforcement learning research framework that supports training for various tasks such as single-agent, multi-agent, offline RL, self-play, and natural language. Developed based on PyTorch, the goal of OpenRL is to provide a simple-to-use, flexible, efficient and sustainable platform for the reinforcement learning research community. It supports a universal interface for all tasks/environments, single-agent and multi-agent tasks, offline RL training with expert dataset, self-play training, reinforcement learning training for natural language tasks, DeepSpeed, Arena for evaluation, importing models and datasets from Hugging Face, user-defined environments, models, and datasets, gymnasium environments, callbacks, visualization tools, unit testing, and code coverage testing. It also supports various algorithms like PPO, DQN, SAC, and environments like Gymnasium, MuJoCo, Atari, and more.

pipecat
Pipecat is an open-source framework designed for building generative AI voice bots and multimodal assistants. It provides code building blocks for interacting with AI services, creating low-latency data pipelines, and transporting audio, video, and events over the Internet. Pipecat supports various AI services like speech-to-text, text-to-speech, image generation, and vision models. Users can implement new services and contribute to the framework. Pipecat aims to simplify the development of applications like personal coaches, meeting assistants, customer support bots, and more by providing a complete framework for integrating AI services.

shards
Shards is a high-performance, multi-platform, type-safe programming language designed for visual development. It is a dataflow visual programming language that enables building full-fledged apps and games without traditional coding. Shards features automatic type checking, optimized shard implementations for high performance, and an intuitive visual workflow for beginners. The language allows seamless round-trip engineering between code and visual models, empowering users to create multi-platform apps easily. Shards also powers an upcoming AI-powered game creation system, enabling real-time collaboration and game development in a low to no-code environment.

easy-dataset
Easy Dataset is a specialized application designed to streamline the creation of fine-tuning datasets for Large Language Models (LLMs). It offers an intuitive interface for uploading domain-specific files, intelligently splitting content, generating questions, and producing high-quality training data for model fine-tuning. With Easy Dataset, users can transform domain knowledge into structured datasets compatible with all OpenAI-format compatible LLM APIs, making the fine-tuning process accessible and efficient.

ai
The react-native-ai repository allows users to run Large Language Models (LLM) locally in a React Native app using the Universal MLC LLM Engine with compatibility for Vercel AI SDK. Please note that this project is experimental and not ready for production. The repository is licensed under MIT and was created with create-react-native-library.

WeKnora
WeKnora is a document understanding and semantic retrieval framework based on large language models (LLM), designed specifically for scenarios with complex structures and heterogeneous content. The framework adopts a modular architecture, integrating multimodal preprocessing, semantic vector indexing, intelligent recall, and large model generation reasoning to build an efficient and controllable document question-answering process. The core retrieval process is based on the RAG (Retrieval-Augmented Generation) mechanism, combining context-relevant segments with language models to achieve higher-quality semantic answers. It supports various document formats, intelligent inference, flexible extension, efficient retrieval, ease of use, and security and control. Suitable for enterprise knowledge management, scientific literature analysis, product technical support, legal compliance review, and medical knowledge assistance.

fastRAG
fastRAG is a research framework designed to build and explore efficient retrieval-augmented generative models. It incorporates state-of-the-art Large Language Models (LLMs) and Information Retrieval to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation. The framework is optimized for Intel hardware, customizable, and includes key features such as optimized RAG pipelines, efficient components, and RAG-efficient components like ColBERT and Fusion-in-Decoder (FiD). fastRAG supports various unique components and backends for running LLMs, making it a versatile tool for research and development in the field of retrieval-augmented generation.

auto-news
Auto-News is an automatic news aggregator tool that utilizes Large Language Models (LLM) to pull information from various sources such as Tweets, RSS feeds, YouTube videos, web articles, Reddit, and journal notes. The tool aims to help users efficiently read and filter content based on personal interests, providing a unified reading experience and organizing information effectively. It features feed aggregation with summarization, transcript generation for videos and articles, noise reduction, task organization, and deep dive topic exploration. The tool supports multiple LLM backends, offers weekly top-k aggregations, and can be deployed on Linux/MacOS using docker-compose or Kubernetes.

Crane
Crane is a high-performance inference framework leveraging Rust's Candle for maximum speed on CPU/GPU. It focuses on accelerating LLM inference speed with optimized kernels, reducing development overhead, and ensuring portability for running models on both CPU and GPU. Supported models include TTS systems like Spark-TTS and Orpheus-TTS, foundation models like Qwen2.5 series and basic LLMs, and multimodal models like Namo-R1 and Qwen2.5-VL. Key advantages of Crane include blazing-fast inference outperforming native PyTorch, Rust-powered to eliminate C++ complexity, Apple Silicon optimized for GPU acceleration via Metal, and hardware agnostic with a unified codebase for CPU/CUDA/Metal execution. Crane simplifies deployment with the ability to add new models with less than 100 lines of code in most cases.

inference
Xorbits Inference (Xinference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. With Xorbits Inference, you can effortlessly deploy and serve your or state-of-the-art built-in models using just a single command. Whether you are a researcher, developer, or data scientist, Xorbits Inference empowers you to unleash the full potential of cutting-edge AI models.

LLM4Decompile
LLM4Decompile is an open-source large language model dedicated to decompilation of Linux x86_64 binaries, supporting GCC's O0 to O3 optimization levels. It focuses on assessing re-executability of decompiled code through HumanEval-Decompile benchmark. The tool includes models with sizes ranging from 1.3 billion to 33 billion parameters, available on Hugging Face. Users can preprocess C code into binary and assembly instructions, then decompile assembly instructions into C using LLM4Decompile. Ongoing efforts aim to expand capabilities to support more architectures and configurations, integrate with decompilation tools like Ghidra and Rizin, and enhance performance with larger training datasets.

axolotl
Axolotl is a lightweight and efficient tool for managing and analyzing large datasets. It provides a user-friendly interface for data manipulation, visualization, and statistical analysis. With Axolotl, users can easily import, clean, and explore data to gain valuable insights and make informed decisions. The tool supports various data formats and offers a wide range of functions for data processing and modeling. Whether you are a data scientist, researcher, or business analyst, Axolotl can help streamline your data workflows and enhance your data analysis capabilities.
For similar tasks

ollama
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications. Ollama is designed to be easy to use and accessible to developers of all levels. It is open source and available for free on GitHub.

LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.

LocalAI
LocalAI is a free and open-source OpenAI alternative that acts as a drop-in replacement REST API compatible with OpenAI (Elevenlabs, Anthropic, etc.) API specifications for local AI inferencing. It allows users to run LLMs, generate images, audio, and more locally or on-premises with consumer-grade hardware, supporting multiple model families and not requiring a GPU. LocalAI offers features such as text generation with GPTs, text-to-audio, audio-to-text transcription, image generation with stable diffusion, OpenAI functions, embeddings generation for vector databases, constrained grammars, downloading models directly from Huggingface, and a Vision API. It provides a detailed step-by-step introduction in its Getting Started guide and supports community integrations such as custom containers, WebUIs, model galleries, and various bots for Discord, Slack, and Telegram. LocalAI also offers resources like an LLM fine-tuning guide, instructions for local building and Kubernetes installation, projects integrating LocalAI, and a how-tos section curated by the community. It encourages users to cite the repository when utilizing it in downstream projects and acknowledges the contributions of various software from the community.

AiTreasureBox
AiTreasureBox is a versatile AI tool that provides a collection of pre-trained models and algorithms for various machine learning tasks. It simplifies the process of implementing AI solutions by offering ready-to-use components that can be easily integrated into projects. With AiTreasureBox, users can quickly prototype and deploy AI applications without the need for extensive knowledge in machine learning or deep learning. The tool covers a wide range of tasks such as image classification, text generation, sentiment analysis, object detection, and more. It is designed to be user-friendly and accessible to both beginners and experienced developers, making AI development more efficient and accessible to a wider audience.

glide
Glide is a cloud-native LLM gateway that provides a unified REST API for accessing various large language models (LLMs) from different providers. It handles LLMOps tasks such as model failover, caching, key management, and more, making it easy to integrate LLMs into applications. Glide supports popular LLM providers like OpenAI, Anthropic, Azure OpenAI, AWS Bedrock (Titan), Cohere, Google Gemini, OctoML, and Ollama. It offers high availability, performance, and observability, and provides SDKs for Python and NodeJS to simplify integration.

jupyter-ai
Jupyter AI connects generative AI with Jupyter notebooks. It provides a user-friendly and powerful way to explore generative AI models in notebooks and improve your productivity in JupyterLab and the Jupyter Notebook. Specifically, Jupyter AI offers: * An `%%ai` magic that turns the Jupyter notebook into a reproducible generative AI playground. This works anywhere the IPython kernel runs (JupyterLab, Jupyter Notebook, Google Colab, Kaggle, VSCode, etc.). * A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant. * Support for a wide range of generative model providers, including AI21, Anthropic, AWS, Cohere, Gemini, Hugging Face, NVIDIA, and OpenAI. * Local model support through GPT4All, enabling use of generative AI models on consumer grade machines with ease and privacy.

langchain_dart
LangChain.dart is a Dart port of the popular LangChain Python framework created by Harrison Chase. LangChain provides a set of ready-to-use components for working with language models and a standard interface for chaining them together to formulate more advanced use cases (e.g. chatbots, Q&A with RAG, agents, summarization, extraction, etc.). The components can be grouped into a few core modules: * **Model I/O:** LangChain offers a unified API for interacting with various LLM providers (e.g. OpenAI, Google, Mistral, Ollama, etc.), allowing developers to switch between them with ease. Additionally, it provides tools for managing model inputs (prompt templates and example selectors) and parsing the resulting model outputs (output parsers). * **Retrieval:** assists in loading user data (via document loaders), transforming it (with text splitters), extracting its meaning (using embedding models), storing (in vector stores) and retrieving it (through retrievers) so that it can be used to ground the model's responses (i.e. Retrieval-Augmented Generation or RAG). * **Agents:** "bots" that leverage LLMs to make informed decisions about which available tools (such as web search, calculators, database lookup, etc.) to use to accomplish the designated task. The different components can be composed together using the LangChain Expression Language (LCEL).

infinity
Infinity is an AI-native database designed for LLM applications, providing incredibly fast full-text and vector search capabilities. It supports a wide range of data types, including vectors, full-text, and structured data, and offers a fused search feature that combines multiple embeddings and full text. Infinity is easy to use, with an intuitive Python API and a single-binary architecture that simplifies deployment. It achieves high performance, with 0.1 milliseconds query latency on million-scale vector datasets and up to 15K QPS.
For similar jobs

h2ogpt
h2oGPT is an Apache V2 open-source project that allows users to query and summarize documents or chat with local private GPT LLMs. It features a private offline database of any documents (PDFs, Excel, Word, Images, Video Frames, Youtube, Audio, Code, Text, MarkDown, etc.), a persistent database (Chroma, Weaviate, or in-memory FAISS) using accurate embeddings (instructor-large, all-MiniLM-L6-v2, etc.), and efficient use of context using instruct-tuned LLMs (no need for LangChain's few-shot approach). h2oGPT also offers parallel summarization and extraction, reaching an output of 80 tokens per second with the 13B LLaMa2 model, HYDE (Hypothetical Document Embeddings) for enhanced retrieval based upon LLM responses, a variety of models supported (LLaMa2, Mistral, Falcon, Vicuna, WizardLM. With AutoGPTQ, 4-bit/8-bit, LORA, etc.), GPU support from HF and LLaMa.cpp GGML models, and CPU support using HF, LLaMa.cpp, and GPT4ALL models. Additionally, h2oGPT provides Attention Sinks for arbitrarily long generation (LLaMa-2, Mistral, MPT, Pythia, Falcon, etc.), a UI or CLI with streaming of all models, the ability to upload and view documents through the UI (control multiple collaborative or personal collections), Vision Models LLaVa, Claude-3, Gemini-Pro-Vision, GPT-4-Vision, Image Generation Stable Diffusion (sdxl-turbo, sdxl) and PlaygroundAI (playv2), Voice STT using Whisper with streaming audio conversion, Voice TTS using MIT-Licensed Microsoft Speech T5 with multiple voices and Streaming audio conversion, Voice TTS using MPL2-Licensed TTS including Voice Cloning and Streaming audio conversion, AI Assistant Voice Control Mode for hands-free control of h2oGPT chat, Bake-off UI mode against many models at the same time, Easy Download of model artifacts and control over models like LLaMa.cpp through the UI, Authentication in the UI by user/password via Native or Google OAuth, State Preservation in the UI by user/password, Linux, Docker, macOS, and Windows support, Easy Windows Installer for Windows 10 64-bit (CPU/CUDA), Easy macOS Installer for macOS (CPU/M1/M2), Inference Servers support (oLLaMa, HF TGI server, vLLM, Gradio, ExLLaMa, Replicate, OpenAI, Azure OpenAI, Anthropic), OpenAI-compliant, Server Proxy API (h2oGPT acts as drop-in-replacement to OpenAI server), Python client API (to talk to Gradio server), JSON Mode with any model via code block extraction. Also supports MistralAI JSON mode, Claude-3 via function calling with strict Schema, OpenAI via JSON mode, and vLLM via guided_json with strict Schema, Web-Search integration with Chat and Document Q/A, Agents for Search, Document Q/A, Python Code, CSV frames (Experimental, best with OpenAI currently), Evaluate performance using reward models, and Quality maintained with over 1000 unit and integration tests taking over 4 GPU-hours.

mistral.rs
Mistral.rs is a fast LLM inference platform written in Rust. We support inference on a variety of devices, quantization, and easy-to-use application with an Open-AI API compatible HTTP server and Python bindings.

ollama
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications. Ollama is designed to be easy to use and accessible to developers of all levels. It is open source and available for free on GitHub.

llama-cpp-agent
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output (objects). It provides a simple yet robust interface and supports llama-cpp-python and OpenAI endpoints with GBNF grammar support (like the llama-cpp-python server) and the llama.cpp backend server. It works by generating a formal GGML-BNF grammar of the user defined structures and functions, which is then used by llama.cpp to generate text valid to that grammar. In contrast to most GBNF grammar generators it also supports nested objects, dictionaries, enums and lists of them.

llama_ros
This repository provides a set of ROS 2 packages to integrate llama.cpp into ROS 2. By using the llama_ros packages, you can easily incorporate the powerful optimization capabilities of llama.cpp into your ROS 2 projects by running GGUF-based LLMs and VLMs.

MITSUHA
OneReality is a virtual waifu/assistant that you can speak to through your mic and it'll speak back to you! It has many features such as: * You can speak to her with a mic * It can speak back to you * Has short-term memory and long-term memory * Can open apps * Smarter than you * Fluent in English, Japanese, Korean, and Chinese * Can control your smart home like Alexa if you set up Tuya (more info in Prerequisites) It is built with Python, Llama-cpp-python, Whisper, SpeechRecognition, PocketSphinx, VITS-fast-fine-tuning, VITS-simple-api, HyperDB, Sentence Transformers, and Tuya Cloud IoT.

wenxin-starter
WenXin-Starter is a spring-boot-starter for Baidu's "Wenxin Qianfan WENXINWORKSHOP" large model, which can help you quickly access Baidu's AI capabilities. It fully integrates the official API documentation of Wenxin Qianfan. Supports text-to-image generation, built-in dialogue memory, and supports streaming return of dialogue. Supports QPS control of a single model and supports queuing mechanism. Plugins will be added soon.

FlexFlow
FlexFlow Serve is an open-source compiler and distributed system for **low latency**, **high performance** LLM serving. FlexFlow Serve outperforms existing systems by 1.3-2.0x for single-node, multi-GPU inference and by 1.4-2.4x for multi-node, multi-GPU inference.