
orbit
An adaptable, open-source context-aware inference engine designed for privacy, control, and independence from proprietary models.
Stars: 133

ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files. ORBIT uses a flexible adapter architecture to connect your data to AI models, creating specialized 'agents' for specific tasks. It supports scenarios like Knowledge Base Q&A and Chat with Your SQL Database, enabling users to interact with AI models seamlessly. The tool offers a RESTful API for programmatic access and includes features like authentication, API key management, system prompts, health monitoring, and file management. ORBIT is designed to streamline AI inference tasks and facilitate interactions between users and AI models.
README:
ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files.
Refer to the Docker Setup Guide.
# Download the latest release
curl -L https://github.com/schmitech/orbit/releases/download/v1.4.3/orbit-1.4.3.tar.gz -o orbit-1.4.3.tar.gz
tar -xzf orbit-1.4.3.tar.gz
cd orbit-1.4.3
# Run the quick setup script (downloads a small model)
cp env.example .env
./install/setup.sh --profile minimal --download-gguf gemma3-270m
# Start the ORBIT server
source venv/bin/activate
./bin/orbit.sh start # Logs are at /logs/orbit.log
Your ORBIT instance is now running at http://localhost:3000
.
Use the orbit-chat
CLI tool to interact with your instance.
# The ORBIT chat client is already available after installation.
pip install schmitech-orbit-client
# Start chatting!
orbit-chat
Using 'orbit-chat' tool. Add -h for usage.
cd clients/chat-widget/theming-app/
npm install
npm run dev
Chatting with ORBIT using the widget theming app. The widget is available as an NPM package
Click to learn more about the Core Components
ORBIT Server (/server/
): FastAPI-based inference middleware
- Inference Layer: Supports multiple LLM providers (OpenAI, Anthropic, Cohere, Ollama, etc.) via unified interface
- RAG System: Retrieval-Augmented Generation with SQL and Vector DB adapters (file-based / multimodal retrieval underway, it will be available in release 2.0.0)
- Authentication: PBKDF2-SHA256 with bearer tokens, MongoDB-backed sessions
- Fault Tolerance: Circuit breaker pattern with exponential backoff for provider failures
- Content Moderation: Multi-layered safety with LLM Guard and configurable moderators
Configuration (/config/
): YAML-based modular configuration
- Main config in
config.yaml
with environment variable support - Separate configs for adapters, datasources, embeddings, inference, moderators, and rerankers
- Dynamic loading with validation and resolver system
Client Libraries:
- React-based chat application with Zustand state management
- Embeddable chat widget with theming support
- Node.js and Python API client libraries
New language clients are available under clients/
with examples and integration tests:
- Elixir:
clients/elixir
—make deps
,make example
,ORBIT_INTEGRATION=1 make test
- Swift:
clients/swift
—make example
,ORBIT_INTEGRATION=1 make test
- Kotlin:
clients/kotlin
—make example
,ORBIT_INTEGRATION=1 make test
- Scala:
clients/scala
—make example
,ORBIT_INTEGRATION=1 make test
- PHP:
clients/php
—make example
,ORBIT_INTEGRATION=1 make test
- Haskell:
clients/haskell
—make example
- Clojure:
clients/clojure
—make example
,ORBIT_INTEGRATION=1 make test
- Perl:
clients/perl
—make example
,ORBIT_INTEGRATION=1 make test
- R:
clients/r
—make example
,ORBIT_INTEGRATION=1 make test
Set ORBIT_INTEGRATION=1
to enable integration tests. Optionally set ORBIT_URL
(defaults to http://localhost:3000
).
- MongoDB (Required): Authentication, RAG storage, conversation history
- Redis (Optional): Caching layer
- Vector DBs (Optional): Chroma, Qdrant, Pinecone, Milvus for semantic search
- SQL DBs (Optional): PostgreSQL, MySQL, SQLite for structured data retrieval
ORBIT uses a flexible adapter architecture to connect your data to AI models. An API key is tied to a specific adapter, effectively creating a specialized "agent" for a certain task. Here are a few examples:
Provide instant, semantically-aware answers from a knowledge base. Perfect for customer support or internal documentation.
Sample Questions:
- "What are the summer camp programs for kids?"
- "How do I register for the contemporary dance class?"
NOTE: You need an instance of MongoDB to enable adapters
Here's the Sample Q/A datasets for this example. The knowledge base corresponds to a municipal services assistant.
#Login as admin first. Default password is admin123. You should change after installing ORBIT.
./bin/orbit.sh login
# Set up SQL Lite database with Q&A data
./examples/sample-db-setup.sh sqlite
Setting up the sample SQLite Q/A dataset
# Test using node client
cd clients/node-api
npm install
npm run build
npm run test-query-from-pairs ../../examples/city-qa-pairs.json "http://localhost:3000" "your-api-key" 5 134444
Testing the Q/A Adapter using the node API client
Ask questions about your data in natural language and get answers without writing SQL.
Sample Questions:
- "Show me all orders from John Smith"
- "What are the top 10 customers by order value?"
# Set up PostgreSQL with a sample schema and data
cd examples/postgres
# Update with your connection parameters
cp env.example .env
# Test connection
python test_connection.py
# Create the DB
python setup_schema.py
# Install faker to generate synthetic data
pip install faker
# Add sample data
python customer-order.py --action insert --clean --customers 100 --orders 1000
# Create an API key for the SQL intent adapter.
# Make sure you are logged in as admin if auth is enabled in `/config/config.yaml`.
python bin/orbit.py key create \
--adapter intent-sql-postgres \
--name "Order Management Assistant" \
--prompt-file examples/postgres/prompts/customer-assistant-enhanced-prompt.txt
#make sure the sample SQL intent adapter is enabled in `/config/adapters.yaml`
- name: "intent-sql-postgres"
enabled: false
type: "retriever"
datasource: "postgres"
adapter: "intent"
implementation: "retrievers.implementations.intent.IntentPostgreSQLRetriever"
inference_provider: "ollama"
# Start or restart the server
./bin/orbit.sh start --delete-logs
# Start chatting with your new key
orbit-chat --url http://localhost:3000 --api-key YOUR_API_KEY
Testing the SQL Intent Adapter using the ORBIT CLI tool
If you find ORBIT useful, please consider giving it a star on GitHub. It helps more people discover the project.
For more detailed information, please refer to the official documentation.
- Installation Guide
- Configuration
- Authentication
- RAG & Adapters
- Development Roadmap
- Contributing Guide
Full API Reference
ORBIT provides a RESTful API for programmatic access. The full API reference with examples is available at /docs
(Swagger UI) when the server is running.
-
POST /v1/chat
- MCP protocol chat endpoint (JSON-RPC 2.0 format) -
GET /health
- Overall system health
-
POST /auth/login
- User authentication -
POST /auth/logout
- End session -
GET /auth/me
- Get current user info -
POST /auth/register
- Register new user -
POST /auth/change-password
- Change user password
-
GET /admin/api-keys
- List API keys -
POST /admin/api-keys
- Create new API key -
DELETE /admin/api-keys/{api_key}
- Delete API key -
POST /admin/api-keys/deactivate
- Deactivate API key -
GET /admin/api-keys/{api_key}/status
- Get API key status
-
GET /admin/prompts
- List system prompts -
POST /admin/prompts
- Create system prompt -
PUT /admin/prompts/{prompt_id}
- Update system prompt -
DELETE /admin/prompts/{prompt_id}
- Delete system prompt
-
GET /health
- System health overview -
GET /health/adapters
- Adapter health status -
GET /health/embedding-services
- Embedding service status -
GET /health/mongodb-services
- MongoDB connection status -
GET /health/ready
- Readiness check -
GET /health/system
- System resource usage
-
POST /upload
- Single file upload -
POST /upload/batch
- Batch file upload -
GET /info/{file_id}
- File information -
DELETE /{file_id}
- Delete file -
GET /status
- File system status
- Questions? Open an issue
- Updates: Check the changelog
- Commercial Support: Contact schmitech.ai
- Maintained by: Remsy Schmilinsky
Apache 2.0 - See LICENSE for details.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for orbit
Similar Open Source Tools

orbit
ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files. ORBIT uses a flexible adapter architecture to connect your data to AI models, creating specialized 'agents' for specific tasks. It supports scenarios like Knowledge Base Q&A and Chat with Your SQL Database, enabling users to interact with AI models seamlessly. The tool offers a RESTful API for programmatic access and includes features like authentication, API key management, system prompts, health monitoring, and file management. ORBIT is designed to streamline AI inference tasks and facilitate interactions between users and AI models.

backend.ai
Backend.AI is a streamlined, container-based computing cluster platform that hosts popular computing/ML frameworks and diverse programming languages, with pluggable heterogeneous accelerator support including CUDA GPU, ROCm GPU, TPU, IPU and other NPUs. It allocates and isolates the underlying computing resources for multi-tenant computation sessions on-demand or in batches with customizable job schedulers with its own orchestrator. All its functions are exposed as REST/GraphQL/WebSocket APIs.

amazon-q-developer-cli
The `amazon-q-developer-cli` monorepo houses core code for the Amazon Q Developer desktop app and CLI. It includes projects like autocomplete, dashboard, figterm, q CLI, fig_desktop, fig_input_method, VSCode plugin, and JetBrains plugin. The repo also contains build scripts, internal rust crates, internal npm packages, protocol buffer message specification, and integration tests. The architecture involves different components communicating via IPC.

pentagi
PentAGI is an innovative tool for automated security testing that leverages cutting-edge artificial intelligence technologies. It is designed for information security professionals, researchers, and enthusiasts who need a powerful and flexible solution for conducting penetration tests. The tool provides secure and isolated operations in a sandboxed Docker environment, fully autonomous AI-powered agent for penetration testing steps, a suite of 20+ professional security tools, smart memory system for storing research results, web intelligence for gathering information, integration with external search systems, team delegation system, comprehensive monitoring and reporting, modern interface, API integration, persistent storage, scalable architecture, self-hosted solution, flexible authentication, and quick deployment through Docker Compose.

forge
Forge is a powerful open-source tool for building modern web applications. It provides a simple and intuitive interface for developers to quickly scaffold and deploy projects. With Forge, you can easily create custom components, manage dependencies, and streamline your development workflow. Whether you are a beginner or an experienced developer, Forge offers a flexible and efficient solution for your web development needs.

openai-edge-tts
This project provides a local, OpenAI-compatible text-to-speech (TTS) API using `edge-tts`. It emulates the OpenAI TTS endpoint (`/v1/audio/speech`), enabling users to generate speech from text with various voice options and playback speeds, just like the OpenAI API. `edge-tts` uses Microsoft Edge's online text-to-speech service, making it completely free. The project supports multiple audio formats, adjustable playback speed, and voice selection options, providing a flexible and customizable TTS solution for users.

well-architected-iac-analyzer
Well-Architected Infrastructure as Code (IaC) Analyzer is a project demonstrating how generative AI can evaluate infrastructure code for alignment with best practices. It features a modern web application allowing users to upload IaC documents, complete IaC projects, or architecture diagrams for assessment. The tool provides insights into infrastructure code alignment with AWS best practices, offers suggestions for improving cloud architecture designs, and can generate IaC templates from architecture diagrams. Users can analyze CloudFormation, Terraform, or AWS CDK templates, architecture diagrams in PNG or JPEG format, and complete IaC projects with supporting documents. Real-time analysis against Well-Architected best practices, integration with AWS Well-Architected Tool, and export of analysis results and recommendations are included.

mcpd
mcpd is a tool developed by Mozilla AI to declaratively manage Model Context Protocol (MCP) servers, enabling consistent interface for defining and running tools across different environments. It bridges the gap between local development and enterprise deployment by providing secure secrets management, declarative configuration, and seamless environment promotion. mcpd simplifies the developer experience by offering zero-config tool setup, language-agnostic tooling, version-controlled configuration files, enterprise-ready secrets management, and smooth transition from local to production environments.

Flowise
Flowise is a tool that allows users to build customized LLM flows with a drag-and-drop UI. It is open-source and self-hostable, and it supports various deployments, including AWS, Azure, Digital Ocean, GCP, Railway, Render, HuggingFace Spaces, Elestio, Sealos, and RepoCloud. Flowise has three different modules in a single mono repository: server, ui, and components. The server module is a Node backend that serves API logics, the ui module is a React frontend, and the components module contains third-party node integrations. Flowise supports different environment variables to configure your instance, and you can specify these variables in the .env file inside the packages/server folder.

dbt-llm-agent
dbt-llm-agent is an LLM-powered agent designed for interacting with dbt projects. It offers features such as question answering, documentation generation, agentic model interpretation, Postgres integration with pgvector, dbt model selection, question tracking, and upcoming Slack integration. The agent utilizes dbt project parsing, PostgreSQL with pgvector, model selection syntax, large language models like GPT-4, and question tracking to provide its functionalities. Users can set up the agent by checking Python version, cloning the repository, installing dependencies, setting up PostgreSQL with pgvector, configuring environment variables, and initializing the database schema. The agent can be initialized in Cloud Mode, Local Mode, or Source Code Mode to load project metadata. Once set up, users can work with model documentation, ask questions, provide feedback, list models, get detailed model information, and contribute to the project.

TalkWithGemini
Talk With Gemini is a web application that allows users to deploy their private Gemini application for free with one click. It supports Gemini Pro and Gemini Pro Vision models. The application features talk mode for direct communication with Gemini, visual recognition for understanding picture content, full Markdown support, automatic compression of chat records, privacy and security with local data storage, well-designed UI with responsive design, fast loading speed, and multi-language support. The tool is designed to be user-friendly and versatile for various deployment options and language preferences.

openai-kotlin
OpenAI Kotlin API client is a Kotlin client for OpenAI's API with multiplatform and coroutines capabilities. It allows users to interact with OpenAI's API using Kotlin programming language. The client supports various features such as models, chat, images, embeddings, files, fine-tuning, moderations, audio, assistants, threads, messages, and runs. It also provides guides on getting started, chat & function call, file source guide, and assistants. Sample apps are available for reference, and troubleshooting guides are provided for common issues. The project is open-source and licensed under the MIT license, allowing contributions from the community.

AirCasting
AirCasting is a platform for gathering, visualizing, and sharing environmental data. It aims to provide a central hub for environmental data, making it easier for people to access and use this information to make informed decisions about their environment.

ai-commits-intellij-plugin
AI Commits is a plugin for IntelliJ-based IDEs and Android Studio that generates commit messages using git diff and OpenAI. It offers features such as generating commit messages from diff using OpenAI API, computing diff only from selected files and lines in the commit dialog, creating custom prompts for commit message generation, using predefined variables and hints to customize prompts, choosing any of the models available in OpenAI API, setting OpenAI network proxy, and setting custom OpenAI compatible API endpoint.

pastemax
PasteMax is a modern file viewer application designed for developers to easily navigate, search, and copy code from repositories. It provides features such as file tree navigation, token counting, search capabilities, selection management, sorting options, dark mode, binary file detection, and smart file exclusion. Built with Electron, React, and TypeScript, PasteMax is ideal for pasting code into ChatGPT or other language models. Users can download the application or build it from source, and customize file exclusions. Troubleshooting steps are provided for common issues, and contributions to the project are welcome under the MIT License.

eliza
Eliza is a versatile AI agent operating system designed to support various models and connectors, enabling users to create chatbots, autonomous agents, handle business processes, create video game NPCs, and engage in trading. It offers multi-agent and room support, document ingestion and interaction, retrievable memory and document store, and extensibility to create custom actions and clients. Eliza is easy to use and provides a comprehensive solution for AI agent development.
For similar tasks

orbit
ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files. ORBIT uses a flexible adapter architecture to connect your data to AI models, creating specialized 'agents' for specific tasks. It supports scenarios like Knowledge Base Q&A and Chat with Your SQL Database, enabling users to interact with AI models seamlessly. The tool offers a RESTful API for programmatic access and includes features like authentication, API key management, system prompts, health monitoring, and file management. ORBIT is designed to streamline AI inference tasks and facilitate interactions between users and AI models.
For similar jobs

sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.

teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.

ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.

classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.

chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.

BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.

griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.