orbit

orbit

An adaptable, open-source context-aware inference engine designed for privacy, control, and independence from proprietary models.

Stars: 133

Visit
 screenshot

ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files. ORBIT uses a flexible adapter architecture to connect your data to AI models, creating specialized 'agents' for specific tasks. It supports scenarios like Knowledge Base Q&A and Chat with Your SQL Database, enabling users to interact with AI models seamlessly. The tool offers a RESTful API for programmatic access and includes features like authentication, API key management, system prompts, health monitoring, and file management. ORBIT is designed to streamline AI inference tasks and facilitate interactions between users and AI models.

README:

ORBIT Logo

License Python Docker Release GitHub stars

ORBIT (Open Retrieval-Based Inference Toolkit) is a middleware platform that provides a unified API for AI inference. It acts as a central gateway, allowing you to connect various local and remote AI models with your private data sources like SQL databases, vector stores, and local files.


🚀 Quick Start

1. Deploy with Docker

Refer to the Docker Setup Guide.

2. Deploy Locally

# Download the latest release
curl -L https://github.com/schmitech/orbit/releases/download/v1.4.3/orbit-1.4.3.tar.gz -o orbit-1.4.3.tar.gz
tar -xzf orbit-1.4.3.tar.gz
cd orbit-1.4.3

# Run the quick setup script (downloads a small model)
cp env.example .env
./install/setup.sh --profile minimal --download-gguf gemma3-270m

# Start the ORBIT server
source venv/bin/activate
./bin/orbit.sh start # Logs are at /logs/orbit.log

Your ORBIT instance is now running at http://localhost:3000.

3. Chat with ORBIT using the CLI tool

Use the orbit-chat CLI tool to interact with your instance.

# The ORBIT chat client is already available after installation.
pip install schmitech-orbit-client

# Start chatting!
orbit-chat
Your browser does not support the video tag.
Using 'orbit-chat' tool. Add -h for usage.

4. Chat with ORBIT using the Web Widget

cd clients/chat-widget/theming-app/
npm install
npm run dev
Your browser does not support the video tag.
Chatting with ORBIT using the widget theming app. The widget is available as an NPM package

🏗️ Architecture Overview

ORBIT Architecture
Click to learn more about the Core Components

Core Components

ORBIT Server (/server/): FastAPI-based inference middleware

  • Inference Layer: Supports multiple LLM providers (OpenAI, Anthropic, Cohere, Ollama, etc.) via unified interface
  • RAG System: Retrieval-Augmented Generation with SQL and Vector DB adapters (file-based / multimodal retrieval underway, it will be available in release 2.0.0)
  • Authentication: PBKDF2-SHA256 with bearer tokens, MongoDB-backed sessions
  • Fault Tolerance: Circuit breaker pattern with exponential backoff for provider failures
  • Content Moderation: Multi-layered safety with LLM Guard and configurable moderators

Configuration (/config/): YAML-based modular configuration

  • Main config in config.yaml with environment variable support
  • Separate configs for adapters, datasources, embeddings, inference, moderators, and rerankers
  • Dynamic loading with validation and resolver system

Client Libraries:

  • React-based chat application with Zustand state management
  • Embeddable chat widget with theming support
  • Node.js and Python API client libraries

Client Libraries

New language clients are available under clients/ with examples and integration tests:

  • Elixir: clients/elixirmake deps, make example, ORBIT_INTEGRATION=1 make test
  • Swift: clients/swiftmake example, ORBIT_INTEGRATION=1 make test
  • Kotlin: clients/kotlinmake example, ORBIT_INTEGRATION=1 make test
  • Scala: clients/scalamake example, ORBIT_INTEGRATION=1 make test
  • PHP: clients/phpmake example, ORBIT_INTEGRATION=1 make test
  • Haskell: clients/haskellmake example
  • Clojure: clients/clojuremake example, ORBIT_INTEGRATION=1 make test
  • Perl: clients/perlmake example, ORBIT_INTEGRATION=1 make test
  • R: clients/rmake example, ORBIT_INTEGRATION=1 make test

Set ORBIT_INTEGRATION=1 to enable integration tests. Optionally set ORBIT_URL (defaults to http://localhost:3000).

Dependencies

  • MongoDB (Required): Authentication, RAG storage, conversation history
  • Redis (Optional): Caching layer
  • Vector DBs (Optional): Chroma, Qdrant, Pinecone, Milvus for semantic search
  • SQL DBs (Optional): PostgreSQL, MySQL, SQLite for structured data retrieval

✨ What Can You Build with ORBIT?

ORBIT uses a flexible adapter architecture to connect your data to AI models. An API key is tied to a specific adapter, effectively creating a specialized "agent" for a certain task. Here are a few examples:

Scenario 1: Knowledge Base Q&A

Provide instant, semantically-aware answers from a knowledge base. Perfect for customer support or internal documentation.

Sample Questions:

  • "What are the summer camp programs for kids?"
  • "How do I register for the contemporary dance class?"

NOTE: You need an instance of MongoDB to enable adapters

Setup the sample SQLite Database with Q/A records about a municipality.

Here's the Sample Q/A datasets for this example. The knowledge base corresponds to a municipal services assistant.

#Login as admin first. Default password is admin123. You should change after installing ORBIT.
./bin/orbit.sh login

# Set up SQL Lite database with Q&A data
./examples/sample-db-setup.sh sqlite
Your browser does not support the video tag.
Setting up the sample SQLite Q/A dataset

Testing with the node client:

# Test using node client
cd clients/node-api
npm install
npm run build
npm run test-query-from-pairs ../../examples/city-qa-pairs.json "http://localhost:3000" "your-api-key" 5 134444
Your browser does not support the video tag.
Testing the Q/A Adapter using the node API client

Scenario 2: Chat with Your SQL Database

Ask questions about your data in natural language and get answers without writing SQL.

Sample Questions:

  • "Show me all orders from John Smith"
  • "What are the top 10 customers by order value?"
# Set up PostgreSQL with a sample schema and data
cd examples/postgres

# Update with  your connection parameters
cp env.example .env

# Test connection
python test_connection.py

# Create the DB
python setup_schema.py

# Install faker to generate synthetic data
pip install faker

# Add sample data
python customer-order.py --action insert --clean --customers 100 --orders 1000

# Create an API key for the SQL intent adapter.
# Make sure you are logged in as admin if auth is enabled in `/config/config.yaml`.
python bin/orbit.py key create \
  --adapter intent-sql-postgres \
  --name "Order Management Assistant" \
  --prompt-file examples/postgres/prompts/customer-assistant-enhanced-prompt.txt

#make sure the sample SQL intent adapter is enabled in `/config/adapters.yaml`
- name: "intent-sql-postgres"
    enabled: false
    type: "retriever"
    datasource: "postgres"
    adapter: "intent"
    implementation: "retrievers.implementations.intent.IntentPostgreSQLRetriever"
    inference_provider: "ollama"

# Start or restart the server
./bin/orbit.sh start --delete-logs

# Start chatting with your new key
orbit-chat --url http://localhost:3000 --api-key YOUR_API_KEY
Your browser does not support the video tag.
Testing the SQL Intent Adapter using the ORBIT CLI tool

⭐ Like this project? Give it a star!

If you find ORBIT useful, please consider giving it a star on GitHub. It helps more people discover the project.

GitHub stars Star History Chart

📖 Documentation

For more detailed information, please refer to the official documentation.

Full API Reference

ORBIT provides a RESTful API for programmatic access. The full API reference with examples is available at /docs (Swagger UI) when the server is running.

Core Chat & Inference

  • POST /v1/chat - MCP protocol chat endpoint (JSON-RPC 2.0 format)
  • GET /health - Overall system health

Authentication

  • POST /auth/login - User authentication
  • POST /auth/logout - End session
  • GET /auth/me - Get current user info
  • POST /auth/register - Register new user
  • POST /auth/change-password - Change user password

API Key Management (Admin)

  • GET /admin/api-keys - List API keys
  • POST /admin/api-keys - Create new API key
  • DELETE /admin/api-keys/{api_key} - Delete API key
  • POST /admin/api-keys/deactivate - Deactivate API key
  • GET /admin/api-keys/{api_key}/status - Get API key status

System Prompts (Admin)

  • GET /admin/prompts - List system prompts
  • POST /admin/prompts - Create system prompt
  • PUT /admin/prompts/{prompt_id} - Update system prompt
  • DELETE /admin/prompts/{prompt_id} - Delete system prompt

Health & Monitoring

  • GET /health - System health overview
  • GET /health/adapters - Adapter health status
  • GET /health/embedding-services - Embedding service status
  • GET /health/mongodb-services - MongoDB connection status
  • GET /health/ready - Readiness check
  • GET /health/system - System resource usage

File Management (Experimental)

  • POST /upload - Single file upload
  • POST /upload/batch - Batch file upload
  • GET /info/{file_id} - File information
  • DELETE /{file_id} - Delete file
  • GET /status - File system status

🤝 Community & Support

📄 License

Apache 2.0 - See LICENSE for details.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for orbit

Similar Open Source Tools

For similar tasks

For similar jobs