
gpt-load
Multi-channel AI proxy with intelligent key rotation. 智能密钥轮询的多渠道 AI 代理。
Stars: 5139

GPT-Load is a high-performance, enterprise-grade AI API transparent proxy service designed for enterprises and developers needing to integrate multiple AI services. Built with Go, it features intelligent key management, load balancing, and comprehensive monitoring capabilities for high-concurrency production environments. The tool serves as a transparent proxy service, preserving native API formats of various AI service providers like OpenAI, Google Gemini, and Anthropic Claude. It supports dynamic configuration, distributed leader-follower deployment, and a Vue 3-based web management interface. GPT-Load is production-ready with features like dual authentication, graceful shutdown, and error recovery.
README:
A high-performance, enterprise-grade AI API transparent proxy service designed specifically for enterprises and developers who need to integrate multiple AI services. Built with Go, featuring intelligent key management, load balancing, and comprehensive monitoring capabilities, designed for high-concurrency production environments.
For detailed documentation, please visit Official Documentation
- Transparent Proxy: Complete preservation of native API formats, supporting OpenAI, Google Gemini, and Anthropic Claude among other formats
- Intelligent Key Management: High-performance key pool with group-based management, automatic rotation, and failure recovery
- Load Balancing: Weighted load balancing across multiple upstream endpoints to enhance service availability
- Smart Failure Handling: Automatic key blacklist management and recovery mechanisms to ensure service continuity
- Dynamic Configuration: System settings and group configurations support hot-reload without requiring restarts
- Enterprise Architecture: Distributed leader-follower deployment supporting horizontal scaling and high availability
- Modern Management: Vue 3-based web management interface that is intuitive and user-friendly
- Comprehensive Monitoring: Real-time statistics, health checks, and detailed request logging
- High-Performance Design: Zero-copy streaming, connection pool reuse, and atomic operations
- Production Ready: Graceful shutdown, error recovery, and comprehensive security mechanisms
- Dual Authentication: Separate authentication for management and proxy, with proxy authentication supporting global and group-level keys
GPT-Load serves as a transparent proxy service, completely preserving the native API formats of various AI service providers:
- OpenAI Format: Official OpenAI API, Azure OpenAI, and other OpenAI-compatible services
- Google Gemini Format: Native APIs for Gemini Pro, Gemini Pro Vision, and other models
- Anthropic Claude Format: Claude series models, supporting high-quality conversations and text generation
- Go 1.23+ (for source builds)
- Docker (for containerized deployment)
- MySQL, PostgreSQL, or SQLite (for database storage)
- Redis (for caching and distributed coordination, optional)
docker run -d --name gpt-load \
-p 3001:3001 \
-e AUTH_KEY=your-secure-key-here \
-v "$(pwd)/data":/app/data \
ghcr.io/tbphp/gpt-load:latest
Please change
your-secure-key-here
to a strong password (never use the default value), then you can log in to the management interface: http://localhost:3001
Installation Commands:
# Create Directory
mkdir -p gpt-load && cd gpt-load
# Download configuration files
wget https://raw.githubusercontent.com/tbphp/gpt-load/refs/heads/main/docker-compose.yml
wget -O .env https://raw.githubusercontent.com/tbphp/gpt-load/refs/heads/main/.env.example
# Edit the .env file and change AUTH_KEY to a strong password. Never use default or simple keys like sk-123456.
# Start services
docker compose up -d
Before deployment, you must change the default admin key (AUTH_KEY). A recommended format is: sk-prod-[32-character random string].
The default installation uses the SQLite version, which is suitable for lightweight, single-instance applications.
If you need to install MySQL, PostgreSQL, and Redis, please uncomment the required services in the docker-compose.yml
file, configure the corresponding environment variables, and restart.
Other Commands:
# Check service status
docker compose ps
# View logs
docker compose logs -f
# Restart Service
docker compose down && docker compose up -d
# Update to latest version
docker compose pull && docker compose down && docker compose up -d
After deployment:
- Access Web Management Interface: http://localhost:3001
- API Proxy Address: http://localhost:3001/proxy
Use your modified AUTH_KEY to log in to the management interface.
Source build requires a locally installed database (SQLite, MySQL, or PostgreSQL) and Redis (optional).
# Clone and build
git clone https://github.com/tbphp/gpt-load.git
cd gpt-load
go mod tidy
# Create configuration
cp .env.example .env
# Edit the .env file and change AUTH_KEY to a strong password. Never use default or simple keys like sk-123456.
# Modify DATABASE_DSN and REDIS_DSN configurations in .env
# REDIS_DSN is optional; if not configured, memory storage will be enabled
# Run
make run
After deployment:
- Access Web Management Interface: http://localhost:3001
- API Proxy Address: http://localhost:3001/proxy
Use your modified AUTH_KEY to log in to the management interface.
Cluster deployment requires all nodes to connect to the same MySQL (or PostgreSQL) and Redis, with Redis being mandatory. It's recommended to use unified distributed MySQL and Redis clusters.
Deployment Requirements:
- All nodes must configure identical
AUTH_KEY
,DATABASE_DSN
,REDIS_DSN
- Leader-follower architecture where follower nodes must configure environment variable:
IS_SLAVE=true
For details, please refer to Cluster Deployment Documentation
GPT-Load adopts a dual-layer configuration architecture:
- Characteristics: Read at application startup, immutable during runtime, requires application restart to take effect
- Purpose: Infrastructure configuration such as database connections, server ports, authentication keys, etc.
-
Management: Set via
.env
files or system environment variables
- System Settings: Stored in database, providing unified behavioral standards for the entire application
- Group Configuration: Behavior parameters customized for specific groups, can override system settings
- Configuration Priority: Group Configuration > System Settings > Environment Configuration
- Characteristics: Supports hot-reload, takes effect immediately after modification without application restart
Static Configuration (Environment Variables)
Server Configuration:
Setting | Environment Variable | Default | Description |
---|---|---|---|
Service Port | PORT |
3001 | HTTP server listening port |
Service Address | HOST |
0.0.0.0 | HTTP server binding address |
Read Timeout | SERVER_READ_TIMEOUT |
60 | HTTP server read timeout (seconds) |
Write Timeout | SERVER_WRITE_TIMEOUT |
600 | HTTP server write timeout (seconds) |
Idle Timeout | SERVER_IDLE_TIMEOUT |
120 | HTTP connection idle timeout (seconds) |
Graceful Shutdown Timeout | SERVER_GRACEFUL_SHUTDOWN_TIMEOUT |
10 | Service graceful shutdown wait time (seconds) |
Follower Mode | IS_SLAVE |
false | Follower node identifier for cluster deployment |
Timezone | TZ |
Asia/Shanghai |
Specify timezone |
Security Configuration:
Setting | Environment Variable | Default | Description |
---|---|---|---|
Admin Key | AUTH_KEY |
- | Access authentication key for the management end, please change it to a strong password |
Encryption Key | ENCRYPTION_KEY |
- | Encrypts API keys at rest. Supports any string or leave empty to disable encryption. See Data Encryption Migration |
Database Configuration:
Setting | Environment Variable | Default | Description |
---|---|---|---|
Database Connection | DATABASE_DSN |
./data/gpt-load.db |
Database connection string (DSN) or file path |
Redis Connection | REDIS_DSN |
- | Redis connection string, uses memory storage when empty |
Performance & CORS Configuration:
Setting | Environment Variable | Default | Description |
---|---|---|---|
Max Concurrent Requests | MAX_CONCURRENT_REQUESTS |
100 | Maximum concurrent requests allowed by system |
Enable CORS | ENABLE_CORS |
false | Whether to enable Cross-Origin Resource Sharing |
Allowed Origins | ALLOWED_ORIGINS |
- | Allowed origins, comma-separated |
Allowed Methods | ALLOWED_METHODS |
GET,POST,PUT,DELETE,OPTIONS |
Allowed HTTP methods |
Allowed Headers | ALLOWED_HEADERS |
* |
Allowed request headers, comma-separated |
Allow Credentials | ALLOW_CREDENTIALS |
false | Whether to allow sending credentials |
Logging Configuration:
Setting | Environment Variable | Default | Description |
---|---|---|---|
Log Level | LOG_LEVEL |
info |
Log level: debug, info, warn, error |
Log Format | LOG_FORMAT |
text |
Log format: text, json |
Enable File Logging | LOG_ENABLE_FILE |
false | Whether to enable file log output |
Log File Path | LOG_FILE_PATH |
./data/logs/app.log |
Log file storage path |
Proxy Configuration:
GPT-Load automatically reads proxy settings from environment variables to make requests to upstream AI providers.
Setting | Environment Variable | Default | Description |
---|---|---|---|
HTTP Proxy | HTTP_PROXY |
- | Proxy server address for HTTP requests |
HTTPS Proxy | HTTPS_PROXY |
- | Proxy server address for HTTPS requests |
No Proxy | NO_PROXY |
- | Comma-separated list of hosts or domains to bypass the proxy |
Supported Proxy Protocol Formats:
-
HTTP:
http://user:pass@host:port
-
HTTPS:
https://user:pass@host:port
-
SOCKS5:
socks5://user:pass@host:port
Dynamic Configuration (Hot-Reload)
Basic Settings:
Setting | Field Name | Default | Group Override | Description |
---|---|---|---|---|
Project URL | app_url |
http://localhost:3001 |
❌ | Project base URL |
Global Proxy Keys | proxy_keys |
Initial value from AUTH_KEY
|
❌ | Globally effective proxy keys, comma-separated |
Log Retention Days | request_log_retention_days |
7 | ❌ | Request log retention days, 0 for no cleanup |
Log Write Interval | request_log_write_interval_minutes |
1 | ❌ | Log write to database cycle (minutes) |
Enable Request Body Logging | enable_request_body_logging |
false | ✅ | Whether to log complete request body content in request logs |
Request Settings:
Setting | Field Name | Default | Group Override | Description |
---|---|---|---|---|
Request Timeout | request_timeout |
600 | ✅ | Forward request complete lifecycle timeout (seconds) |
Connection Timeout | connect_timeout |
15 | ✅ | Timeout for establishing connection with upstream service (seconds) |
Idle Connection Timeout | idle_conn_timeout |
120 | ✅ | HTTP client idle connection timeout (seconds) |
Response Header Timeout | response_header_timeout |
600 | ✅ | Timeout for waiting upstream response headers (seconds) |
Max Idle Connections | max_idle_conns |
100 | ✅ | Connection pool maximum total idle connections |
Max Idle Connections Per Host | max_idle_conns_per_host |
50 | ✅ | Maximum idle connections per upstream host |
Proxy URL | proxy_url |
- | ✅ | HTTP/HTTPS proxy for forwarding requests, uses environment if empty |
Key Configuration:
Setting | Field Name | Default | Group Override | Description |
---|---|---|---|---|
Max Retries | max_retries |
3 | ✅ | Maximum retry count using different keys for single request |
Blacklist Threshold | blacklist_threshold |
3 | ✅ | Number of consecutive failures before key enters blacklist |
Key Validation Interval | key_validation_interval_minutes |
60 | ✅ | Background scheduled key validation cycle (minutes) |
Key Validation Concurrency | key_validation_concurrency |
10 | ✅ | Concurrency for background validation of invalid keys |
Key Validation Timeout | key_validation_timeout_seconds |
20 | ✅ | API request timeout for validating individual keys in background (seconds) |
GPT-Load supports encrypted storage of API keys. You can enable, disable, or change the encryption key at any time.
View Data Encryption Migration Details
-
Enable Encryption: Encrypt plaintext data for storage - Use
--to <new-key>
-
Disable Encryption: Decrypt encrypted data to plaintext - Use
--from <current-key>
-
Change Encryption Key: Replace the encryption key - Use
--from <current-key> --to <new-key>
# 1. Update the image (ensure using the latest version)
docker compose pull
# 2. Stop the service
docker compose down
# 3. Backup the database (strongly recommended)
# Before migration, you must manually backup the database or export your keys to avoid key loss due to operations or exceptions.
# 4. Execute migration command
# Enable encryption (your-32-char-secret-key is your key, recommend using 32+ character random string)
docker compose run --rm gpt-load migrate-keys --to "your-32-char-secret-key"
# Disable encryption
docker compose run --rm gpt-load migrate-keys --from "your-current-key"
# Change encryption key
docker compose run --rm gpt-load migrate-keys --from "old-key" --to "new-32-char-secret-key"
# 5. Update configuration file
# Edit .env file, set ENCRYPTION_KEY to match the --to parameter
# If disabling encryption, remove ENCRYPTION_KEY or set it to empty
vim .env
# Add or modify: ENCRYPTION_KEY=your-32-char-secret-key
# 6. Restart the service
docker compose up -d
# 1. Stop the service
# Stop the running service process (Ctrl+C or kill process)
# 2. Backup the database (strongly recommended)
# Before migration, you must manually backup the database or export your keys to avoid key loss due to operations or exceptions.
# 3. Execute migration command
# Enable encryption
make migrate-keys ARGS="--to your-32-char-secret-key"
# Disable encryption
make migrate-keys ARGS="--from your-current-key"
# Change encryption key
make migrate-keys ARGS="--from old-key --to new-32-char-secret-key"
# 4. Update configuration file
# Edit .env file, set ENCRYPTION_KEY to match the --to parameter
echo "ENCRYPTION_KEY=your-32-char-secret-key" >> .env
# 5. Restart the service
make run
- Once ENCRYPTION_KEY is lost, encrypted data CANNOT be recovered! Please securely backup this key. Consider using a password manager or secure key management system
- Service must be stopped before migration to avoid data inconsistency
- Strongly recommended to backup the database in case migration fails and recovery is needed
- Keys should use 32 characters or longer random strings for security
- Ensure
ENCRYPTION_KEY
in.env
matches the--to
parameter after migration - If disabling encryption, remove or clear the
ENCRYPTION_KEY
configuration
# Generate secure random key (32 characters)
openssl rand -base64 32 | tr -d "=+/" | cut -c1-32
Access the management console at: http://localhost:3001 (default address)
The web management interface provides the following features:
- Dashboard: Real-time statistics and system status overview
- Key Management: Create and configure AI service provider groups, add, delete, and monitor API keys
- Request Logs: Detailed request history and debugging information
- System Settings: Global configuration management and hot-reload
Proxy Interface Invocation
GPT-Load routes requests to different AI services through group names. Usage is as follows:
http://localhost:3001/proxy/{group_name}/{original_api_path}
-
{group_name}
: Group name created in the management interface -
{original_api_path}
: Maintain complete consistency with original AI service paths
Configure Proxy Keys in the web management interface, which supports system-level and group-level proxy keys.
- Authentication Method: Consistent with the native API, but replace the original key with the configured proxy key.
- Key Scope: Global Proxy Keys configured in system settings can be used in all groups. Group Proxy Keys configured in a group are only valid for the current group.
- Format: Multiple keys are separated by commas.
Assuming a group named openai
was created:
Original invocation:
curl -X POST https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer sk-your-openai-key" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4.1-mini", "messages": [{"role": "user", "content": "Hello"}]}'
Proxy invocation:
curl -X POST http://localhost:3001/proxy/openai/v1/chat/completions \
-H "Authorization: Bearer your-proxy-key" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-4.1-mini", "messages": [{"role": "user", "content": "Hello"}]}'
Changes required:
- Replace
https://api.openai.com
withhttp://localhost:3001/proxy/openai
- Replace original API Key with the Proxy Key
Assuming a group named gemini
was created:
Original invocation:
curl -X POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent?key=your-gemini-key \
-H "Content-Type: application/json" \
-d '{"contents": [{"parts": [{"text": "Hello"}]}]}'
Proxy invocation:
curl -X POST http://localhost:3001/proxy/gemini/v1beta/models/gemini-2.5-pro:generateContent?key=your-proxy-key \
-H "Content-Type: application/json" \
-d '{"contents": [{"parts": [{"text": "Hello"}]}]}'
Changes required:
- Replace
https://generativelanguage.googleapis.com
withhttp://localhost:3001/proxy/gemini
- Replace
key=your-gemini-key
in URL parameter with the Proxy Key
Assuming a group named anthropic
was created:
Original invocation:
curl -X POST https://api.anthropic.com/v1/messages \
-H "x-api-key: sk-ant-api03-your-anthropic-key" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{"model": "claude-sonnet-4-20250514", "messages": [{"role": "user", "content": "Hello"}]}'
Proxy invocation:
curl -X POST http://localhost:3001/proxy/anthropic/v1/messages \
-H "x-api-key: your-proxy-key" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{"model": "claude-sonnet-4-20250514", "messages": [{"role": "user", "content": "Hello"}]}'
Changes required:
- Replace
https://api.anthropic.com
withhttp://localhost:3001/proxy/anthropic
- Replace the original API Key in
x-api-key
header with the Proxy Key
OpenAI Format:
-
/v1/chat/completions
- Chat conversations -
/v1/completions
- Text completion -
/v1/embeddings
- Text embeddings -
/v1/models
- Model list - And all other OpenAI-compatible interfaces
Gemini Format:
-
/v1beta/models/*/generateContent
- Content generation -
/v1beta/models
- Model list - And all other Gemini native interfaces
Anthropic Format:
-
/v1/messages
- Message conversations -
/v1/models
- Model list (if available) - And all other Anthropic native interfaces
OpenAI Python SDK:
from openai import OpenAI
client = OpenAI(
api_key="your-proxy-key", # Use the proxy key
base_url="http://localhost:3001/proxy/openai" # Use proxy endpoint
)
response = client.chat.completions.create(
model="gpt-4.1-mini",
messages=[{"role": "user", "content": "Hello"}]
)
Google Gemini SDK (Python):
import google.generativeai as genai
# Configure API key and base URL
genai.configure(
api_key="your-proxy-key", # Use the proxy key
client_options={"api_endpoint": "http://localhost:3001/proxy/gemini"}
)
model = genai.GenerativeModel('gemini-2.5-pro')
response = model.generate_content("Hello")
Anthropic SDK (Python):
from anthropic import Anthropic
client = Anthropic(
api_key="your-proxy-key", # Use the proxy key
base_url="http://localhost:3001/proxy/anthropic" # Use proxy endpoint
)
response = client.messages.create(
model="claude-sonnet-4-20250514",
messages=[{"role": "user", "content": "Hello"}]
)
Important Note: As a transparent proxy service, GPT-Load completely preserves the native API formats and authentication methods of various AI services. You only need to replace the endpoint address and use the Proxy Key configured in the management interface for seamless migration.
- New API - Excellent AI model aggregation management and distribution system
Thanks to all the developers who have contributed to GPT-Load!
MIT License - see LICENSE file for details.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for gpt-load
Similar Open Source Tools

gpt-load
GPT-Load is a high-performance, enterprise-grade AI API transparent proxy service designed for enterprises and developers needing to integrate multiple AI services. Built with Go, it features intelligent key management, load balancing, and comprehensive monitoring capabilities for high-concurrency production environments. The tool serves as a transparent proxy service, preserving native API formats of various AI service providers like OpenAI, Google Gemini, and Anthropic Claude. It supports dynamic configuration, distributed leader-follower deployment, and a Vue 3-based web management interface. GPT-Load is production-ready with features like dual authentication, graceful shutdown, and error recovery.

Backlog.md
Backlog.md is a Markdown-native Task Manager & Kanban visualizer for any Git repository. It turns any folder with a Git repo into a self-contained project board powered by plain Markdown files and a zero-config CLI. Features include managing tasks as plain .md files, private & offline usage, instant terminal Kanban visualization, board export, modern web interface, AI-ready CLI, rich query commands, cross-platform support, and MIT-licensed open-source. Users can create tasks, view board, assign tasks to AI, manage documentation, make decisions, and configure settings easily.

optillm
optillm is an OpenAI API compatible optimizing inference proxy implementing state-of-the-art techniques to enhance accuracy and performance of LLMs, focusing on reasoning over coding, logical, and mathematical queries. By leveraging additional compute at inference time, it surpasses frontier models across diverse tasks.

terminator
Terminator is an AI-powered desktop automation tool that is open source, MIT-licensed, and cross-platform. It works across all apps and browsers, inspired by GitHub Actions & Playwright. It is 100x faster than generic AI agents, with over 95% success rate and no vendor lock-in. Users can create automations that work across any desktop app or browser, achieve high success rates without costly consultant armies, and pre-train workflows as deterministic code.

mcp-apache-spark-history-server
The MCP Server for Apache Spark History Server is a tool that connects AI agents to Apache Spark History Server for intelligent job analysis and performance monitoring. It enables AI agents to analyze job performance, identify bottlenecks, and provide insights from Spark History Server data. The server bridges AI agents with existing Apache Spark infrastructure, allowing users to query job details, analyze performance metrics, compare multiple jobs, investigate failures, and generate insights from historical execution data.

wa_llm
WhatsApp Group Summary Bot is an AI-powered tool that joins WhatsApp groups, tracks conversations, and generates intelligent summaries. It features automated group chat responses, LLM-based conversation summaries, knowledge base integration, persistent message history with PostgreSQL, support for multiple message types, group management, and a REST API with Swagger docs. Prerequisites include Docker, Python 3.12+, PostgreSQL with pgvector extension, Voyage AI API key, and a WhatsApp account for the bot. The tool can be quickly set up by cloning the repository, configuring environment variables, starting services, and connecting devices. It offers API usage for loading new knowledge base topics and generating & dispatching summaries to managed groups. The project architecture includes FastAPI backend, WhatsApp Web API client, PostgreSQL database with vector storage, and AI-powered message processing.

gollama
Gollama is a tool designed for managing Ollama models through a Text User Interface (TUI). Users can list, inspect, delete, copy, and push Ollama models, as well as link them to LM Studio. The application offers interactive model selection, sorting by various criteria, and actions using hotkeys. It provides features like sorting and filtering capabilities, displaying model metadata, model linking, copying, pushing, and more. Gollama aims to be user-friendly and useful for managing models, especially for cleaning up old models.

cortex.cpp
Cortex.cpp is an open-source platform designed as the brain for robots, offering functionalities such as vision, speech, language, tabular data processing, and action. It provides an AI platform for running AI models with multi-engine support, hardware optimization with automatic GPU detection, and an OpenAI-compatible API. Users can download models from the Hugging Face model hub, run models, manage resources, and access advanced features like multiple quantizations and engine management. The tool is under active development, promising rapid improvements for users.

BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

llm.nvim
llm.nvim is a universal plugin for a large language model (LLM) designed to enable users to interact with LLM within neovim. Users can customize various LLMs such as gpt, glm, kimi, and local LLM. The plugin provides tools for optimizing code, comparing code, translating text, and more. It also supports integration with free models from Cloudflare, Github models, siliconflow, and others. Users can customize tools, chat with LLM, quickly translate text, and explain code snippets. The plugin offers a flexible window interface for easy interaction and customization.

agentscope
AgentScope is a multi-agent platform designed to empower developers to build multi-agent applications with large-scale models. It features three high-level capabilities: Easy-to-Use, High Robustness, and Actor-Based Distribution. AgentScope provides a list of `ModelWrapper` to support both local model services and third-party model APIs, including OpenAI API, DashScope API, Gemini API, and ollama. It also enables developers to rapidly deploy local model services using libraries such as ollama (CPU inference), Flask + Transformers, Flask + ModelScope, FastChat, and vllm. AgentScope supports various services, including Web Search, Data Query, Retrieval, Code Execution, File Operation, and Text Processing. Example applications include Conversation, Game, and Distribution. AgentScope is released under Apache License 2.0 and welcomes contributions.

StableToolBench
StableToolBench is a new benchmark developed to address the instability of Tool Learning benchmarks. It aims to balance stability and reality by introducing features such as a Virtual API System with caching and API simulators, a new set of solvable queries determined by LLMs, and a Stable Evaluation System using GPT-4. The Virtual API Server can be set up either by building from source or using a prebuilt Docker image. Users can test the server using provided scripts and evaluate models with Solvable Pass Rate and Solvable Win Rate metrics. The tool also includes model experiments results comparing different models' performance.

packages
This repository is a monorepo for NPM packages published under the `@elevenlabs` scope. It contains multiple packages in the `packages` folder. The setup allows for easy development, linking packages, creating new packages, and publishing them with GitHub actions.

rpaframework
RPA Framework is an open-source collection of libraries and tools for Robotic Process Automation (RPA), designed to be used with Robot Framework and Python. It offers well-documented core libraries for Software Robot Developers, optimized for Robocorp Control Room and Developer Tools, and accepts external contributions. The project includes various libraries for tasks like archiving, browser automation, date/time manipulations, cloud services integration, encryption operations, database interactions, desktop automation, document processing, email operations, Excel manipulation, file system operations, FTP interactions, web API interactions, image manipulation, AI services, and more. The development of the repository is Python-based and requires Python version 3.8+, with tooling based on poetry and invoke for compiling, building, and running the package. The project is licensed under the Apache License 2.0.

skpro
skpro is a library for supervised probabilistic prediction in python. It provides `scikit-learn`-like, `scikit-base` compatible interfaces to: * tabular **supervised regressors for probabilistic prediction** \- interval, quantile and distribution predictions * tabular **probabilistic time-to-event and survival prediction** \- instance-individual survival distributions * **metrics to evaluate probabilistic predictions** , e.g., pinball loss, empirical coverage, CRPS, survival losses * **reductions** to turn `scikit-learn` regressors into probabilistic `skpro` regressors, such as bootstrap or conformal * building **pipelines and composite models** , including tuning via probabilistic performance metrics * symbolic **probability distributions** with value domain of `pandas.DataFrame`-s and `pandas`-like interface

StableToolBench
StableToolBench is a new benchmark developed to address the instability of Tool Learning benchmarks. It aims to balance stability and reality by introducing features like Virtual API System, Solvable Queries, and Stable Evaluation System. The benchmark ensures consistency through a caching system and API simulators, filters queries based on solvability using LLMs, and evaluates model performance using GPT-4 with metrics like Solvable Pass Rate and Solvable Win Rate.
For similar tasks

gpt-load
GPT-Load is a high-performance, enterprise-grade AI API transparent proxy service designed for enterprises and developers needing to integrate multiple AI services. Built with Go, it features intelligent key management, load balancing, and comprehensive monitoring capabilities for high-concurrency production environments. The tool serves as a transparent proxy service, preserving native API formats of various AI service providers like OpenAI, Google Gemini, and Anthropic Claude. It supports dynamic configuration, distributed leader-follower deployment, and a Vue 3-based web management interface. GPT-Load is production-ready with features like dual authentication, graceful shutdown, and error recovery.
For similar jobs

sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.

teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.

ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.

classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.

chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.

BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.

griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.