TEN-Agent

TEN-Agent

TEN Agent is the world’s first real-time multimodal agent integrated with the OpenAI Realtime API, RTC, and features weather checks, web search, vision, and RAG capabilities.

Stars: 686

Visit
 screenshot

TEN Agent is an open-source multimodal agent powered by the world’s first real-time multimodal framework, TEN Framework. It offers high-performance real-time multimodal interactions, multi-language and multi-platform support, edge-cloud integration, flexibility beyond model limitations, and real-time agent state management. Users can easily build complex AI applications through drag-and-drop programming, integrating audio-visual tools, databases, RAG, and more.

README:

TEN Agent banner

Follow on X Discussion posts Commits Issues closed PRs Welcome GitHub license

Discord TEN Community

GitHub watchers GitHub forks GitHub stars

README in English 简体中文操作指南 日本語のREADME README in 한국어 README en Español README en Français README in Italiano

Getting Started   •   Create Extensions   •   TEN Framework Repository


🎉 TEN Agent with OpenAI Realtime API and RTC

OpenAI Realtime API provides ultra-low latency, and RTC offers AI noise suppression. TEN Agent has both. Feel free to try it out at agent.theten.ai.

TEN Agent with OpenAI Realtime API and RTC


TEN Agent Features

Aside from being world's the first agent to integrate the OpenAI Realtime API and RTC, TEN Agent also offers the following features:

  1. High-Performance Real-Time Multimodal Interactions: Offers high-performance, low-latency solutions for complex audio-visual AI applications.

  2. Multi-Language and Multi-Platform Support : Supports extension development in C++, Go, Python, etc. Runs on Windows, Mac, Linux, and mobile devices.

  3. Edge-Cloud Integration: Flexibly combines edge and cloud-deployed extensions, balancing privacy, cost, and performance.

  4. Flexibility Beyond Model Limitations: Easily build complex AI applications through simple drag-and-drop programming, integrating audio-visual tools, databases, RAG, and more.

  5. Real-Time Agent State Management: Manages and adjusts agent behavior in real-time for dynamic responsiveness.


Stay Tuned

Before we get started, be sure to star our repository and get instant notifications for all new releases!

TEN star us gif


How to build TEN Agent locally

Prerequisites

Keys

Installation

Minimum system requirements

  • CPU >= 2 Core
  • RAM >= 4 GB

Docker setting on Apple Silicon

You will need to uncheck "Use Rosetta for x86_64/amd64 emulation on Apple Silicon" option for Docker if you are on Apple Silicon, otherwise the server is not going to work.

Docker Setting

Next step

1. Modify config files

In the root of the project, use cp command to create .env from the example.

It will be used to store information for docker compose later.

cp ./.env.example ./.env

2. Setup API keys

Open the .env file and fill in the keys and regions. This is also where you can choose to use any different extensions:

# Agora App ID and Agora App Certificate
AGORA_APP_ID=
# Leave empty unless you have enabled the certificate within the Agora account.
AGORA_APP_CERTIFICATE=

# Azure STT key and region
AZURE_STT_KEY=
AZURE_STT_REGION=

# Azure TTS key and region
AZURE_TTS_KEY=
AZURE_TTS_REGION=

# OpenAI API key
OPENAI_API_KEY=

3. Start agent development containers

In the same directory, run the docker compose up command to compose containers:

docker compose up

4. Enter container and build agent

Open up a separate terminal window, enter the container and build the agent:

docker exec -it ten_agent_dev bash

make build

5. Start the server

Once the build is done, make run-server on port 8080:

make run-server

Finish and verify

TEN Agent

Open up localhost:3000 in browser to play and test the TEN Agent.

TEN Graph Designer

Open up another tab go to localhost:3001, and use Graph Designer to create, connect and edit extensions on canvas.

TEN Graph Designer



Join Community


Code Contributors

TEN


Contribution Guidelines

Contributions are welcome! Please read the contribution guidelines first.


License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for TEN-Agent

Similar Open Source Tools

For similar tasks

For similar jobs