esp-ai

esp-ai

The simplest and most cost-effective AI integration solution, enabling any device to achieve intelligent conversation functionality (based on ESP development boards). If you like this project, please give it a Star! | 最简单、最低成本的AI接入方案,让任何物品都能实现智能对话功能(基于ESP开发板)。喜欢本项目的话点个 Star 吧,您的一个 Star 对目前的仓库发展非常重要

Stars: 227

Visit
 screenshot

ESP-AI provides a complete AI conversation solution for your development board, including IAT+LLM+TTS integration solutions for ESP32 series development boards. It can be injected into projects without affecting existing ones. By providing keys from platforms like iFlytek, Jiling, and local services, you can run the services without worrying about interactions between services or between development boards and services. The project's server-side code is based on Node.js, and the hardware code is based on Arduino IDE.

README:

logo.png

ESP-AI

硬件接入AI最简单、最低成本的方案
The simplest and lowest cost solution for any item to access AI

NPM version NPM downloads

Changelog · 中文文档 · English Docs

logo.png

Provides a complete AI dialogue solution for your development board, including but not limited to the IAT(ASR)+LLM+TTS integration solution for the ESP32 series development boards. It is injected into the project as a dependency without affecting existing projects.

For developing the dialogue functionality of robots, you only need to prepare the IAT(ASR), LLM, and TTS services, and leave the rest to ESP-AI.

The server-side code of this project is based on Node.js, and the hardware code is based on Arduino/IDF.

Open source is not easy, click the Star button in the upper right corner to show your support~

🖥 Website

✨ Features

  • ✔️ Customizable offline wake words with multiple built-in wake-up methods (voice, button, serial port, Tianwen ASRPro)
  • ✔️ Complete conversation chain: IAT (ASR) ➡️ LLM/RAG ➡️ TTS
  • ✔️ Fast response algorithms for TTS/LLM, designed to balance service cost while providing the quickest response time
  • ✔️ Supports conversation interruption
  • ✔️ Recognizes user commands (appliance control, singing, etc.) and can dynamically respond based on context
  • ✔️ Configurable
  • ✔️ Plugin-based, allowing integration with any LLM/TTS/IAT using plugins
  • ✔️ One-to-many relationship between service and clients, with independent configuration for each client (hardware)
  • ✔️ Connection supports authentication
  • ✔️ Full-chain streaming data interaction
  • ✔️ Developer platform offers: free services, visual configuration, etc.
  • ✔️ Client configuration webpage provided
  • ✔️ Easily handles high concurrency scenarios (requires Nginx for load balancing)
  • ✔️ Ready to use out of the box

🧐 Next Steps

  • [ ] 🤔 Improve accuracy of built-in offline wake-up (currently recommended to use Tianwen ASRPro)
  • [ ] 🤔 Incorporate AI into user intent inference (e.g., "Turn off the light quickly" and "Turn on the light quickly" will both be recognized as "Turn on the light" command)
  • [ ] 🤔 Online wake word generation
  • [ ] 🤔 Client OTA support
  • [ ] 🤔 Develop plugins in other languages (to avoid relying solely on Node.js for plugin development)
  • [ ] 🤔 Provide a dedicated development board (to avoid current complex wiring)

📦 Install

Server

docker run -itd -p 8080:8080 -v /esp-ai-server/index.js:/server/index.js --name esp-ai-server registry.cn-shanghai.aliyuncs.com/xiaomingio/esp-ai:1.0.0

Client

Download the dependency on the release page and burn it to the development board, see details: Client Install

🔨 Inject Soul into Your Robot with Just a Few Lines of Code

Below are the Node.js and Arduino codes you need to write if you only require dialogue functionality.

📜 Commercial Use

For commercial use, please purchase a commercial license. For more details, see Commercial License.
The commercial license is granted per unit of Company Name + Product Name + Version, for example: "Shenzhen XXX Company Smart Doll V1.0".
The commercial license fee for a single product version is 10,000 RMB, payable in a one-time payment.

License inquiry/report address: https://espai.fun/commercial-license.html.

🏪 Discussion Group

QQ Discussion Group: 854445223

🎥 Case Study Video

bilibili

🤝 Contributing PRs Welcome

Let's build a better esp-ai together.

We warmly invite contributions from everyone. Feel free to share your ideas through Pull Requests or GitHub Issues.

Top Contributors of wangzongming/esp-ai - Last 28 days Performance Stats of wangzongming/esp-ai - Last 28 days
Active Contributors of wangzongming/esp-ai - Last 28 days

🌍 Star geographical distribution

Star Geographical Distribution of wangzongming/esp-ai

quote

If this project has helped your research, please cite us:

@software{ESP-AI,
    title        = {{ESP-AI}},
    author       = {小明IO},
    year         = 2024,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/wangzongming/esp-ai}}
}

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for esp-ai

Similar Open Source Tools

For similar tasks

For similar jobs