whisplay-ai-chatbot

whisplay-ai-chatbot

Pocket-sized AI chatbot built using a RPI Zero 2w / 5

Stars: 281

Visit
 screenshot

Whisplay-AI-Chatbot is a pocket-sized AI chatbot device built using a Raspberry Pi Zero 2w. It features a PiSugar Whisplay HAT with an LCD screen, on-board speaker, and microphone. Users can interact with the chatbot by pressing a button, speaking, and receiving responses, similar to a futuristic walkie-talkie. The tool supports various functionalities such as adjusting volume autonomously, resetting conversation history, local ASR and TTS capabilities, image generation, and integration with APIs like Google Gemini and Grok. It also offers support for LLM8850 AI Accelerator for offline capabilities like ASR, TTS, and LLM API. The chatbot saves conversation history and generated images in a data folder, and users can customize the tool with different enclosure cases available for Pi02 and Pi5 models.

README:

Whisplay-AI-Chatbot

Whisplay AI Chatbot

This is a pocket-sized AI chatbot device built using a Raspberry Pi Zero 2w. Just press the button, speak, and it talks back—like a futuristic walkie-talkie with a mind of its own.

Test Video Playlist: https://www.youtube.com/watch?v=lOVA0Gui-4Q

Tutorial: https://www.youtube.com/watch?v=Nwu2DruSuyI

Tutorial (offline version build on RPi 5):

https://youtu.be/kFmhSTh167U

https://youtu.be/QNbHdJUW6z8

https://youtu.be/xGzvFzdBAwc

Hardware

  • Raspberry Pi zero 2w (Recommand RRi 5, 8G RAM for offline build)
  • PiSugar Whisplay HAT (including LCD screen, on-board speaker and microphone)
  • PiSugar 3 1200mAh

Pre-build Image

Drivers

You need to firstly install the audio drivers for the Whisplay HAT. Follow the instructions in the Whisplay HAT repository.

Installation Steps

  1. Clone the repository:
    git clone https://github.com/PiSugar/whisplay-ai-chatbot.git
    cd whisplay-ai-chatbot
  2. Install dependencies:
    bash install_dependencies.sh
    source ~/.bashrc
    Running source ~/.bashrc is necessary to load the new environment variables.
  3. Create a .env file based on the .env.template file and fill in the necessary environment variables.
  4. Build the project:
    bash build.sh
  5. Start the chatbot service:
    bash run_chatbot.sh
  6. Optionally, set up the chatbot service to start on boot:
    sudo bash startup.sh
    Please note that this will disable the graphical interface and set the system to multi-user mode, which is suitable for headless operation. You can find the output logs at chatbot.log. Running tail -f chatbot.log will also display the logs in real-time.

Build After Code Changes

If you make changes to the node code or just pull the new code from this repository, you need to rebuild the project. You can do this by running:

bash build.sh

If If you encounter ModuleNotFoundError or there's new third-party libraries to the python code, please run the following command to update the dependencies for python:

cd python
pip install -r requirements.txt --break-system-packages

The env template may be updated from time to time. If you want to upgrade your existing .env file based on the latest .env.template, you can run the following command:

bash upgrade-env.sh

Update Environment Variables

If you need to update the environment variables, you can edit the .env file directly. After making changes, please restart the chatbot service with:

sudo systemctl restart chatbot.service

Image Generation

You can enable image generation by setting the IMAGE_GENERATION_SERVER variable in the .env file. Options include: OPENAI, GEMINI, VOLCENGINE.

Then you can use prompts like "A children's book drawing of a veterinarian using a stethoscope to listen to the heartbeat of a baby otter." to generate images.

The generated images will be displayed on the screen and saved in the data/images folder.

Display Battery Level

The battery level display depends on the pisugar-power-manager. If you are using PiSugar2 or PiSugar3, you need to install the pisugar-power-manager first. You can find the installation instructions in the PiSugar Power Manager repository.

Or use the following command to install it:

wget https://cdn.pisugar.com/release/pisugar-power-manager.sh
bash pisugar-power-manager.sh -c release

Data Folder

The chatbot saves conversation history and generated images in the data folder. It's a temporal folder and can be deleted if you want to clear the history.

Enclosure

Whisplay Chatbot Case for Pi02

Whisplay Chatbot Case (FDM) for Pi02

Whisplay Chatbot Case (FDM) for Pi5

Whisplay Chatbot Case (FDM) for Pi5 & LLM8850

LLM8850 Support

If you have a LLM8850 AI Accelerator, you can set up the LLM8850 services for local ASR, TTS, and LLM API to enable offline capabilities.

Please refer to the LLM8850 Integration Guide for detailed setup instructions.

Goals

  • Integrate the tool with the API ✅
  • Enable the AI assistant to adjust the volume autonomously ✅
  • Reset the conversation history if there is no speech for five minutes ✅
  • Support local llm server ✅
  • Support local asr (whisper/vosk) ✅
  • Support local tts (piper) ✅
  • Support image generation (openai/gemini/volcengine) ✅
  • Refactor python render thread, better performance ✅
  • Add Google Gemini API support ✅
  • Add Grok API support ✅
  • RPI camera support ✅
  • Support LLM8850 whisper ✅
  • Support LLM8850 melottsTTS ✅
  • Support LLM8850 Qwen3 llm api (not support tool) ✅
  • Support LLM8850 FastVLM
  • Support LLM8850 image generation
  • Support speaker recognition

Star History

Star History Chart

License

GPL-3.0

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for whisplay-ai-chatbot

Similar Open Source Tools

No tools available

For similar tasks

For similar jobs