biniou
a self-hosted webui for 30+ generative ai
Stars: 512
biniou is a self-hosted webui for various GenAI (generative artificial intelligence) tasks. It allows users to generate multimedia content using AI models and chatbots on their own computer, even without a dedicated GPU. The tool can work offline once deployed and required models are downloaded. It offers a wide range of features for text, image, audio, video, and 3D object generation and modification. Users can easily manage the tool through a control panel within the webui, with support for various operating systems and CUDA optimization. biniou is powered by Huggingface and Gradio, providing a cross-platform solution for AI content generation.
README:
biniou is a self-hosted webui for several kinds of GenAI (generative artificial intelligence). You can generate multimedia contents with AI and use a chatbot on your own computer, even without dedicated GPU and starting from 8GB RAM. Can work offline (once deployed and required models downloaded).
GNU/Linux [ OpenSUSE base | RHEL base | Debian base ] โข Windows โข macOS Intel (experimental) โข Docker Documentation โ | Showroom ๐ผ๏ธ | Video presentation (by @Natlamir) ๐๏ธ | Windows installation tutorial (by Fahd Mirza) ๐๏ธ
-
๐ 2025-01-12 : ๐ฅ New high-end Chatbot model ๐ฅ > Phi-4 is now available in biniou, via microsoft/phi-4-gguf. This model is absolutely awesome and cleary out of his league. It only weight 8.4 GB, and -so far- answered perfectly to all submited requests. Definitely a must try !
-
๐ 2025-01-11 : ๐ฅ This week's updates ๐ฅ >
- Add support for Chatbot high-end model bartowski/Qwentile2.5-32B-Instruct-GGUF and Chatbot model cognitivecomputations/Dolphin3.0-Llama3.1-8B-GGUF.
- Update of Chatbot default model to bartowski/Meta-Llama-3.1-8B-Instruct-GGUF and update of Qwen2.5-Coder-7B to bartowski/Qwen2.5.1-Coder-7B-Instruct-GGUF.
- Add support for Flux LoRA models keturn/woodcut-illustrations-Trousset-LoRA, glif-loradex-trainer/AP123_flux_dev_cutaway_style, glif-loradex-trainer/i12bp8_i12bp8_povshots_v1, glif-loradex-trainer/fabian3000_bosch, glif-loradex-trainer/i12bp8_i12bp8_greeksculptures_v1 and leonel4rd/DBZFLUX.
- Add preliminary suppport for ControlNet Flux models Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro and InstantX/FLUX.1-dev-Controlnet-Union. You currently had to manually select these models if you want to use them during inferences.
- Add support for Flux model AlekseyCalvin/AuraFlux_merge_diffusers. Please note that this huge (but highly qualitatives !) model will requires at least 70GB RAM for CPU-only inferences.
-
๐ 2025-01-04 : ๐ฅ This week's updates ๐ฅ >
- Happy new year 2025 to everyone !
- Add support for Chatbot specialized model bartowski/HuatuoGPT-o1-8B-GGUF, Chatbot High-end model DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B-GGUF and Chatbot models bartowski/DRT-o1-7B-GGUF and bartowski/Llama-3.1-8B-Open-SFT-GGUF.
- Add support for Flux LoRA models renderartist/toyboxflux, EvanZhouDev/open-genmoji, aixonlab/FLUX.1-dev-LoRA-Cinematic-1940s, glif-loradex-trainer/shipley_flux_dev_bookFoldArt_v1, fofr/flux-80s-cyberpunk, veryVANYA/ps1-style-flux, alvdansen/the-point-flux, mgwr/Cine-Aesthetic, WizWhite/Wizards_vintage_romance_novel-FLUX, WizWhite/wizard-s-paper-model-universe, leonel4rd/FluxDisney and Weiii722/SouthParkVibe.
- Add support for text2image with Flux models in IP-Adapter module
- Add support for Flux model enhanceaiteam/Mystic. Please note that this huge (but highly qualitatives !) model will requires at least 70GB RAM for CPU-only inferences.
- Bugfix for Flux pipelines following Diffusers upgrade.
-
๐ 2024-12-28 : ๐ฅ This week's updates ๐ฅ >
- Add support for Chatbot specialized models bartowski/Tulu-MathLingo-8B-GGUF, AlicanKiraz0/SenecaLLM-Q4_K_M-GGUF and Chatbot model bartowski/moxin-chat-7b-GGUF.
- Add support for SD 3.5 Large model aipicasso/emi-3.
- Add support for SD 3.5 Medium Turbo (8 steps) model tensorart/stable-diffusion-3.5-medium-turbo.
- Add support for SDXL fast (4 steps) LoRA model alimama-creative/slam-lora-sdxl.
- Add support for Flux LoRA models Datou1111/Yoji_Shinkawa, youknownothing/flux_dreamscape, Datou1111/shou_xin, glif-loradex-trainer/swapagrawal14_tattoo_artist_v1, gokaygokay/Flux-Engrave-LoRA, gokaygokay/Flux-Mosaic-LoRA, glif-loradex-trainer/quitters_flux_dev_gc_boxart and glif-loradex-trainer/x_bulbul_x_Playstation_2_Game_Covers.
- Add support for Flux models black-forest-labs/FLUX.1-schnell, ChuckMcSneed/FLUX.1-dev and sayakpaul/FLUX.1-merged. Please note that these 3 huges (but highly qualitatives !) models will requires at least 70GB RAM for CPU-only inferences.
-
๐ 2024-12-21 : ๐ฅ This week's updates ๐ฅ >
- Add support for Chatbot specialized model bartowski/oxy-1-small-GGUF and upgrade of Chatbot model Granite 3.0 8B to bartowski/granite-3.1-8b-instruct-GGUF.
- Add support for SDXL model Spestly/OdysseyXL-1.0.
- Add support for SD 3.5 "fast" LoRA model sunhaha123/stable-diffusion-3.5-medium-turbo (8 steps inferences).
- Add support for Flux LoRA models SouthbayJay/t-shirt-vector-style-flux, fofr/flux-handwriting, Norod78/chalk-board-drawing-flux, Norod78/humeow-flux, Norod78/ms-fluentui-style-emoji-flux, leonel4rd/Shogunflux, strangerzonehf/Flux-3D-Emojies-LoRA and Nishitbaria/Anime-style-flux-lora-Large.
-
๐ 2024-12-14 : ๐ฅ This week's updates ๐ฅ >
- Add support for Chatbot model bartowski/EuroLLM-9B-Instruct-GGUF and Chatbot "Tiny" model bartowski/Hermes-3-Llama-3.2-3B-GGUF.
- Add support for SDXL model dataautogpt3/ProteusSigma.
- Add support for Flux "fast" LoRA model alimama-creative/FLUX.1-Turbo-Alpha (8 steps inferences).
- Add support for Flux LoRA models strangerzonehf/Flux-Icon-Kit-LoRA, strangerzonehf/Flux-Ghibli-Art-LoRA, UmeAiRT/FLUX.1-dev-LoRA-Modern_Pixel_art, prithivMLmods/Flux.1-Dev-Stamp-Art-LoRA, glif-loradex-trainer/insectagon_Eightball_comics and strangerzonehf/Flux-Xmas-3D-LoRA.
- Update of Flux LoRA model kudzueye/Boreal to kudzueye/boreal-flux-dev-v2.
- Bugfixes for WebUI when using Flux models and crash following peft upgrade.
โข Features
โข Prerequisites
โข Installation
ย ย ย ย GNU/Linux
ย ย ย ย ย ย OpenSUSE Leap 15.5 / OpenSUSE Tumbleweed
ย ย ย ย ย ย Rocky 9.3 / Alma 9.3 / CentOS Stream 9 / Fedora 39
ย ย ย ย ย ย Debian 12 / Ubuntu 22.04.3 / Ubuntu 24.04 / Linux Mint 21.2
ย ย ย ย Windows 10 / Windows 11
ย ย ย ย macOS Intel Homebrew install
ย ย ย ย Dockerfile
โข CUDA support
โข How To Use
โข Good to know
โข Credits
โข License
-
Text generation using :
- โ๏ธ llama-cpp based chatbot module (uses .gguf models)
- ๐๏ธ Llava multimodal chatbot module (uses .gguf models)
- ๐๏ธ Microsoft GIT image captioning module
- ๐ Whisper speech-to-text module
- ๐ฅ nllb translation module (200 languages)
- ๐ Prompt generator (require 16GB+ RAM for ChatGPT output type)
-
Image generation and modification using :
- ๐ผ๏ธ Stable Diffusion module
- ๐ผ๏ธ Kandinsky module (require 16GB+ RAM)
- ๐ผ๏ธ Latent Consistency Models module
- ๐ผ๏ธ Midjourney-mini module
- ๐ผ๏ธPixArt-Alpha module
- ๐๏ธ Stable Diffusion Img2img module
- ๐๏ธ IP-Adapter module
- ๐ผ๏ธ Stable Diffusion Image variation module (require 16GB+ RAM)
- ๐๏ธ Instruct Pix2Pix module
- ๐๏ธ MagicMix module
- ๐๏ธ Stable Diffusion Inpaint module
- ๐๏ธ Fantasy Studio Paint by Example module (require 16GB+ RAM)
- ๐๏ธ Stable Diffusion Outpaint module (require 16GB+ RAM)
- ๐ผ๏ธ Stable Diffusion ControlNet module
- ๐ผ๏ธ Photobooth module
- ๐ญ Insight Face faceswapping module
- ๐ Real ESRGAN upscaler module
- ๐GFPGAN face restoration module
-
Audio generation using :
- ๐ถ MusicGen module
- ๐ถ MusicGen Melody module (require 16GB+ RAM)
- ๐ถ MusicLDM module
- ๐ Audiogen module (require 16GB+ RAM)
- ๐ Harmonai module
- ๐ฃ๏ธ Bark module
-
Video generation and modification using :
- ๐ผ Modelscope module (require 16GB+ RAM)
- ๐ผ Text2Video-Zero module
- ๐ผ AnimateDiff module (require 16GB+ RAM)
- ๐ผ Stable Video Diffusion module (require 16GB+ RAM)
- ๐๏ธ Video Instruct-Pix2Pix module (require 16GB+ RAM)
-
3D objects generation using :
- ๐ง Shap-E txt2shape module
- ๐ง Shap-E img2shape module (require 16GB+ RAM)
-
Other features
- Zeroconf installation through one-click installers or Windows exe.
- User friendly : Everything required to run biniou is installed automatically, either at install time or at first use.
- WebUI in English, French, Chinese (traditional).
- Easy management through a control panel directly inside webui : update, restart, shutdown, activate authentication, control network access or share your instance online with a single click.
- Easy management of models through a simple interface.
- Communication between modules : send an output as an input to another module
- Powered by ๐ค Huggingface and gradio
- Cross platform : GNU/Linux, Windows 10/11 and macOS(experimental, via homebrew)
- Convenient Dockerfile for cloud instances
- Generation settings saved as metadatas in each content.
- Support for CUDA (see CUDA support)
- Experimental support for ROCm (see here)
- Support for Stable Diffusion SD-1.5, SD-2.1, SD-Turbo, SDXL, SDXL-Turbo, SDXL-Lightning, Hyper-SD, Stable Diffusion 3, LCM, VegaRT, Segmind, Playground-v2, Koala, Pixart-Alpha, Pixart-Sigma, Kandinsky and compatible models, through built-in model list or standalone .safetensors files
- Support for LoRA models (SD 1.5, SDXL and SD3)
- Support for textual inversion
- Support llama-cpp-python optimizations CUDA, OpenBLAS, OpenCL BLAS, ROCm and Vulkan through a simple setting
- Support for Llama/2/3, Mistral, Mixtral and compatible GGUF quantized models, through built-in model list or standalone .gguf files.
- Easy copy/paste integration for TheBloke GGUF quantized models.
-
Minimal hardware :
- 64bit CPU (AMD64 architecture ONLY)
- 8GB RAM
- Storage requirements :
- for GNU/Linux : at least 20GB for installation without models.
- for Windows : at least 30GB for installation without models.
- for macOS : at least ??GB for installation without models.
- Storage type : HDD
- Internet access (required only for installation and models download) : unlimited bandwidth optical fiber internet access
-
Recommended hardware :
- Massively multicore 64bit CPU (AMD64 architecture ONLY) and a GPU compatible with CUDA or ROCm
- 16GB+ RAM
- Storage requirements :
- for GNU/Linux : around 200GB for installation including all defaults models.
- for Windows : around 200GB for installation including all defaults models.
- for macOS : around ??GB for installation including all defaults models.
- Storage type : SSD Nvme
- Internet access (required only for installation and models download) : unlimited bandwidth optical fiber internet access
-
Operating system :
- a 64 bit OS :
- Debian 12
- Ubuntu 22.04.3 / 24.04
- Linux Mint 21.2+ / 22
- Rocky 9.3
- Alma 9.3
- CentOS Stream 9
- Fedora 39
- OpenSUSE Leap 15.5
- OpenSUSE Tumbleweed
- Windows 10 22H2
- Windows 11 22H2
- macOS ???
- a 64 bit OS :
Note : biniou supports Cuda or ROCm but does not require a dedicated GPU to run. You can install it in a virtual machine.
- Copy/paste and execute the following command in a terminal :
sh <(curl https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-opensuse.sh || wget -O - https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-opensuse.sh)
- Copy/paste and execute the following command in a terminal :
sh <(curl https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-rhel.sh || wget -O - https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-rhel.sh)
- Copy/paste and execute the following command in a terminal :
sh <(curl https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-debian.sh || wget -O - https://raw.githubusercontent.com/Woolverine94/biniou/main/oci-debian.sh)
- Install the pre-requisites as root :
apt install git pip python3 python3-venv gcc perl make ffmpeg openssl
- Clone this repository as user :
git clone https://github.com/Woolverine94/biniou.git
- Launch the installer :
cd ./biniou
./install.sh
- (optional, but highly recommended) Install TCMalloc as root to optimize memory management :
apt install google-perftools
Windows installation has more prerequisites than GNU/Linux one, and requires following softwares (which will be installed automatically) :
- Git
- Python 3.11 (and specifically 3.11 version)
- OpenSSL
- Visual Studio Build tools
- Windows 10/11 SDK
- Vcredist
- ffmpeg
- ... and all their dependencies.
It's a lot of changes on your operating system, and this could potentially bring unwanted behaviors on your system, depending on which softwares are already installed on it.
-
Download and execute : biniou_netinstall.exe
OR
-
Download and execute : install_win.cmd (right-click on the link and select "Save Target/Link as ..." to download)
All the installation is automated, but Windows UAC will ask you for confirmation for each software installed during the "prerequisites" phase. You can avoid this by running the chosen installer as administrator.
install_win.cmd
Proceed as follow :
- Download and edit install_win.cmd
- Modify
set DEFAULT_BINIOU_DIR="%userprofile%"
toset DEFAULT_BINIOU_DIR="E:\datas\somedir"
(for example) - Only use absolute path (e.g.:
E:\datas\somedir
and not.\datas\somedir
) - Don't add a trailing slash (e.g.:
E:\datas\somedir
and notE:\datas\somedir\
) - Don't add a "biniou" suffix to your path (e.g.:
E:\datas\somedir\biniou
), as the biniou directory will be created by the git clone command - Save and launch install_win.cmd
-
Install Homebrew for your operating system
-
Install required homebrew "bottles" :
brew install git python3 gcc gcc@11 perl make ffmpeg openssl
- Install python virtualenv :
python3 -m pip install virtualenv
- Clone this repository as user :
git clone https://github.com/Woolverine94/biniou.git
- Launch the installer :
cd ./biniou
./install.sh
These instructions assumes that you already have a configured and working docker environment.
- Create the docker image :
docker build -t biniou https://github.com/Woolverine94/biniou.git
or, for CUDA support :
docker build -t biniou https://raw.githubusercontent.com/Woolverine94/biniou/main/CUDA/Dockerfile
- Launch the container :
docker run -it --restart=always -p 7860:7860 \
-v biniou_outputs:/home/biniou/biniou/outputs \
-v biniou_models:/home/biniou/biniou/models \
-v biniou_cache:/home/biniou/.cache/huggingface \
-v biniou_gfpgan:/home/biniou/biniou/gfpgan \
biniou:latest
or, for CUDA support :
docker run -it --gpus all --restart=always -p 7860:7860 \
-v biniou_outputs:/home/biniou/biniou/outputs \
-v biniou_models:/home/biniou/biniou/models \
-v biniou_cache:/home/biniou/.cache/huggingface \
-v biniou_gfpgan:/home/biniou/biniou/gfpgan \
biniou:latest
-
Access the webui by the url :
https://127.0.0.1:7860 or https://127.0.0.1:7860/?__theme=dark for dark theme (recommended)
... or replace 127.0.0.1 by ip of your container
Note : to save storage space, the previous container launch command defines common shared volumes for all biniou containers and ensure that the container auto-restart in case of OOM crash. Remove
--restart
and-v
arguments if you didn't want these behaviors.
biniou is natively cpu-only, to ensure compatibility with a wide range of hardware, but you can easily activate CUDA support through Nvidia CUDA (if you have a functional CUDA 12.1 environment) or AMD ROCm (if you have a functional ROCm 5.6 environment) by selecting the type of optimization to activate (CPU, CUDA or ROCm for Linux), in the WebUI control module.
Currently, all modules except Chatbot, Llava and faceswap modules, could benefits from CUDA optimization.
- Launch by executing from the biniou directory :
- for GNU/Linux :
cd /home/$USER/biniou
./webui.sh
- for Windows :
Double-click webui.cmd in the biniou directory (C:\Users\%username%\biniou\). When asked by the UAC, configure the firewall according to your network type to authorize access to the webui
Note : First start could be very slow on Windows 11 (comparing to others OS).
-
Access the webui by the url :
https://127.0.0.1:7860 or https://127.0.0.1:7860/?__theme=dark for dark theme (recommended)
You can also access biniou from any device (including smartphones) on the same LAN/Wifi network by replacing 127.0.0.1 in the url with biniou host ip address. -
Quit by using the keyboard shortcut CTRL+C in the Terminal
-
Update this application (biniou + python virtual environment) by using the WebUI control updates options.
-
Most frequent cause of crash is not enough memory on the host. Symptom is biniou program closing and returning to/closing the terminal without specific error message. You can use biniou with 8GB RAM, but 16GB at least is recommended to avoid OOM (out of memory) error.
-
biniou use a lot of differents AI models, which requires a lot of space : if you want to use all the modules in biniou, you will need around 200GB of disk space only for the default model of each module. Models are downloaded on the first run of each module or when you select a new model in a module and generate content. Models are stored in the directory /models of the biniou installation. Unused models could be deleted to save some space.
-
... consequently, you will need a fast internet access to download models.
-
A backup of every content generated is available inside the /outputs directory of the biniou folder.
-
biniou natively only rely on CPU for all operations. It use a specific CPU-only version of PyTorch. The result is a better compatibility with a wide range of hardware, but degraded performances. Depending on your hardware, expect slowness. See here for Nvidia CUDA support and AMD ROCm experimental support (GNU/Linux only).
-
Defaults settings are selected to permit generation of contents on low-end computers, with the best ratio performance/quality. If you have a configuration above the minimal settings, you could try using other models, increasing media dimensions or duration, modifying inference parameters or other settings (like token merging for images) to obtain better quality contents.
-
biniou is licensed under GNU GPL3, but each model used in biniou has its own license. Please consult each model license to know what you can and cannot do with the models. For each model, you can find a link to the huggingface page of the model in the "About" section of the associated module.
-
Don't have too much expectations : biniou is in an early stage of development, and most open source software used in it are in development (some are still experimental).
-
Every biniou modules offers 2 accordions elements About and Settings :
- About is a quick help feature that describes the module and gives instructions and tips on how to use it.
- Settings is a panel setting specific to the module that lets you configure the generation parameters.
This application uses the following softwares and technologies :
- ๐ค Huggingface : Diffusers and Transformers libraries and almost all the generative models.
- Gradio : webUI
- llama-cpp-python : python bindings for llama-cpp
- Llava
- BakLLava
- Microsoft GIT : Image2text
- Whisper : speech2text
- nllb translation : language translation
- Stable Diffusion : txt2img, img2img, Image variation, inpaint, ControlNet, Text2Video-Zero, img2vid
- Kandinsky : txt2img
- Latent consistency models : txt2img
- PixArt-Alpha : PixArt-Alpha
- IP-Adapter : IP-Adapter img2img
- Instruct pix2pix : pix2pix
- MagicMix : MagicMix
- Fantasy Studio Paint by Example : paintbyex
- Controlnet Auxiliary models : preview models for ControlNet module
- IP-Adapter FaceID : Adapter model for Photobooth module
- Photomaker Adapter model for Photobooth module
- Insight Face : faceswapping
- Real ESRGAN : upscaler
- GFPGAN : face restoration
- Audiocraft : musicgen, musicgen melody, audiogen
- MusicLDM : MusicLDM
- Harmonai : harmonai
- Bark : text2speech
- Modelscope text-to-video-synthesis : txt2vid
- AnimateLCM : txt2vid
- Open AI Shap-E : txt2shape, img2shape
-
compel : Prompt enhancement for various
StableDiffusionPipeline
-based modules -
tomesd : Token merging for various
StableDiffusionPipeline
-based modules - Python
- PyTorch
- Git
- ffmpeg
... and all their dependencies
GNU General Public License v3.0
GitHub @Woolverine94 ย ยทย
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for biniou
Similar Open Source Tools
biniou
biniou is a self-hosted webui for various GenAI (generative artificial intelligence) tasks. It allows users to generate multimedia content using AI models and chatbots on their own computer, even without a dedicated GPU. The tool can work offline once deployed and required models are downloaded. It offers a wide range of features for text, image, audio, video, and 3D object generation and modification. Users can easily manage the tool through a control panel within the webui, with support for various operating systems and CUDA optimization. biniou is powered by Huggingface and Gradio, providing a cross-platform solution for AI content generation.
amazon-sagemaker-generativeai
Repository for training and deploying Generative AI models, including text-text, text-to-image generation, prompt engineering playground and chain of thought examples using SageMaker Studio. The tool provides a platform for users to experiment with generative AI techniques, enabling them to create text and image outputs based on input data. It offers a range of functionalities for training and deploying models, as well as exploring different generative AI applications.
lite.koboldai.net
KoboldAI Lite is a standalone Web UI that serves as a text editor designed for use with generative LLMs. It is compatible with KoboldAI United and KoboldAI Client, bundled with KoboldCPP, and integrates with the AI Horde for text and image generation. The UI offers multiple modes for different writing styles, supports various file formats, includes premade scenarios, and allows easy sharing of stories. Users can enjoy features such as memory, undo/redo, text-to-speech, and a range of samplers and configurations. The tool is mobile-friendly and can be used directly from a browser without any setup or installation.
open-ai
Open AI is a powerful tool for artificial intelligence research and development. It provides a wide range of machine learning models and algorithms, making it easier for developers to create innovative AI applications. With Open AI, users can explore cutting-edge technologies such as natural language processing, computer vision, and reinforcement learning. The platform offers a user-friendly interface and comprehensive documentation to support users in building and deploying AI solutions. Whether you are a beginner or an experienced AI practitioner, Open AI offers the tools and resources you need to accelerate your AI projects and stay ahead in the rapidly evolving field of artificial intelligence.
Deep-Live-Cam
Deep-Live-Cam is a software tool designed to assist artists in tasks such as animating custom characters or using characters as models for clothing. The tool includes built-in checks to prevent unethical applications, such as working on inappropriate media. Users are expected to use the tool responsibly and adhere to local laws, especially when using real faces for deepfake content. The tool supports both CPU and GPU acceleration for faster processing and provides a user-friendly GUI for swapping faces in images or videos.
SystemAnimatorOnline
XR Animator is a video/webcam-based AI motion capture application designed for VTubing and the metaverse era. It uses machine learning solutions to detect 3D poses from a live webcam video, driving a 3D avatar as if controlled by the user's body. It supports full-body AI motion tracking, face tracking, and various XR/3D purposes. The tool can be used for VTubing, recording mocap motion, exporting motions to different formats, customizing backgrounds and scenes, and animating 3D models in other applications. It also supports AR on Android Chrome browser, AR selfie feature, and has relatively low system requirements for wide device compatibility.
ten_framework
TEN Framework, short for Transformative Extensions Network, is the world's first real-time multimodal AI agent framework. It offers native support for high-performance, real-time multimodal interactions, supports multiple languages and platforms, enables edge-cloud integration, provides flexibility beyond model limitations, and allows for real-time agent state management. The framework facilitates the development of complex AI applications that transcend the limitations of large models by offering a drag-and-drop programming approach. It is suitable for scenarios like simultaneous interpretation, speech-to-text conversion, multilingual chat rooms, audio interaction, and audio-visual interaction.
exo
Run your own AI cluster at home with everyday devices. Exo is experimental software that unifies existing devices into a powerful GPU, supporting wide model compatibility, dynamic model partitioning, automatic device discovery, ChatGPT-compatible API, and device equality. It does not use a master-worker architecture, allowing devices to connect peer-to-peer. Exo supports different partitioning strategies like ring memory weighted partitioning. Installation is recommended from source. Documentation includes example usage on multiple MacOS devices and information on inference engines and networking modules. Known issues include the iOS implementation lagging behind Python.
AI_Spectrum
AI_Spectrum is a versatile machine learning library that provides a wide range of tools and algorithms for building and deploying AI models. It offers a user-friendly interface for data preprocessing, model training, and evaluation. With AI_Spectrum, users can easily experiment with different machine learning techniques and optimize their models for various tasks. The library is designed to be flexible and scalable, making it suitable for both beginners and experienced data scientists.
Main
This repository contains material related to the new book _Synthetic Data and Generative AI_ by the author, including code for NoGAN, DeepResampling, and NoGAN_Hellinger. NoGAN is a tabular data synthesizer that outperforms GenAI methods in terms of speed and results, utilizing state-of-the-art quality metrics. DeepResampling is a fast NoGAN based on resampling and Bayesian Models with hyperparameter auto-tuning. NoGAN_Hellinger combines NoGAN and DeepResampling with the Hellinger model evaluation metric.
hoarder
A self-hostable bookmark-everything app with a touch of AI for data hoarders. Features include bookmarking links, taking notes, storing images, automatic fetching for link details, full-text search, AI-based automatic tagging, Chrome and Firefox plugins, iOS and Android apps, dark mode support, and self-hosting. Built to address the need for archiving and previewing links with automatic tagging. Developed by a systems engineer to stay connected with web development and cater to personal use cases.
big-AGI
big-AGI is an AI suite designed for professionals seeking function, form, simplicity, and speed. It offers best-in-class Chats, Beams, and Calls with AI personas, visualizations, coding, drawing, side-by-side chatting, and more, all wrapped in a polished UX. The tool is powered by the latest models from 12 vendors and open-source servers, providing users with advanced AI capabilities and a seamless user experience. With continuous updates and enhancements, big-AGI aims to stay ahead of the curve in the AI landscape, catering to the needs of both developers and AI enthusiasts.
airbroke
Airbroke is an open-source error catcher tool designed for modern web applications. It provides a PostgreSQL-based backend with an Airbrake-compatible HTTP collector endpoint and a React-based frontend for error management. The tool focuses on simplicity, maintaining a small database footprint even under heavy data ingestion. Users can ask AI about issues, replay HTTP exceptions, and save/manage bookmarks for important occurrences. Airbroke supports multiple OAuth providers for secure user authentication and offers occurrence charts for better insights into error occurrences. The tool can be deployed in various ways, including building from source, using Docker images, deploying on Vercel, Render.com, Kubernetes with Helm, or Docker Compose. It requires Node.js, PostgreSQL, and specific system resources for deployment.
databerry
Chaindesk is a no-code platform that allows users to easily set up a semantic search system for personal data without technical knowledge. It supports loading data from various sources such as raw text, web pages, files (Word, Excel, PowerPoint, PDF, Markdown, Plain Text), and upcoming support for web sites, Notion, and Airtable. The platform offers a user-friendly interface for managing datastores, querying data via a secure API endpoint, and auto-generating ChatGPT Plugins for each datastore. Chaindesk utilizes a Vector Database (Qdrant), Openai's text-embedding-ada-002 for embeddings, and has a chunk size of 1024 tokens. The technology stack includes Next.js, Joy UI, LangchainJS, PostgreSQL, Prisma, and Qdrant, inspired by the ChatGPT Retrieval Plugin.
Pichome
PicHome is a powerful open-source cloud storage program that efficiently manages various types of files and excels in image and media file management. Its highlights include robust file sharing features and advanced AI-assisted management tools, providing users with a convenient and intelligent file management experience. The program offers diverse list modes, customizable file information display, enhanced quick file preview, advanced tagging, custom cover and preview images, multiple preview images, and multi-library management. Additionally, PicHome features strong file sharing capabilities, allowing users to share entire libraries, create personalized showcase web pages, and build complete data sharing websites. The AI-assisted management aspect includes AI file renaming, tagging, description writing, batch annotation, and file Q&A services, all aimed at improving file management efficiency. PicHome supports a wide range of file formats and can be applied in various scenarios such as e-commerce, gaming, design, development, enterprises, schools, labs, media, and entertainment institutions.
sciml.ai
SciML.ai is an open source software organization dedicated to unifying packages for scientific machine learning. It focuses on developing modular scientific simulation support software, including differential equation solvers, inverse problems methodologies, and automated model discovery. The organization aims to provide a diverse set of tools with a common interface, creating a modular, easily-extendable, and highly performant ecosystem for scientific simulations. The website serves as a platform to showcase SciML organization's packages and share news within the ecosystem. Pull requests are encouraged for contributions.
For similar tasks
Awesome-AITools
This repo collects AI-related utilities. ## All Categories * All Categories * ChatGPT and other closed-source LLMs * AI Search engine * Open Source LLMs * GPT/LLMs Applications * LLM training platform * Applications that integrate multiple LLMs * AI Agent * Writing * Programming Development * Translation * AI Conversation or AI Voice Conversation * Image Creation * Speech Recognition * Text To Speech * Voice Processing * AI generated music or sound effects * Speech translation * Video Creation * Video Content Summary * OCR(Optical Character Recognition)
NSMusicS
NSMusicS is a local music software that is expected to support multiple platforms with AI capabilities and multimodal features. The goal of NSMusicS is to integrate various functions (such as artificial intelligence, streaming, music library management, cross platform, etc.), which can be understood as similar to Navidrome but with more features than Navidrome. It wants to become a plugin integrated application that can almost have all music functions.
biniou
biniou is a self-hosted webui for various GenAI (generative artificial intelligence) tasks. It allows users to generate multimedia content using AI models and chatbots on their own computer, even without a dedicated GPU. The tool can work offline once deployed and required models are downloaded. It offers a wide range of features for text, image, audio, video, and 3D object generation and modification. Users can easily manage the tool through a control panel within the webui, with support for various operating systems and CUDA optimization. biniou is powered by Huggingface and Gradio, providing a cross-platform solution for AI content generation.
generative-ai-js
Generative AI JS is a JavaScript library that provides tools for creating generative art and music using artificial intelligence techniques. It allows users to generate unique and creative content by leveraging machine learning models. The library includes functions for generating images, music, and text based on user input and preferences. With Generative AI JS, users can explore the intersection of art and technology, experiment with different creative processes, and create dynamic and interactive content for various applications.
pictureChange
The 'pictureChange' repository is a plugin that supports image processing using Baidu AI, stable diffusion webui, and suno music composition AI. It also allows for file summarization and image summarization using AI. The plugin supports various stable diffusion models, administrator control over group chat features, concurrent control, and custom templates for image and text generation. It can be deployed on WeChat enterprise accounts, personal accounts, and public accounts.
Generative-AI-Indepth-Basic-to-Advance
Generative AI Indepth Basic to Advance is a repository focused on providing tutorials and resources related to generative artificial intelligence. The repository covers a wide range of topics from basic concepts to advanced techniques in the field of generative AI. Users can find detailed explanations, code examples, and practical demonstrations to help them understand and implement generative AI algorithms. The goal of this repository is to help beginners get started with generative AI and to provide valuable insights for more experienced practitioners.
nodetool
NodeTool is a platform designed for AI enthusiasts, developers, and creators, providing a visual interface to access a variety of AI tools and models. It simplifies access to advanced AI technologies, offering resources for content creation, data analysis, automation, and more. With features like a visual editor, seamless integration with leading AI platforms, model manager, and API integration, NodeTool caters to both newcomers and experienced users in the AI field.
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
For similar jobs
weave
Weave is a toolkit for developing Generative AI applications, built by Weights & Biases. With Weave, you can log and debug language model inputs, outputs, and traces; build rigorous, apples-to-apples evaluations for language model use cases; and organize all the information generated across the LLM workflow, from experimentation to evaluations to production. Weave aims to bring rigor, best-practices, and composability to the inherently experimental process of developing Generative AI software, without introducing cognitive overhead.
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
VisionCraft
The VisionCraft API is a free API for using over 100 different AI models. From images to sound.
kaito
Kaito is an operator that automates the AI/ML inference model deployment in a Kubernetes cluster. It manages large model files using container images, avoids tuning deployment parameters to fit GPU hardware by providing preset configurations, auto-provisions GPU nodes based on model requirements, and hosts large model images in the public Microsoft Container Registry (MCR) if the license allows. Using Kaito, the workflow of onboarding large AI inference models in Kubernetes is largely simplified.
PyRIT
PyRIT is an open access automation framework designed to empower security professionals and ML engineers to red team foundation models and their applications. It automates AI Red Teaming tasks to allow operators to focus on more complicated and time-consuming tasks and can also identify security harms such as misuse (e.g., malware generation, jailbreaking), and privacy harms (e.g., identity theft). The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model. This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.
tabby
Tabby is a self-hosted AI coding assistant, offering an open-source and on-premises alternative to GitHub Copilot. It boasts several key features: * Self-contained, with no need for a DBMS or cloud service. * OpenAPI interface, easy to integrate with existing infrastructure (e.g Cloud IDE). * Supports consumer-grade GPUs.
spear
SPEAR (Simulator for Photorealistic Embodied AI Research) is a powerful tool for training embodied agents. It features 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each environment is designed by a professional artist and features detailed geometry, photorealistic materials, and a unique floor plan and object layout. SPEAR is implemented as Unreal Engine assets and provides an OpenAI Gym interface for interacting with the environments via Python.
Magick
Magick is a groundbreaking visual AIDE (Artificial Intelligence Development Environment) for no-code data pipelines and multimodal agents. Magick can connect to other services and comes with nodes and templates well-suited for intelligent agents, chatbots, complex reasoning systems and realistic characters.