STMP
SillyTavern MultiPlayer is an LLM chat interface, created by RossAscends, that allows multiple users to chat together with each other and an AI.
Stars: 69
SillyTavern MultiPlayer (STMP) is an LLM chat interface that enables multiple users to chat with an AI. It features a sidebar chat for users, tools for the Host to manage the AI's behavior and moderate users. Users can change display names, chat in different windows, and the Host can control AI settings. STMP supports Text Completions, Chat Completions, and HordeAI. Users can add/edit APIs, manage past chats, view user lists, and control delays. Hosts have access to various controls, including AI configuration, adding presets, and managing characters. Planned features include smarter retry logic, host controls enhancements, and quality of life improvements like user list fading and highlighting exact usernames in AI responses.
README:
SillyTavern MultiPlayer is an LLM chat interface that allows multiple users to chat together with an AI. It also includes a sidebar chat for users only, and many tools for the Host to control the behavior of the AI and to moderate users.
Created by RossAscends
If this software brings you and your friend's joy, donations to Ross can be made via:
Ko-fi |
Patreon |
For tech support or to contact RossAscends directly, join the SillyTavern Discord.
- Make sure Node JS is installed on your system and has access through your firewall.
- Clone this github repo into a new folder NOT the main SillyTavern folder. (
git clone https://github.com/RossAscends/STMP/
in a command line, or use a Github GUI like Github Desktop) - Setup the API (see below).
- Run
STMP.bat
to install the required Node modules and start the server. - On the first run, the server will create an empty
secrets.json
and a defaultconfig.json
, as well as the/api-presets/
and/chats/
folders. - Open
http://localhost:8181/
in your web browser.
(instructions coming soon)
You can use Horde as an anonymous user, but that generally leads to slower queue times.
To use your Horde API key in STMP, add it to secrets.json
like this (server should run at least once):
{
// some content
"horde_key": "YourKeyHere",
// some other content
}
Don't have one? Register here (it's free).
This must be done AFTER completing all installation steps above.
- Make sure your STMP server is running.
- Run
Remote-Link.cmd
to downloadcloudflared.exe
(only one time, 57MB). - the Cloudflared server will auto-start and generate a random tunnel URL for your STMP server.
- Copy the URL displayed in the middle of the large box in the center of the console window.
DO NOT CLOSE THE CLOUDFLARE CONSOLE WINDOW
- Share the generated cloudflared URL with the guest users.
- User will be able to directly and securely connect to your PC by opening the URL in their browser.
- User can change their display name at any time using either of the inputs at the top of the respective chat displays.
- You can have a different name for the User Chat and AI Chat.
- Usernames are stored in browser localStorage.
- Chatting can be done in either chat windows by typing into the appropriate box and then either pressing the Send button (βοΈ), or pressing Enter.
-
Shift+Enter
can be used to add newlines to the input. - Markdown formatting is respected.
- Users with the Host role can hover over any chat message in either chats to see editing and deletion buttons.
The host will see the following controls:
AI Config Section
-
Mode
can be clicked to switch between TC/CC mode, and HordeAI mode.- π = Completions
- π§ = Horde
-
Context
defines how long your API prompt should be. Longer = more chat history sent, but slower processing times. -
Response
defines how many tokens long the AI response can be. -
Streaming
togles streamed responses on or off. -
AutoAI
Toggle to determine whether the AI should respond to every user input, or only on command from the Host. -
Instruct
sets the type of instruct sequences to use when crafting the API prompt. -
Sampler
sets the hyperparameter preset, which affects the response style. -
API
selector to choose which LLM API to use, and anEdit
button to change its configuration. -
Models
allows selection of models served by the connected API.
Insertions Section
-
System Prompt
defines what will be placed at the very top of the prompt. -
Author Note(D4)
defines what will be inserted as a system message at Depth 4 in the prompt. -
Final Instruction(D1, "JB")
defines what to send as a system message at Depth 1 in the prompt. Not needed for TC APIs, but useful as a 'JB prompt' for CC APIs.
Currently STMP supports Text Completions (TC), Chat Completions (CC), and HordeAI via SillyTavern.
External Text Completion and Chat Completion connections are currently a Work in Progress! We have tested connecting to OpenAI compatible APIs, as well as Anthropic's Claude. Most other LLM backends that provide an API in the same format should be compatible with STMP.
- TabbyAPI and KoboldCPP are confirmed to work with STMP's TC API mode.
- Oobabooga Textgeneration Webui works with their OpenAI-compatible CC API mode.
- OpenRouter is supported in CC mode.
- We suspect Aphrodite should be compatible as well, but have not tested it yet.
- select
Add new API
from theAPI
selector to open the API Editing panel. - A new panel will be displayed with new inputs:
-
Name
- the label you want to remember the API as -
Endpoint URL
- this is the base server URL for the LLM API. If the usual URL does not work, try addingv1/
to the end. -
Key
- If your API requires a key, put it in here. -
Endpoint Type
- select from Text Completions or Chat Completions as appropriate for the API endpoint. -
Claude
- select this if the API is based on Anthropic's Claude model, because it needs special prompt formatting. -
Close
button will cancel the API editing/creating process and return you to the main AI Config panel. -
Test
button sends a simple test message to the API to get a response. This may not work on all APIs. -
Save
confirms the API addition/edit and saves it to the database. -
Delete
removes the APi from the database.
- When all of the fields are filled out, press
Save
to return to the main Control panel display.
- A list of past AI Chats, click one to load it.
- Information listed with each past session item:
- AI characters in the chat history
- number of messages in AI Chat
- timestamp for the last message
- the list is ordered in reverse chronological (newest chats first)
- The right side of the screen contains two users lists, one for each of the chats.
- Users with the Host role will have a π next to their name.
- The AI Characters will have a π€ next to their names in the AI Chat User List.
- (π€β³) sets the number of seconds delay between inputs to the AI Chat.
- (π§β³) does the same, but for the User-to-User chat.
- During the delay period the (βοΈ) for that chat will become (π«), and no input will be possible.
- (ποΈ) toggles visibility of the Host Control Panel.
- (πΌοΈ) toggles the chat windows between three modes: maximize AI chat >> maximize User Chat >> return to normal dual display.
- this is very helpful for mobile users!
- (π) toggles display of both User lists.
- (
βΆοΈ /βΈοΈ) allows for manual disconnect/reconnect to the server. - (π οΈ) opens a Profile management menu which contains:
- Two text fields to change your displayed username in either the User Chat or AI Chat.
- (π) opens a text box for input of the Host key in order to gain the Host role.
- Once a vlid key has been entered, the page will automatically refresh to show the host controls.
- The Host key can be found in the server console at startup.
- After the user enters the key and presses Enter, their page will refresh and they will see the Host controls.
- (β) clears the saved Usernames and UniqueID from localStorage.
- If you are not the primary Host you will lose any roles you were given.
- You will be asked to register a new username next time you sign in on the same browser.
- (π) (only on mobile) will cause a 10 minute silent audio to play when you minimize the browser window, preventing the websocket from being disconnected while the window is minimized. This is played each time you minimize the app.
- AI Chat: a selector to set the active AI character
- Both chats: (ποΈ) to clear either chat.
- clearing the AI Chat will automatically create a new chat with the selected Character.
- (π€) Manually triggering an AI response without user Input
- (βοΈ) Deleting the last message in the AI Chat
- (π) Retry, i.e. Remove the last chat message and prompt the AI character to give a new response.
- Place any SillyTavern compatible character card into the
/characters/
folder and restart the server.- We will add a way to add characters without restarting the server soon.
- Characters can be selected at the top of the AI Chat panel.
- Characters can be swapped at any time without resetting the chat, allowing you to manually simulate a group chat.
- Hosts will see a (π§ ) next to the Character Selector. This will open a popup with the character definitions.
- STMP only handles three types of character definitions:
-
Name
- What is displayed as the character's name in the chat. -
Description
- What goes into the prompt at either the top, or at D4 if 'D4 Char Defs' box is checked. -
First Message
- What gets auto-inserted into a new chat with that character.
-
-
Save
will update the character definitions. -
Close
will close the popup with no changes.
What about Personality, Scenario, and Example Messages?
- Personality and Scenario are outdated distinctions without a meaningful purpose, and should be incorporated into the Description.
- Example Message can also be incorporated into the Description. We recommend doing so in the form of AliChat.
What about v2 Card Spec values?
STMP will read from the v2 spec values if they are present, but will only write to the v1 card spec space.
What about embedded lorebooks?
Incorporating embedded lorebooks is under consideration.
- If you want to add more presets for Instruct formats or hyperparameter Samplers, put the JSON file into the appropriate folder:
- Samplers go in
/public/api-presets/
- Instruct formats go in
/public/instructFormats/
- It's highly reccomended to review the structure of the default STMP preset files.
- SillyTavern preset files may not work, or may have unintended effects!
- Smarter retry logic (add entity metadata to each chat message; only remove the last AI response)
- Toggle for locking AI chat for users? (this is already kind of done with AutoResponse off)
- Drag-sort list to set User Turn Order for AI chatting?
- Ability to rename chats.
- ability for Host to edit a User's role from the UI
- ability to change the max length of chat inputs (currently 1000 characters)
- make control for AI replying every X messages
- make control for host, to autoclear chat every X messages
- disallow names that are only spaces, punctuations, or non ASCII (non-Latin?) characters
- require at least 3? A-Za-z characters
- disallow registering of names that are already in the DB
- make control for guests to clear DISPLAY of either chat (without affecting the chat database) to prevent browser lag
- highlight exact username matches in AI response with their color
- fade out users in user chat list who havent chatted in X minutes (add a css class with opacity 50%)
- fade out users in ai chat list who are no longer connected, or are faded out in user list (same)
- show which users in the User Chat are using which name in the AI Chat
- add a link to the User message that the AI is responding to at the top of each AI message.
- When an AI resposne is triggered by UserX's input, UserX sees that response highlighted in the chat
- Multiple AI characters active at once (group chats)
- Download chats as text, JSON, or SillyTavern-compatible JSONL?
- UI themes?
- Bridge extension for SillyTavern to enable intra-server communication.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for STMP
Similar Open Source Tools
STMP
SillyTavern MultiPlayer (STMP) is an LLM chat interface that enables multiple users to chat with an AI. It features a sidebar chat for users, tools for the Host to manage the AI's behavior and moderate users. Users can change display names, chat in different windows, and the Host can control AI settings. STMP supports Text Completions, Chat Completions, and HordeAI. Users can add/edit APIs, manage past chats, view user lists, and control delays. Hosts have access to various controls, including AI configuration, adding presets, and managing characters. Planned features include smarter retry logic, host controls enhancements, and quality of life improvements like user list fading and highlighting exact usernames in AI responses.
azure-search-openai-demo
This sample demonstrates a few approaches for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access a GPT model (gpt-35-turbo), and Azure AI Search for data indexing and retrieval. The repo includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Electronics, and the experience allows its employees to ask questions about the benefits, internal policies, as well as job descriptions and roles.
vector-vein
VectorVein is a no-code AI workflow software inspired by LangChain and langflow, aiming to combine the powerful capabilities of large language models and enable users to achieve intelligent and automated daily workflows through simple drag-and-drop actions. Users can create powerful workflows without the need for programming, automating all tasks with ease. The software allows users to define inputs, outputs, and processing methods to create customized workflow processes for various tasks such as translation, mind mapping, summarizing web articles, and automatic categorization of customer reviews.
serverless-pdf-chat
The serverless-pdf-chat repository contains a sample application that allows users to ask natural language questions of any PDF document they upload. It leverages serverless services like Amazon Bedrock, AWS Lambda, and Amazon DynamoDB to provide text generation and analysis capabilities. The application architecture involves uploading a PDF document to an S3 bucket, extracting metadata, converting text to vectors, and using a LangChain to search for information related to user prompts. The application is not intended for production use and serves as a demonstration and educational tool.
gpt-pilot
GPT Pilot is a core technology for the Pythagora VS Code extension, aiming to provide the first real AI developer companion. It goes beyond autocomplete, helping with writing full features, debugging, issue discussions, and reviews. The tool utilizes LLMs to generate production-ready apps, with developers overseeing the implementation. GPT Pilot works step by step like a developer, debugging issues as they arise. It can work at any scale, filtering out code to show only relevant parts to the AI during tasks. Contributions are welcome, with debugging and telemetry being key areas of focus for improvement.
azure-search-openai-javascript
This sample demonstrates a few approaches for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model (gpt-35-turbo), and Azure AI Search for data indexing and retrieval.
Open_Data_QnA
Open Data QnA is a Python library that allows users to interact with their PostgreSQL or BigQuery databases in a conversational manner, without needing to write SQL queries. The library leverages Large Language Models (LLMs) to bridge the gap between human language and database queries, enabling users to ask questions in natural language and receive informative responses. It offers features such as conversational querying with multiturn support, table grouping, multi schema/dataset support, SQL generation, query refinement, natural language responses, visualizations, and extensibility. The library is built on a modular design and supports various components like Database Connectors, Vector Stores, and Agents for SQL generation, validation, debugging, descriptions, embeddings, responses, and visualizations.
ComfyUI-Tara-LLM-Integration
Tara is a powerful node for ComfyUI that integrates Large Language Models (LLMs) to enhance and automate workflow processes. With Tara, you can create complex, intelligent workflows that refine and generate content, manage API keys, and seamlessly integrate various LLMs into your projects. It comprises nodes for handling OpenAI-compatible APIs, saving and loading API keys, composing multiple texts, and using predefined templates for OpenAI and Groq. Tara supports OpenAI and Grok models with plans to expand support to together.ai and Replicate. Users can install Tara via Git URL or ComfyUI Manager and utilize it for tasks like input guidance, saving and loading API keys, and generating text suitable for chaining in workflows.
DAILA
DAILA is a unified interface for AI systems in decompilers, supporting various decompilers and AI systems. It allows users to utilize local and remote LLMs, like ChatGPT and Claude, and local models such as VarBERT. DAILA can be used as a decompiler plugin with GUI or as a scripting library. It also provides a Docker container for offline installations and supports tasks like summarizing functions and renaming variables in decompilation.
airbroke
Airbroke is an open-source error catcher tool designed for modern web applications. It provides a PostgreSQL-based backend with an Airbrake-compatible HTTP collector endpoint and a React-based frontend for error management. The tool focuses on simplicity, maintaining a small database footprint even under heavy data ingestion. Users can ask AI about issues, replay HTTP exceptions, and save/manage bookmarks for important occurrences. Airbroke supports multiple OAuth providers for secure user authentication and offers occurrence charts for better insights into error occurrences. The tool can be deployed in various ways, including building from source, using Docker images, deploying on Vercel, Render.com, Kubernetes with Helm, or Docker Compose. It requires Node.js, PostgreSQL, and specific system resources for deployment.
gpt-subtrans
GPT-Subtrans is an open-source subtitle translator that utilizes large language models (LLMs) as translation services. It supports translation between any language pairs that the language model supports. Note that GPT-Subtrans requires an active internet connection, as subtitles are sent to the provider's servers for translation, and their privacy policy applies.
civitai
Civitai is a platform where people can share their stable diffusion models (textual inversions, hypernetworks, aesthetic gradients, VAEs, and any other crazy stuff people do to customize their AI generations), collaborate with others to improve them, and learn from each other's work. The platform allows users to create an account, upload their models, and browse models that have been shared by others. Users can also leave comments and feedback on each other's models to facilitate collaboration and knowledge sharing.
ai-clone-whatsapp
This repository provides a tool to create an AI chatbot clone of yourself using your WhatsApp chats as training data. It utilizes the Torchtune library for finetuning and inference. The code includes preprocessing of WhatsApp chats, finetuning models, and chatting with the AI clone via a command-line interface. Supported models are Llama3-8B-Instruct and Mistral-7B-Instruct-v0.2. Hardware requirements include approximately 16 GB vRAM for QLoRa Llama3 finetuning with a 4k context length. The repository addresses common issues like adjusting parameters for training and preprocessing non-English chats.
generative-ai-application-builder-on-aws
The Generative AI Application Builder on AWS (GAAB) is a solution that provides a web-based management dashboard for deploying customizable Generative AI (Gen AI) use cases. Users can experiment with and compare different combinations of Large Language Model (LLM) use cases, configure and optimize their use cases, and integrate them into their applications for production. The solution is targeted at novice to experienced users who want to experiment and productionize different Gen AI use cases. It uses LangChain open-source software to configure connections to Large Language Models (LLMs) for various use cases, with the ability to deploy chat use cases that allow querying over users' enterprise data in a chatbot-style User Interface (UI) and support custom end-user implementations through an API.
CLI
Bito CLI provides a command line interface to the Bito AI chat functionality, allowing users to interact with the AI through commands. It supports complex automation and workflows, with features like long prompts and slash commands. Users can install Bito CLI on Mac, Linux, and Windows systems using various methods. The tool also offers configuration options for AI model type, access key management, and output language customization. Bito CLI is designed to enhance user experience in querying AI models and automating tasks through the command line interface.
HackBot
HackBot is an AI-powered cybersecurity chatbot designed to provide accurate answers to cybersecurity-related queries, conduct code analysis, and scan analysis. It utilizes the Meta-LLama2 AI model through the 'LlamaCpp' library to respond coherently. The chatbot offers features like local AI/Runpod deployment support, cybersecurity chat assistance, interactive interface, clear output presentation, static code analysis, and vulnerability analysis. Users can interact with HackBot through a command-line interface and utilize it for various cybersecurity tasks.
For similar tasks
STMP
SillyTavern MultiPlayer (STMP) is an LLM chat interface that enables multiple users to chat with an AI. It features a sidebar chat for users, tools for the Host to manage the AI's behavior and moderate users. Users can change display names, chat in different windows, and the Host can control AI settings. STMP supports Text Completions, Chat Completions, and HordeAI. Users can add/edit APIs, manage past chats, view user lists, and control delays. Hosts have access to various controls, including AI configuration, adding presets, and managing characters. Planned features include smarter retry logic, host controls enhancements, and quality of life improvements like user list fading and highlighting exact usernames in AI responses.
For similar jobs
STMP
SillyTavern MultiPlayer (STMP) is an LLM chat interface that enables multiple users to chat with an AI. It features a sidebar chat for users, tools for the Host to manage the AI's behavior and moderate users. Users can change display names, chat in different windows, and the Host can control AI settings. STMP supports Text Completions, Chat Completions, and HordeAI. Users can add/edit APIs, manage past chats, view user lists, and control delays. Hosts have access to various controls, including AI configuration, adding presets, and managing characters. Planned features include smarter retry logic, host controls enhancements, and quality of life improvements like user list fading and highlighting exact usernames in AI responses.