swarms
The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework Join our Community: https://discord.com/servers/agora-999382051935506503
Stars: 1245
Swarms provides simple, reliable, and agile tools to create your own Swarm tailored to your specific needs. Currently, Swarms is being used in production by RBC, John Deere, and many AI startups.
README:
The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework
🐦 Twitter • 📢 Discord • Swarms Platform • 📙 Documentation
Swarms is an enterprise grade and production ready multi-agent collaboration framework that enables you to orchestrate many agents to work collaboratively at scale to automate real-world activities.
-
python3.10
or above! -
$ pip install -U swarms
And, don't forget to install swarms! -
.env
file with API keys from your providers likeOPENAI_API_KEY
,ANTHROPIC_API_KEY
- Set an
.env
Variable with your desired workspace dir:WORKSPACE_DIR="agent_workspace"
or do it in your terminal withexport WORKSPACE_DIR="agent_workspace"
Refer to our documentation for production grade implementation details.
Section | Links |
---|---|
Installation | Installation |
Quickstart | Get Started |
Agent Internal Mechanisms | Agent Architecture |
Agent API | Agent API |
Integrating External Agents Griptape, Autogen, etc | Integrating External APIs |
Creating Agents from YAML | Creating Agents from YAML |
Why You Need Swarms | Why MultiAgent Collaboration is Necessary |
Swarm Architectures Analysis | Swarm Architectures |
Choosing the Right Swarm for Your Business Problem¶ | CLICK HERE |
AgentRearrange Docs | CLICK HERE |
$ pip3 install -U swarms
Now that you have downloaded swarms with pip3 install -U swarms
, we get access to the CLI
. Get Onboarded with CLI Now with:
swarms onboarding
You can also run this command for help:
swarms help
_________
/ _____/_ _ _______ _______ _____ ______
\_____ \ \/ \/ /\__ \_ __ \/ \ / ___/
/ \ / / __ \| | \/ Y Y \___ \
/_______ / \/\_/ (____ /__| |__|_| /____ >
\/ \/ \/ \/
Swarms CLI - Help
Commands:
onboarding : Starts the onboarding process
help : Shows this help message
get-api-key : Retrieves your API key from the platform
check-login : Checks if you're logged in and starts the cache
read-docs : Redirects you to swarms cloud documentation!
run-agents : Run your Agents from your agents.yaml
For more details, visit: https://docs.swarms.world
For more documentation on the CLI CLICK HERE
Here are some simple examples but we have more comprehensive documentation at our docs here
The Agent
class is a fundamental component of the Swarms framework, designed to execute tasks autonomously. It fuses llms, tools and long-term memory capabilities to create a full stack agent. The Agent
class is highly customizable, allowing for fine-grained control over its behavior and interactions.
The run
method is the primary entry point for executing tasks with an Agent
instance. It accepts a task string as the main input task and processes it according to the agent's configuration. And, it can also accept an img
parameter such as img="image_filepath.png
to process images if you have a VLM
The Agent
class offers a range of settings to tailor its behavior to specific needs. Some key settings include:
Setting | Description | Default Value |
---|---|---|
agent_name |
The name of the agent. | "DefaultAgent" |
system_prompt |
The system prompt to use for the agent. | "Default system prompt." |
llm |
The language model to use for processing tasks. |
OpenAIChat instance |
max_loops |
The maximum number of loops to execute for a task. | 1 |
autosave |
Enables or disables autosaving of the agent's state. | False |
dashboard |
Enables or disables the dashboard for the agent. | False |
verbose |
Controls the verbosity of the agent's output. | False |
dynamic_temperature_enabled |
Enables or disables dynamic temperature adjustment for the language model. | False |
saved_state_path |
The path to save the agent's state. | "agent_state.json" |
user_name |
The username associated with the agent. | "default_user" |
retry_attempts |
The number of retry attempts for failed tasks. | 1 |
context_length |
The maximum length of the context to consider for tasks. | 200000 |
return_step_meta |
Controls whether to return step metadata in the output. | False |
output_type |
The type of output to return (e.g., "json", "string"). | "string" |
import os
from swarms import Agent
from swarm_models import OpenAIChat
from swarms.prompts.finance_agent_sys_prompt import (
FINANCIAL_AGENT_SYS_PROMPT,
)
from dotenv import load_dotenv
load_dotenv()
# Get the OpenAI API key from the environment variable
api_key = os.getenv("OPENAI_API_KEY")
# Create an instance of the OpenAIChat class
model = OpenAIChat(
openai_api_key=api_key, model_name="gpt-4o-mini", temperature=0.1
)
# Initialize the agent
agent = Agent(
agent_name="Financial-Analysis-Agent",
system_prompt=FINANCIAL_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
autosave=True,
dashboard=False,
verbose=True,
dynamic_temperature_enabled=True,
saved_state_path="finance_agent.json",
user_name="swarms_corp",
retry_attempts=1,
context_length=200000,
return_step_meta=False,
# output_type="json",
)
out = agent.run(
"How can I establish a ROTH IRA to buy stocks and get a tax break? What are the criteria"
)
print(out)
Agent
equipped with quasi-infinite long term memory using RAG (Relational Agent Graph) for advanced document understanding, analysis, and retrieval capabilities.
Mermaid Diagram for RAG Integration
graph TD
A[Initialize Agent with RAG] --> B[Receive Task]
B --> C[Query Long-Term Memory]
C --> D[Process Task with Context]
D --> E[Generate Response]
E --> F[Update Long-Term Memory]
F --> G[Return Output]
Step 1: Initialize the ChromaDB Client
import os
from swarms_memory import ChromaDB
# Initialize the ChromaDB client for long-term memory management
chromadb = ChromaDB(
metric="cosine", # Metric for similarity measurement
output_dir="finance_agent_rag", # Directory for storing RAG data
# docs_folder="artifacts", # Uncomment and specify the folder containing your documents
)
Step 2: Define the Model
from swarm_models import Anthropic
from swarms.prompts.finance_agent_sys_prompt import (
FINANCIAL_AGENT_SYS_PROMPT,
)
# Define the Anthropic model for language processing
model = Anthropic(anthropic_api_key=os.getenv("ANTHROPIC_API_KEY"))
Step 3: Initialize the Agent with RAG
from swarms import Agent
# Initialize the agent with RAG capabilities
agent = Agent(
agent_name="Financial-Analysis-Agent",
system_prompt=FINANCIAL_AGENT_SYS_PROMPT,
agent_description="Agent creates a comprehensive financial analysis",
llm=model,
max_loops="auto", # Auto-adjusts loops based on task complexity
autosave=True, # Automatically saves agent state
dashboard=False, # Disables dashboard for this example
verbose=True, # Enables verbose mode for detailed output
streaming_on=True, # Enables streaming for real-time processing
dynamic_temperature_enabled=True, # Dynamically adjusts temperature for optimal performance
saved_state_path="finance_agent.json", # Path to save agent state
user_name="swarms_corp", # User name for the agent
retry_attempts=3, # Number of retry attempts for failed tasks
context_length=200000, # Maximum length of the context to consider
long_term_memory=chromadb, # Integrates ChromaDB for long-term memory management
)
# Run the agent with a sample task
agent.run(
"What are the components of a startups stock incentive equity plan"
)
We provide vast array of features to save agent states using json, yaml, toml, upload pdfs, batched jobs, and much more!
Method Table
Method | Description |
---|---|
to_dict() |
Converts the agent object to a dictionary. |
to_toml() |
Converts the agent object to a TOML string. |
model_dump_json() |
Dumps the model to a JSON file. |
model_dump_yaml() |
Dumps the model to a YAML file. |
ingest_docs() |
Ingests documents into the agent's knowledge base. |
receive_message() |
Receives a message from a user and processes it. |
send_agent_message() |
Sends a message from the agent to a user. |
filtered_run() |
Runs the agent with a filtered system prompt. |
bulk_run() |
Runs the agent with multiple system prompts. |
add_memory() |
Adds a memory to the agent. |
check_available_tokens() |
Checks the number of available tokens for the agent. |
tokens_checks() |
Performs token checks for the agent. |
print_dashboard() |
Prints the dashboard of the agent. |
get_docs_from_doc_folders() |
Fetches all the documents from the doc folders. |
activate_agentops() |
Activates agent operations. |
check_end_session_agentops() |
Checks the end of the session for agent operations. |
# # Convert the agent object to a dictionary
print(agent.to_dict())
print(agent.to_toml())
print(agent.model_dump_json())
print(agent.model_dump_yaml())
# Ingest documents into the agent's knowledge base
agent.ingest_docs("your_pdf_path.pdf")
# Receive a message from a user and process it
agent.receive_message(name="agent_name", message="message")
# Send a message from the agent to a user
agent.send_agent_message(agent_name="agent_name", message="message")
# Ingest multiple documents into the agent's knowledge base
agent.ingest_docs("your_pdf_path.pdf", "your_csv_path.csv")
# Run the agent with a filtered system prompt
agent.filtered_run(
"How can I establish a ROTH IRA to buy stocks and get a tax break? What are the criteria?"
)
# Run the agent with multiple system prompts
agent.bulk_run(
[
"How can I establish a ROTH IRA to buy stocks and get a tax break? What are the criteria?",
"Another system prompt",
]
)
# Add a memory to the agent
agent.add_memory("Add a memory to the agent")
# Check the number of available tokens for the agent
agent.check_available_tokens()
# Perform token checks for the agent
agent.tokens_checks()
# Print the dashboard of the agent
agent.print_dashboard()
# Fetch all the documents from the doc folders
agent.get_docs_from_doc_folders()
# Activate agent ops
agent.activate_agentops()
agent.check_end_session_agentops()
# Dump the model to a JSON file
agent.model_dump_json()
print(agent.to_toml())
The following is an example of an agent that intakes a pydantic basemodel and outputs it at the same time:
from pydantic import BaseModel, Field
from swarms import Agent
from swarm_models import Anthropic
# Initialize the schema for the person's information
class Schema(BaseModel):
name: str = Field(..., title="Name of the person")
agent: int = Field(..., title="Age of the person")
is_student: bool = Field(..., title="Whether the person is a student")
courses: list[str] = Field(
..., title="List of courses the person is taking"
)
# Convert the schema to a JSON string
tool_schema = Schema(
name="Tool Name",
agent=1,
is_student=True,
courses=["Course1", "Course2"],
)
# Define the task to generate a person's information
task = "Generate a person's information based on the following schema:"
# Initialize the agent
agent = Agent(
agent_name="Person Information Generator",
system_prompt=(
"Generate a person's information based on the following schema:"
),
# Set the tool schema to the JSON string -- this is the key difference
tool_schema=tool_schema,
llm=Anthropic(),
max_loops=3,
autosave=True,
dashboard=False,
streaming_on=True,
verbose=True,
interactive=True,
# Set the output type to the tool schema which is a BaseModel
output_type=tool_schema, # or dict, or str
metadata_output_type="json",
# List of schemas that the agent can handle
list_base_models=[tool_schema],
function_calling_format_type="OpenAI",
function_calling_type="json", # or soon yaml
)
# Run the agent to generate the person's information
generated_data = agent.run(task)
# Print the generated data
print(f"Generated data: {generated_data}")
Run the agent with multiple modalities useful for various real-world tasks in manufacturing, logistics, and health.
import os
from dotenv import load_dotenv
from swarms import Agent
from swarm_models import GPT4VisionAPI
# Load the environment variables
load_dotenv()
# Initialize the language model
llm = GPT4VisionAPI(
openai_api_key=os.environ.get("OPENAI_API_KEY"),
max_tokens=500,
)
# Initialize the task
task = (
"Analyze this image of an assembly line and identify any issues such as"
" misaligned parts, defects, or deviations from the standard assembly"
" process. IF there is anything unsafe in the image, explain why it is"
" unsafe and how it could be improved."
)
img = "assembly_line.jpg"
## Initialize the workflow
agent = Agent(
agent_name = "Multi-ModalAgent",
llm=llm,
max_loops="auto",
autosave=True,
dashboard=True,
multi_modal=True
)
# Run the workflow on a task
agent.run(task, img)
ToolAgent is an agent that can use tools through JSON function calling. It intakes any open source model from huggingface and is extremely modular and plug in and play. We need help adding general support to all models soon.
from pydantic import BaseModel, Field
from transformers import AutoModelForCausalLM, AutoTokenizer
from swarms import ToolAgent
from swarms.utils.json_utils import base_model_to_json
# Load the pre-trained model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
"databricks/dolly-v2-12b",
load_in_4bit=True,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("databricks/dolly-v2-12b")
# Initialize the schema for the person's information
class Schema(BaseModel):
name: str = Field(..., title="Name of the person")
agent: int = Field(..., title="Age of the person")
is_student: bool = Field(
..., title="Whether the person is a student"
)
courses: list[str] = Field(
..., title="List of courses the person is taking"
)
# Convert the schema to a JSON string
tool_schema = base_model_to_json(Schema)
# Define the task to generate a person's information
task = (
"Generate a person's information based on the following schema:"
)
# Create an instance of the ToolAgent class
agent = ToolAgent(
name="dolly-function-agent",
description="Ana gent to create a child data",
model=model,
tokenizer=tokenizer,
json_schema=tool_schema,
)
# Run the agent to generate the person's information
generated_data = agent.run(task)
# Print the generated data
print(f"Generated data: {generated_data}")
Integrating external agents from other agent frameworks is easy with swarms.
Steps:
- Create a new class that inherits
Agent
- Create a
.run(task: str) -> str
method that runs the agent and returns the response. - The new Agent must return a string of the response. But you may add additional methods to save the output to JSON.
For example, here's an example on how to create an agent from griptape.
Here’s how you can create a custom Griptape agent that integrates with the Swarms framework by inheriting from the Agent
class in Swarms and overriding the run(task: str) -> str
method.
from swarms import (
Agent as SwarmsAgent,
) # Import the base Agent class from Swarms
from griptape.structures import Agent as GriptapeAgent
from griptape.tools import (
WebScraperTool,
FileManagerTool,
PromptSummaryTool,
)
# Create a custom agent class that inherits from SwarmsAgent
class GriptapeSwarmsAgent(SwarmsAgent):
def __init__(self, *args, **kwargs):
# Initialize the Griptape agent with its tools
self.agent = GriptapeAgent(
input="Load {{ args[0] }}, summarize it, and store it in a file called {{ args[1] }}.",
tools=[
WebScraperTool(off_prompt=True),
PromptSummaryTool(off_prompt=True),
FileManagerTool(),
],
*args,
**kwargs,
# Add additional settings
)
# Override the run method to take a task and execute it using the Griptape agent
def run(self, task: str) -> str:
# Extract URL and filename from task (you can modify this parsing based on task structure)
url, filename = task.split(
","
) # Example of splitting task string
# Execute the Griptape agent with the task inputs
result = self.agent.run(url.strip(), filename.strip())
# Return the final result as a string
return str(result)
# Example usage:
griptape_swarms_agent = GriptapeSwarmsAgent()
output = griptape_swarms_agent.run(
"https://griptape.ai, griptape.txt"
)
print(output)
-
GriptapeSwarmsAgent: A custom class that inherits from the
SwarmsAgent
class and integrates the Griptape agent. - run(task: str) -> str: A method that takes a task string, processes it (e.g., splitting into a URL and filename), and runs the Griptape agent with the provided inputs.
-
Griptape Tools: The tools integrated into the Griptape agent (e.g.,
WebScraperTool
,PromptSummaryTool
,FileManagerTool
) allow for web scraping, summarization, and file management.
You can now easily plug this custom Griptape agent into the Swarms Framework and use it to run tasks!
Swarms was designed to facilitate the communication between many different and specialized agents from a vast array of other frameworks such as langchain, autogen, crew, and more.
In traditional swarm theory, there are many types of swarms usually for very specialized use-cases and problem sets. Such as Hiearchical and sequential are great for accounting and sales, because there is usually a boss coordinator agent that distributes a workload to other specialized agents.
Name | Description | Code Link | Use Cases |
---|---|---|---|
Hierarchical Swarms | A system where agents are organized in a hierarchy, with higher-level agents coordinating lower-level agents to achieve complex tasks. | Code Link | Manufacturing process optimization, multi-level sales management, healthcare resource coordination |
Agent Rearrange | A setup where agents rearrange themselves dynamically based on the task requirements and environmental conditions. | Code Link | Adaptive manufacturing lines, dynamic sales territory realignment, flexible healthcare staffing |
Concurrent Workflows | Agents perform different tasks simultaneously, coordinating to complete a larger goal. | Code Link | Concurrent production lines, parallel sales operations, simultaneous patient care processes |
Sequential Coordination | Agents perform tasks in a specific sequence, where the completion of one task triggers the start of the next. | Code Link | Step-by-step assembly lines, sequential sales processes, stepwise patient treatment workflows |
Parallel Processing | Agents work on different parts of a task simultaneously to speed up the overall process. | Code Link | Parallel data processing in manufacturing, simultaneous sales analytics, concurrent medical tests |
Mixture of Agents | A heterogeneous swarm where agents with different capabilities are combined to solve complex problems. | Code Link | Financial forecasting, complex problem-solving requiring diverse skills |
Graph Workflow | Agents collaborate in a directed acyclic graph (DAG) format to manage dependencies and parallel tasks. | Code Link | AI-driven software development pipelines, complex project management |
Group Chat | Agents engage in a chat-like interaction to reach decisions collaboratively. | Code Link | Real-time collaborative decision-making, contract negotiations |
Agent Registry | A centralized registry where agents are stored, retrieved, and invoked dynamically. | Code Link | Dynamic agent management, evolving recommendation engines |
Spreadsheet Swarm | Manages tasks at scale, tracking agent outputs in a structured format like CSV files. | Code Link | Large-scale marketing analytics, financial audits |
Forest Swarm | A swarm structure that organizes agents in a tree-like hierarchy for complex decision-making processes. | Code Link | Multi-stage workflows, hierarchical reinforcement learning |
Sequential Workflow enables you to sequentially execute tasks with Agent
and then pass the output into the next agent and onwards until you have specified your max loops.
graph LR
A[Agent 1] --> B[Agent 2]
B --> C[Agent 3]
C --> D[Agent 4]
D --> E[Max Loops]
E --> F[End]
In this example, each Agent
represents a task that is executed sequentially. The output of each agent is passed to the next agent in the sequence until the maximum number of loops is reached. This workflow is particularly useful for tasks that require a series of steps to be executed in a specific order, such as data processing pipelines or complex calculations that rely on the output of previous steps.
from swarms import Agent, SequentialWorkflow
from swarm_models import Anthropic
# Initialize the language model agent (e.g., GPT-3)
llm = Anthropic()
# Initialize agents for individual tasks
agent1 = Agent(
agent_name="Blog generator",
system_prompt="Generate a blog post like stephen king",
llm=llm,
max_loops=1,
dashboard=False,
tools=[],
)
agent2 = Agent(
agent_name="summarizer",
system_prompt="Sumamrize the blog post",
llm=llm,
max_loops=1,
dashboard=False,
tools=[],
)
# Create the Sequential workflow
workflow = SequentialWorkflow(
agents=[agent1, agent2], max_loops=1, verbose=False
)
# Run the workflow
workflow.run(
"Generate a blog post on how swarms of agents can help businesses grow."
)
The AgentRearrange
orchestration technique, inspired by Einops and einsum, allows you to define and map out the relationships between various agents. It provides a powerful tool for orchestrating complex workflows, enabling you to specify linear and sequential relationships such as a -> a1 -> a2 -> a3
, or concurrent relationships where the first agent sends a message to 3 agents simultaneously: a -> a1, a2, a3
. This level of customization allows for the creation of highly efficient and dynamic workflows, where agents can work in parallel or in sequence as needed. The AgentRearrange
technique is a valuable addition to the swarms library, providing a new level of flexibility and control over the orchestration of agents. For more detailed information and examples, please refer to the official documentation.
from swarms import Agent, AgentRearrange
from swarm_models import Anthropic
# Initialize the director agent
director = Agent(
agent_name="Director",
system_prompt="Directs the tasks for the workers",
llm=Anthropic(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="director.json",
)
# Initialize worker 1
worker1 = Agent(
agent_name="Worker1",
system_prompt="Generates a transcript for a youtube video on what swarms are",
llm=Anthropic(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="worker1.json",
)
# Initialize worker 2
worker2 = Agent(
agent_name="Worker2",
system_prompt="Summarizes the transcript generated by Worker1",
llm=Anthropic(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="worker2.json",
)
# Create a list of agents
agents = [director, worker1, worker2]
# Define the flow pattern
flow = "Director -> Worker1 -> Worker2"
# Using AgentRearrange class
agent_system = AgentRearrange(agents=agents, flow=flow)
output = agent_system.run(
"Create a format to express and communicate swarms of llms in a structured manner for youtube"
)
print(output)
Coming soon...
The GraphSwarm
is a workflow management system designed to orchestrate complex tasks by leveraging the power of graph theory. It enables the creation of a directed acyclic graph (DAG) to model dependencies between tasks and agents. This allows for efficient task assignment, execution, and monitoring.
Here's a breakdown of how the GraphSwarm
works:
-
Node Creation: The
GraphSwarm
workflow is composed of nodes, which can be either agents or tasks. Agents are responsible for executing tasks, and tasks represent specific operations that need to be performed. In the example, two agents (agent1
andagent2
) and one task (task1
) are created. -
Edge Definition: Edges are used to define the relationships between nodes. In this case, edges are created to connect
agent1
andagent2
totask1
, indicating that both agents are capable of executingtask1
. -
Entry and End Points: The
GraphSwarm
workflow requires the definition of entry points (where the workflow starts) and end points (where the workflow concludes). In this example,agent1
andagent2
are set as entry points, andtask1
is set as the end point. -
Visualization: The
GraphSwarm
provides a visualization feature to graphically represent the workflow. This allows for easy understanding and debugging of the workflow structure. -
Execution: The
GraphSwarm
workflow is executed by traversing the graph from the entry points to the end points. In this case, bothagent1
andagent2
executetask1
concurrently, and the results are collected. -
Results: The final results of the workflow execution are aggregated and returned. In this example, the result of executing
task1
is "Task completed".
The GraphSwarm
offers several benefits, including:
- Concurrency: Enables the execution of tasks concurrently, improving overall workflow efficiency.
- Flexibility: Allows for dynamic task assignment based on agent availability and task requirements.
- Scalability: Supports the addition of new agents and tasks as needed, making it suitable for large-scale workflows.
- Visualization: Provides a graphical representation of the workflow, facilitating understanding and debugging.
By leveraging the GraphSwarm
, complex workflows can be efficiently managed, and tasks can be executed in a coordinated and scalable manner.
import os
from dotenv import load_dotenv
from swarms import Agent, Edge, GraphWorkflow, Node, NodeType
from swarm_models import OpenAIChat
load_dotenv()
api_key = os.environ.get("OPENAI_API_KEY")
llm = OpenAIChat(
temperature=0.5, openai_api_key=api_key, max_tokens=4000
)
agent1 = Agent(llm=llm, max_loops=1, autosave=True, dashboard=True)
agent2 = Agent(llm=llm, max_loops=1, autosave=True, dashboard=True)
def sample_task():
print("Running sample task")
return "Task completed"
wf_graph = GraphWorkflow()
wf_graph.add_node(Node(id="agent1", type=NodeType.AGENT, agent=agent1))
wf_graph.add_node(Node(id="agent2", type=NodeType.AGENT, agent=agent2))
wf_graph.add_node(
Node(id="task1", type=NodeType.TASK, callable=sample_task)
)
wf_graph.add_edge(Edge(source="agent1", target="task1"))
wf_graph.add_edge(Edge(source="agent2", target="task1"))
wf_graph.set_entry_points(["agent1", "agent2"])
wf_graph.set_end_points(["task1"])
print(wf_graph.visualize())
# Run the workflow
results = wf_graph.run()
print("Execution results:", results)
This is an implementation based on the paper: "Mixture-of-Agents Enhances Large Language Model Capabilities" by together.ai, available at https://arxiv.org/abs/2406.04692. It achieves state-of-the-art (SOTA) results on AlpacaEval 2.0, MT-Bench, and FLASK, surpassing GPT-4 Omni. This architecture is particularly suitable for tasks that require parallelization followed by sequential processing in another loop.
from swarms import Agent, OpenAIChat, MixtureOfAgents
# Initialize the director agent
director = Agent(
agent_name="Director",
system_prompt="Directs the tasks for the accountants",
llm=OpenAIChat(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="director.json",
)
# Initialize accountant 1
accountant1 = Agent(
agent_name="Accountant1",
system_prompt="Prepares financial statements",
llm=OpenAIChat(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="accountant1.json",
)
# Initialize accountant 2
accountant2 = Agent(
agent_name="Accountant2",
system_prompt="Audits financial records",
llm=OpenAIChat(),
max_loops=1,
dashboard=False,
streaming_on=True,
verbose=True,
stopping_token="<DONE>",
state_save_file_type="json",
saved_state_path="accountant2.json",
)
# Create a list of agents
agents = [director, accountant1, accountant2]
# Swarm
swarm = MixtureOfAgents(
name="Mixture of Accountants",
agents=agents,
layers=3,
final_agent=director,
)
# Run the swarm
out = swarm.run("Prepare financial statements and audit financial records")
print(out)
A revolutionary swarm architecture designed for concurrent management and oversight of thousands of agents, facilitating a one-to-many approach for efficient task processing and output analysis.
- Concurrency: Enables the simultaneous execution of multiple agents, significantly reducing processing time and increasing overall system efficiency.
- One-to-Many: Allows a single task to be dynamically distributed among multiple agents, ensuring that each agent is utilized to its full potential.
- Scalability: Supports the integration of thousands of agents, making it an ideal solution for large-scale task processing and data analysis.
graph LR
A[Task] -->|Distributed|> B[Agent 1]
A -->|Distributed|> C[Agent 2]
A -->|Distributed|> D[Agent 3]
B -->|Output|> E[SpreadSheetSwarm]
C -->|Output|> E
D -->|Output|> E
E -->|Aggregated Output|> F[User]
import os
from swarms import Agent
from swarm_models import OpenAIChat
from swarms.structs.spreadsheet_swarm import SpreadSheetSwarm
# Define custom system prompts for each social media platform
TWITTER_AGENT_SYS_PROMPT = """
You are a Twitter marketing expert specializing in real estate. Your task is to create engaging, concise tweets to promote properties, analyze trends to maximize engagement, and use appropriate hashtags and timing to reach potential buyers.
"""
INSTAGRAM_AGENT_SYS_PROMPT = """
You are an Instagram marketing expert focusing on real estate. Your task is to create visually appealing posts with engaging captions and hashtags to showcase properties, targeting specific demographics interested in real estate.
"""
FACEBOOK_AGENT_SYS_PROMPT = """
You are a Facebook marketing expert for real estate. Your task is to craft posts optimized for engagement and reach on Facebook, including using images, links, and targeted messaging to attract potential property buyers.
"""
LINKEDIN_AGENT_SYS_PROMPT = """
You are a LinkedIn marketing expert for the real estate industry. Your task is to create professional and informative posts, highlighting property features, market trends, and investment opportunities, tailored to professionals and investors.
"""
EMAIL_AGENT_SYS_PROMPT = """
You are an Email marketing expert specializing in real estate. Your task is to write compelling email campaigns to promote properties, focusing on personalization, subject lines, and effective call-to-action strategies to drive conversions.
"""
# Example usage:
api_key = os.getenv("OPENAI_API_KEY")
# Model
model = OpenAIChat(
openai_api_key=api_key, model_name="gpt-4o-mini", temperature=0.1
)
# Initialize your agents for different social media platforms
agents = [
Agent(
agent_name="Twitter-RealEstate-Agent",
system_prompt=TWITTER_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
dynamic_temperature_enabled=True,
saved_state_path="twitter_realestate_agent.json",
user_name="realestate_swarms",
retry_attempts=1,
),
Agent(
agent_name="Instagram-RealEstate-Agent",
system_prompt=INSTAGRAM_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
dynamic_temperature_enabled=True,
saved_state_path="instagram_realestate_agent.json",
user_name="realestate_swarms",
retry_attempts=1,
),
Agent(
agent_name="Facebook-RealEstate-Agent",
system_prompt=FACEBOOK_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
dynamic_temperature_enabled=True,
saved_state_path="facebook_realestate_agent.json",
user_name="realestate_swarms",
retry_attempts=1,
),
Agent(
agent_name="LinkedIn-RealEstate-Agent",
system_prompt=LINKEDIN_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
dynamic_temperature_enabled=True,
saved_state_path="linkedin_realestate_agent.json",
user_name="realestate_swarms",
retry_attempts=1,
),
Agent(
agent_name="Email-RealEstate-Agent",
system_prompt=EMAIL_AGENT_SYS_PROMPT,
llm=model,
max_loops=1,
dynamic_temperature_enabled=True,
saved_state_path="email_realestate_agent.json",
user_name="realestate_swarms",
retry_attempts=1,
),
]
# Create a Swarm with the list of agents
swarm = SpreadSheetSwarm(
name="Real-Estate-Marketing-Swarm",
description="A swarm that processes real estate marketing tasks using multiple agents on different threads.",
agents=agents,
autosave_on=True,
save_file_path="real_estate_marketing_spreadsheet.csv",
run_all_agents=False,
max_loops=2,
)
# Run the swarm
swarm.run(
task="""
Create posts to promote luxury properties in North Texas, highlighting their features, location, and investment potential. Include relevant hashtags, images, and engaging captions.
Property:
$10,399,000
1609 Meandering Way Dr, Roanoke, TX 76262
Link to the property: https://www.zillow.com/homedetails/1609-Meandering-Way-Dr-Roanoke-TX-76262/308879785_zpid/
What's special
Unveiling a new custom estate in the prestigious gated Quail Hollow Estates! This impeccable residence, set on a sprawling acre surrounded by majestic trees, features a gourmet kitchen equipped with top-tier Subzero and Wolf appliances. European soft-close cabinets and drawers, paired with a double Cambria Quartzite island, perfect for family gatherings. The first-floor game room&media room add extra layers of entertainment. Step into the outdoor sanctuary, where a sparkling pool and spa, and sunken fire pit, beckon leisure. The lavish master suite features stunning marble accents, custom his&her closets, and a secure storm shelter.Throughout the home,indulge in the visual charm of designer lighting and wallpaper, elevating every space. The property is complete with a 6-car garage and a sports court, catering to the preferences of basketball or pickleball enthusiasts. This residence seamlessly combines luxury&recreational amenities, making it a must-see for the discerning buyer.
Facts & features
Interior
Bedrooms & bathrooms
Bedrooms: 6
Bathrooms: 8
Full bathrooms: 7
1/2 bathrooms: 1
Primary bedroom
Bedroom
Features: Built-in Features, En Suite Bathroom, Walk-In Closet(s)
Cooling
Central Air, Ceiling Fan(s), Electric
Appliances
Included: Built-In Gas Range, Built-In Refrigerator, Double Oven, Dishwasher, Gas Cooktop, Disposal, Ice Maker, Microwave, Range, Refrigerator, Some Commercial Grade, Vented Exhaust Fan, Warming Drawer, Wine Cooler
Features
Wet Bar, Built-in Features, Dry Bar, Decorative/Designer Lighting Fixtures, Eat-in Kitchen, Elevator, High Speed Internet, Kitchen Island, Pantry, Smart Home, Cable TV, Walk-In Closet(s), Wired for Sound
Flooring: Hardwood
Has basement: No
Number of fireplaces: 3
Fireplace features: Living Room, Primary Bedroom
Interior area
Total interior livable area: 10,466 sqft
Total spaces: 12
Parking features: Additional Parking
Attached garage spaces: 6
Carport spaces: 6
Features
Levels: Two
Stories: 2
Patio & porch: Covered
Exterior features: Built-in Barbecue, Barbecue, Gas Grill, Lighting, Outdoor Grill, Outdoor Living Area, Private Yard, Sport Court, Fire Pit
Pool features: Heated, In Ground, Pool, Pool/Spa Combo
Fencing: Wrought Iron
Lot
Size: 1.05 Acres
Details
Additional structures: Outdoor Kitchen
Parcel number: 42232692
Special conditions: Standard
Construction
Type & style
Home type: SingleFamily
Architectural style: Contemporary/Modern,Detached
Property subtype: Single Family Residence
"""
)
The ForestSwarm
architecture is designed for efficient task assignment by dynamically selecting the most suitable agent from a collection of trees. This is achieved through asynchronous task processing, where agents are chosen based on their relevance to the task at hand. The relevance is determined by calculating the similarity between the system prompts associated with each agent and the keywords present in the task itself. For a more in-depth understanding of how ForestSwarm
works, please refer to the official documentation.
from swarms.structs.tree_swarm import TreeAgent, Tree, ForestSwarm
# Example Usage:
# Create agents with varying system prompts and dynamically generated distances/keywords
agents_tree1 = [
TreeAgent(
system_prompt="Stock Analysis Agent",
agent_name="Stock Analysis Agent",
),
TreeAgent(
system_prompt="Financial Planning Agent",
agent_name="Financial Planning Agent",
),
TreeAgent(
agent_name="Retirement Strategy Agent",
system_prompt="Retirement Strategy Agent",
),
]
agents_tree2 = [
TreeAgent(
system_prompt="Tax Filing Agent",
agent_name="Tax Filing Agent",
),
TreeAgent(
system_prompt="Investment Strategy Agent",
agent_name="Investment Strategy Agent",
),
TreeAgent(
system_prompt="ROTH IRA Agent", agent_name="ROTH IRA Agent"
),
]
# Create trees
tree1 = Tree(tree_name="Financial Tree", agents=agents_tree1)
tree2 = Tree(tree_name="Investment Tree", agents=agents_tree2)
# Create the ForestSwarm
multi_agent_structure = ForestSwarm(trees=[tree1, tree2])
# Run a task
task = "Our company is incorporated in delaware, how do we do our taxes for free?"
output = multi_agent_structure.run(task)
print(output)
Get onboarded now with the creator and lead maintainer of Swarms, Kye Gomez, who will show you how to get started with the installation, usage examples, and starting to build your custom use case! CLICK HERE
Documentation is located here at: docs.swarms.world
The swarms package has been meticlously crafted for extreme use-ability and understanding, the swarms package is split up into various modules such as swarms.agents
that holds pre-built agents, swarms.structs
that holds a vast array of structures like Agent
and multi agent structures. The 3 most important are structs
, models
, and agents
.
├── __init__.py
├── agents
├── artifacts
├── memory
├── schemas
├── models -> swarm_models
├── prompts
├── structs
├── telemetry
├── tools
├── utils
└── workers
The easiest way to contribute is to pick any issue with the good first issue
tag 💪. Read the Contributing guidelines here. Bug Report? File here | Feature Request? File here
Swarms is an open-source project, and contributions are VERY welcome. If you want to contribute, you can create new features, fix bugs, or improve the infrastructure. Please refer to the CONTRIBUTING.md and our contributing board to participate in Roadmap discussions!
Accelerate Bugs, Features, and Demos to implement by supporting us here:
Join our growing community around the world, for real-time support, ideas, and discussions on Swarms 😊
- View our official Blog
- Chat live with us on Discord
- Follow us on Twitter
- Connect with us on LinkedIn
- Visit us on YouTube
- Join the Swarms community on Discord!
- Join our Swarms Community Gathering every Thursday at 1pm NYC Time to unlock the potential of autonomous agents in automating your daily tasks Sign up here
Creative Commons Attribution 4.0 International Public License
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for swarms
Similar Open Source Tools
swarms
Swarms provides simple, reliable, and agile tools to create your own Swarm tailored to your specific needs. Currently, Swarms is being used in production by RBC, John Deere, and many AI startups.
continuous-eval
Open-Source Evaluation for LLM Applications. `continuous-eval` is an open-source package created for granular and holistic evaluation of GenAI application pipelines. It offers modularized evaluation, a comprehensive metric library covering various LLM use cases, the ability to leverage user feedback in evaluation, and synthetic dataset generation for testing pipelines. Users can define their own metrics by extending the Metric class. The tool allows running evaluation on a pipeline defined with modules and corresponding metrics. Additionally, it provides synthetic data generation capabilities to create user interaction data for evaluation or training purposes.
redisvl
Redis Vector Library (RedisVL) is a Python client library for building AI applications on top of Redis. It provides a high-level interface for managing vector indexes, performing vector search, and integrating with popular embedding models and providers. RedisVL is designed to make it easy for developers to build and deploy AI applications that leverage the speed, flexibility, and reliability of Redis.
mlx-llm
mlx-llm is a library that allows you to run Large Language Models (LLMs) on Apple Silicon devices in real-time using Apple's MLX framework. It provides a simple and easy-to-use API for creating, loading, and using LLM models, as well as a variety of applications such as chatbots, fine-tuning, and retrieval-augmented generation.
rl
TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch. It provides pytorch and **python-first** , low and high level abstractions for RL that are intended to be **efficient** , **modular** , **documented** and properly **tested**. The code is aimed at supporting research in RL. Most of it is written in python in a highly modular way, such that researchers can easily swap components, transform them or write new ones with little effort.
litdata
LitData is a tool designed for blazingly fast, distributed streaming of training data from any cloud storage. It allows users to transform and optimize data in cloud storage environments efficiently and intuitively, supporting various data types like images, text, video, audio, geo-spatial, and multimodal data. LitData integrates smoothly with frameworks such as LitGPT and PyTorch, enabling seamless streaming of data to multiple machines. Key features include multi-GPU/multi-node support, easy data mixing, pause & resume functionality, support for profiling, memory footprint reduction, cache size configuration, and on-prem optimizations. The tool also provides benchmarks for measuring streaming speed and conversion efficiency, along with runnable templates for different data types. LitData enables infinite cloud data processing by utilizing the Lightning.ai platform to scale data processing with optimized machines.
xlstm
xLSTM is a new Recurrent Neural Network architecture based on ideas of the original LSTM. Through Exponential Gating with appropriate normalization and stabilization techniques and a new Matrix Memory it overcomes the limitations of the original LSTM and shows promising performance on Language Modeling when compared to Transformers or State Space Models. The package is based on PyTorch and was tested for versions >=1.8. For the CUDA version of xLSTM, you need Compute Capability >= 8.0. The xLSTM tool provides two main components: xLSTMBlockStack for non-language applications or integrating in other architectures, and xLSTMLMModel for language modeling or other token-based applications.
syncode
SynCode is a novel framework for the grammar-guided generation of Large Language Models (LLMs) that ensures syntactically valid output with respect to defined Context-Free Grammar (CFG) rules. It supports general-purpose programming languages like Python, Go, SQL, JSON, and more, allowing users to define custom grammars using EBNF syntax. The tool compares favorably to other constrained decoders and offers features like fast grammar-guided generation, compatibility with HuggingFace Language Models, and the ability to work with various decoding strategies.
ecologits
EcoLogits tracks energy consumption and environmental impacts of generative AI models through APIs. It provides estimated environmental impacts of the inference, such as energy consumption and GHG emissions. The tool supports integration with various providers like Anthropic, Cohere, Google GenerativeAI, Huggingface Hub, MistralAI, and OpenAI. Users can easily install EcoLogits using pip and access detailed documentation on ecologits.ai. The project welcomes contributions and is licensed under MPL-2.0.
camel
CAMEL is an open-source library designed for the study of autonomous and communicative agents. We believe that studying these agents on a large scale offers valuable insights into their behaviors, capabilities, and potential risks. To facilitate research in this field, we implement and support various types of agents, tasks, prompts, models, and simulated environments.
LightRAG
LightRAG is a PyTorch library designed for building and optimizing Retriever-Agent-Generator (RAG) pipelines. It follows principles of simplicity, quality, and optimization, offering developers maximum customizability with minimal abstraction. The library includes components for model interaction, output parsing, and structured data generation. LightRAG facilitates tasks like providing explanations and examples for concepts through a question-answering pipeline.
clarifai-python-grpc
This is the official Clarifai gRPC Python client for interacting with their recognition API. Clarifai offers a platform for data scientists, developers, researchers, and enterprises to utilize artificial intelligence for image, video, and text analysis through computer vision and natural language processing. The client allows users to authenticate, predict concepts in images, and access various functionalities provided by the Clarifai API. It follows a versioning scheme that aligns with the backend API updates and includes specific instructions for installation and troubleshooting. Users can explore the Clarifai demo, sign up for an account, and refer to the documentation for detailed information.
litserve
LitServe is a high-throughput serving engine for deploying AI models at scale. It generates an API endpoint for a model, handles batching, streaming, autoscaling across CPU/GPUs, and more. Built for enterprise scale, it supports every framework like PyTorch, JAX, Tensorflow, and more. LitServe is designed to let users focus on model performance, not the serving boilerplate. It is like PyTorch Lightning for model serving but with broader framework support and scalability.
SenseVoice
SenseVoice is a speech foundation model focusing on high-accuracy multilingual speech recognition, speech emotion recognition, and audio event detection. Trained with over 400,000 hours of data, it supports more than 50 languages and excels in emotion recognition and sound event detection. The model offers efficient inference with low latency and convenient finetuning scripts. It can be deployed for service with support for multiple client-side languages. SenseVoice-Small model is open-sourced and provides capabilities for Mandarin, Cantonese, English, Japanese, and Korean. The tool also includes features for natural speech generation and fundamental speech recognition tasks.
candle-vllm
Candle-vllm is an efficient and easy-to-use platform designed for inference and serving local LLMs, featuring an OpenAI compatible API server. It offers a highly extensible trait-based system for rapid implementation of new module pipelines, streaming support in generation, efficient management of key-value cache with PagedAttention, and continuous batching. The tool supports chat serving for various models and provides a seamless experience for users to interact with LLMs through different interfaces.
beyondllm
Beyond LLM offers an all-in-one toolkit for experimentation, evaluation, and deployment of Retrieval-Augmented Generation (RAG) systems. It simplifies the process with automated integration, customizable evaluation metrics, and support for various Large Language Models (LLMs) tailored to specific needs. The aim is to reduce LLM hallucination risks and enhance reliability.
For similar tasks
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.
onnxruntime-genai
ONNX Runtime Generative AI is a library that provides the generative AI loop for ONNX models, including inference with ONNX Runtime, logits processing, search and sampling, and KV cache management. Users can call a high level `generate()` method, or run each iteration of the model in a loop. It supports greedy/beam search and TopP, TopK sampling to generate token sequences, has built in logits processing like repetition penalties, and allows for easy custom scoring.
jupyter-ai
Jupyter AI connects generative AI with Jupyter notebooks. It provides a user-friendly and powerful way to explore generative AI models in notebooks and improve your productivity in JupyterLab and the Jupyter Notebook. Specifically, Jupyter AI offers: * An `%%ai` magic that turns the Jupyter notebook into a reproducible generative AI playground. This works anywhere the IPython kernel runs (JupyterLab, Jupyter Notebook, Google Colab, Kaggle, VSCode, etc.). * A native chat UI in JupyterLab that enables you to work with generative AI as a conversational assistant. * Support for a wide range of generative model providers, including AI21, Anthropic, AWS, Cohere, Gemini, Hugging Face, NVIDIA, and OpenAI. * Local model support through GPT4All, enabling use of generative AI models on consumer grade machines with ease and privacy.
khoj
Khoj is an open-source, personal AI assistant that extends your capabilities by creating always-available AI agents. You can share your notes and documents to extend your digital brain, and your AI agents have access to the internet, allowing you to incorporate real-time information. Khoj is accessible on Desktop, Emacs, Obsidian, Web, and Whatsapp, and you can share PDF, markdown, org-mode, notion files, and GitHub repositories. You'll get fast, accurate semantic search on top of your docs, and your agents can create deeply personal images and understand your speech. Khoj is self-hostable and always will be.
langchain_dart
LangChain.dart is a Dart port of the popular LangChain Python framework created by Harrison Chase. LangChain provides a set of ready-to-use components for working with language models and a standard interface for chaining them together to formulate more advanced use cases (e.g. chatbots, Q&A with RAG, agents, summarization, extraction, etc.). The components can be grouped into a few core modules: * **Model I/O:** LangChain offers a unified API for interacting with various LLM providers (e.g. OpenAI, Google, Mistral, Ollama, etc.), allowing developers to switch between them with ease. Additionally, it provides tools for managing model inputs (prompt templates and example selectors) and parsing the resulting model outputs (output parsers). * **Retrieval:** assists in loading user data (via document loaders), transforming it (with text splitters), extracting its meaning (using embedding models), storing (in vector stores) and retrieving it (through retrievers) so that it can be used to ground the model's responses (i.e. Retrieval-Augmented Generation or RAG). * **Agents:** "bots" that leverage LLMs to make informed decisions about which available tools (such as web search, calculators, database lookup, etc.) to use to accomplish the designated task. The different components can be composed together using the LangChain Expression Language (LCEL).
danswer
Danswer is an open-source Gen-AI Chat and Unified Search tool that connects to your company's docs, apps, and people. It provides a Chat interface and plugs into any LLM of your choice. Danswer can be deployed anywhere and for any scale - on a laptop, on-premise, or to cloud. Since you own the deployment, your user data and chats are fully in your own control. Danswer is MIT licensed and designed to be modular and easily extensible. The system also comes fully ready for production usage with user authentication, role management (admin/basic users), chat persistence, and a UI for configuring Personas (AI Assistants) and their Prompts. Danswer also serves as a Unified Search across all common workplace tools such as Slack, Google Drive, Confluence, etc. By combining LLMs and team specific knowledge, Danswer becomes a subject matter expert for the team. Imagine ChatGPT if it had access to your team's unique knowledge! It enables questions such as "A customer wants feature X, is this already supported?" or "Where's the pull request for feature Y?"
infinity
Infinity is an AI-native database designed for LLM applications, providing incredibly fast full-text and vector search capabilities. It supports a wide range of data types, including vectors, full-text, and structured data, and offers a fused search feature that combines multiple embeddings and full text. Infinity is easy to use, with an intuitive Python API and a single-binary architecture that simplifies deployment. It achieves high performance, with 0.1 milliseconds query latency on million-scale vector datasets and up to 15K QPS.
For similar jobs
weave
Weave is a toolkit for developing Generative AI applications, built by Weights & Biases. With Weave, you can log and debug language model inputs, outputs, and traces; build rigorous, apples-to-apples evaluations for language model use cases; and organize all the information generated across the LLM workflow, from experimentation to evaluations to production. Weave aims to bring rigor, best-practices, and composability to the inherently experimental process of developing Generative AI software, without introducing cognitive overhead.
agentcloud
AgentCloud is an open-source platform that enables companies to build and deploy private LLM chat apps, empowering teams to securely interact with their data. It comprises three main components: Agent Backend, Webapp, and Vector Proxy. To run this project locally, clone the repository, install Docker, and start the services. The project is licensed under the GNU Affero General Public License, version 3 only. Contributions and feedback are welcome from the community.
oss-fuzz-gen
This framework generates fuzz targets for real-world `C`/`C++` projects with various Large Language Models (LLM) and benchmarks them via the `OSS-Fuzz` platform. It manages to successfully leverage LLMs to generate valid fuzz targets (which generate non-zero coverage increase) for 160 C/C++ projects. The maximum line coverage increase is 29% from the existing human-written targets.
LLMStack
LLMStack is a no-code platform for building generative AI agents, workflows, and chatbots. It allows users to connect their own data, internal tools, and GPT-powered models without any coding experience. LLMStack can be deployed to the cloud or on-premise and can be accessed via HTTP API or triggered from Slack or Discord.
VisionCraft
The VisionCraft API is a free API for using over 100 different AI models. From images to sound.
kaito
Kaito is an operator that automates the AI/ML inference model deployment in a Kubernetes cluster. It manages large model files using container images, avoids tuning deployment parameters to fit GPU hardware by providing preset configurations, auto-provisions GPU nodes based on model requirements, and hosts large model images in the public Microsoft Container Registry (MCR) if the license allows. Using Kaito, the workflow of onboarding large AI inference models in Kubernetes is largely simplified.
PyRIT
PyRIT is an open access automation framework designed to empower security professionals and ML engineers to red team foundation models and their applications. It automates AI Red Teaming tasks to allow operators to focus on more complicated and time-consuming tasks and can also identify security harms such as misuse (e.g., malware generation, jailbreaking), and privacy harms (e.g., identity theft). The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model. This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.
Azure-Analytics-and-AI-Engagement
The Azure-Analytics-and-AI-Engagement repository provides packaged Industry Scenario DREAM Demos with ARM templates (Containing a demo web application, Power BI reports, Synapse resources, AML Notebooks etc.) that can be deployed in a customer’s subscription using the CAPE tool within a matter of few hours. Partners can also deploy DREAM Demos in their own subscriptions using DPoC.