acte
A framework to build GUI-like Agent Tools, enhancement to Function Calling of LLM AI.
Stars: 113
Acte is a framework designed to build GUI-like tools for AI Agents. It aims to address the issues of cognitive load and freedom degrees when interacting with multiple APIs in complex scenarios. By providing a graphical user interface (GUI) for Agents, Acte helps reduce cognitive load and constraints interaction, similar to how humans interact with computers through GUIs. The tool offers APIs for starting new sessions, executing actions, and displaying screens, accessible via HTTP requests or the SessionManager class.
README:
Acte is a framework to build GUI-like tools for AI Agents.
The current tool solution, as known as Function Calling, is based on API calling, like weather API, Google API, etc. It has two main problems in complex scenarios:
- Multiple APIs increase the cognitive load for AI Agents, potentially leading to reasoning errors, especially when the APIs have a strict calling order.
- Directly calling APIs is a way with too much freedom degrees, lacking constraints, potentially resulting in data errors.
The way Agents interact with APIs reminds me of how human interacts with computers in old days. A guy faces a black and thick screen and types the keyboard, while looking up commands in a manual. Not just Agent, we also faces these two problems.
Then a technique that surprised Steve Jobs comes up: Graphical User Interface (GUI).
Since then, most of us no longer directly interact with command lines (a kind of API). We interact with API through GUI. GUI generally solved these two problems by constraining interaction to reduce cognitive load and degrees of freedom.
This is why Agents also need GUI-like tools. I prefer calling it Agentic User Interface (AUI).
pip install actefrom acte import Component, v
from acte.chatbot import OpenaiChatbot
from acte.server import Server
from acte.session import SessionManager
class HelloWorld(Component):
def view(self) -> None:
v.text("Hello World")
server = Server(
session_manager=SessionManager(HelloWorld, debug=True),
chatbot=OpenaiChatbot( # default model is gpt-4o
api_key="YOUR OPENAI API KEY",
)
)
if __name__ == '__main__':
server.run()-
Copy the code to a Python file, and set OpenAI Api Key.
-
Run the code, and visit the playground: http://127.0.0.1:8000.
-
Input "Hi, please new a session, and tell what you see." Then, the Agent will interact with Screen, and give you a response.
Note: You can also interact with Screen to debug your app.
from acte import Component, v
from acte.chatbot import OpenaiChatbot
from acte.server import Server
from acte.session import SessionManager
from acte.state.signal import Signal
class Counter(Component):
def __init__(self) -> None:
self._n = Signal(0)
def view(self) -> None:
v.text("This is a counter.")
with v.div():
v.text(lambda: f"Current Value: {self._n.value}")
# _n is Signal, so you need to use self._n.value in lambda
v.button("add", on_press=self._add)
async def _add(self) -> None:
await self._n.set(self._n.value + 1)
server = Server(
session_manager=SessionManager(Counter, debug=True),
chatbot=OpenaiChatbot(
api_key="YOUR OPENAI API KEY",
)
)
if __name__ == '__main__':
server.run()-
Agent can press the button in Screen. The number on the right-top of the button is Interactive ID. Agent interacts with Screen by pointing to ID.
-
Interaction's result will show in Screen. You can click "View"s in Dialog to check the Screen status in each step.
from typing import Callable, Awaitable
from acte.chatbot import OpenaiChatbot
from acte.schema import IntSchema
from acte.server import Server
from acte.session import SessionManager
from acte import Component, v, Prop, as_prop
from acte.state.signal import Signal
class MenuItem(Component):
def __init__(
self,
name: Prop[str], # Prop = Ref[T] | T
price: Prop[float],
quantity: Prop[int],
on_quantity_change: Callable[[str, float, int], Awaitable[None]]
) -> None:
self._name = as_prop(name) # as_prop is to convert T to Ref[T]
self._price = as_prop(price)
self._quantity = as_prop(quantity)
self._on_quantity_change = on_quantity_change
def view(self) -> None:
with v.div():
v.text(lambda: f"{self._name.value}: ${self._price.value}")
v.input("quantity", self._quantity, self._on_set, schema=IntSchema())
async def _on_set(self, value: str) -> None:
await self._on_quantity_change(
self._name.value,
self._price.value,
0 if value == '' else int(value)
)
class Menu(Component):
def __init__(self) -> None:
self._menu = {
"Pizza": {"price": 10.0, "quantity": Signal(0)}, # Signal is a kind of Ref, but can be set
"Coke": {"price": 2.0, "quantity": Signal(0)},
}
def view(self) -> None:
v.text("Super Restaurant Menu")
for key, value in self._menu.items():
v.component(
MenuItem(
name=key,
price=value['price'],
quantity=value['quantity'],
on_quantity_change=self._on_quantity_change,
)
)
v.text(self.total)
v.button("checkout", on_press=self._checkout)
def total(self) -> str:
total = 0
for key, value in self._menu.items():
total += value['price'] * value['quantity'].value
return f"Total: ${total}"
async def _on_quantity_change(self, name: str, price: float, quantity: int) -> None:
await self._menu[name]['quantity'].set(quantity)
async def _checkout(self) -> None:
total = self.total()
for value in self._menu.values():
await value['quantity'].set(0)
print(f"Checkout: {total}")
server = Server(
session_manager=SessionManager(Menu, debug=True),
chatbot=OpenaiChatbot(
system_message="You are restaurant assistant to help customers to order food through App. "
"You should confirm before Checkout",
api_key="YOUR OPENAI API KEY",
)
)
if __name__ == '__main__':
server.run()-
Agent can fill input fields in Screen.
-
Input fields also have their own Interactive ID. Agent can take multiple actions in one calling.
-
You can define the backend logic after Agent press the button, such as make a request.
Acte Tool has 3 APIs: new_session, execute, and display,
which can be accessed by HTTP request or SessionManager
Start a new App session, then display the session's latest screen.
POST /sessionfrom acte.session import SessionManager
sm = SessionManager(...)
sm.new_session(){
"session_id": str,
"screen": str,
}
Execute one or more action(s), then display the session's latest screen.
POST /execute
json:
{
"session_id": "str",
"actions": [
{
"target_id": "str",
"action_type": "str",
"value": "str | None",
},
]
}from acte.session import SessionManager
sm = SessionManager(...)
sm.execute(
{
"session_id": str,
"actions": [
{
"target_id": str, # interactive id
"action_type": str, # one of ["press", "fill"]
"value": str | None, # required when action_type is "fill"
},
...
]
}
){
"session_id": str,
"screen": str,
}Display the session's latest screen.
POST /display
json:
{
"session_id": "str",
}from acte.session import SessionManager
sm = SessionManager(...)
sm.execute(
{
"session_id": str,
}
){
"session_id": str,
"screen": str,
}-
[ ] Full Document
-
[ ] Test Code
Note: The project is in Alpha stage. The API may change frequently.
The project is licensed under the terms of the MIT license.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for acte
Similar Open Source Tools
acte
Acte is a framework designed to build GUI-like tools for AI Agents. It aims to address the issues of cognitive load and freedom degrees when interacting with multiple APIs in complex scenarios. By providing a graphical user interface (GUI) for Agents, Acte helps reduce cognitive load and constraints interaction, similar to how humans interact with computers through GUIs. The tool offers APIs for starting new sessions, executing actions, and displaying screens, accessible via HTTP requests or the SessionManager class.
instructor
Instructor is a tool that provides structured outputs from Large Language Models (LLMs) in a reliable manner. It simplifies the process of extracting structured data by utilizing Pydantic for validation, type safety, and IDE support. With Instructor, users can define models and easily obtain structured data without the need for complex JSON parsing, error handling, or retries. The tool supports automatic retries, streaming support, and extraction of nested objects, making it production-ready for various AI applications. Trusted by a large community of developers and companies, Instructor is used by teams at OpenAI, Google, Microsoft, AWS, and YC startups.
pocketgroq
PocketGroq is a tool that provides advanced functionalities for text generation, web scraping, web search, and AI response evaluation. It includes features like an Autonomous Agent for answering questions, web crawling and scraping capabilities, enhanced web search functionality, and flexible integration with Ollama server. Users can customize the agent's behavior, evaluate responses using AI, and utilize various methods for text generation, conversation management, and Chain of Thought reasoning. The tool offers comprehensive methods for different tasks, such as initializing RAG, error handling, and tool management. PocketGroq is designed to enhance development processes and enable the creation of AI-powered applications with ease.
llm-sandbox
LLM Sandbox is a lightweight and portable sandbox environment designed to securely execute large language model (LLM) generated code in a safe and isolated manner using Docker containers. It provides an easy-to-use interface for setting up, managing, and executing code in a controlled Docker environment, simplifying the process of running code generated by LLMs. The tool supports multiple programming languages, offers flexibility with predefined Docker images or custom Dockerfiles, and allows scalability with support for Kubernetes and remote Docker hosts.
aiotdlib
aiotdlib is a Python asyncio Telegram client based on TDLib. It provides automatic generation of types and functions from tl schema, validation, good IDE type hinting, and high-level API methods for simpler work with tdlib. The package includes prebuilt TDLib binaries for macOS (arm64) and Debian Bullseye (amd64). Users can use their own binary by passing `library_path` argument to `Client` class constructor. Compatibility with other versions of the library is not guaranteed. The tool requires Python 3.9+ and users need to get their `api_id` and `api_hash` from Telegram docs for installation and usage.
instructor
Instructor is a Python library that makes it a breeze to work with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API to manage validation, retries, and streaming responses. Get ready to supercharge your LLM workflows!
aio-pika
Aio-pika is a wrapper around aiormq for asyncio and humans. It provides a completely asynchronous API, object-oriented API, transparent auto-reconnects with complete state recovery, Python 3.7+ compatibility, transparent publisher confirms support, transactions support, and complete type-hints coverage.
UnrealGenAISupport
The Unreal Engine Generative AI Support Plugin is a tool designed to integrate various cutting-edge LLM/GenAI models into Unreal Engine for game development. It aims to simplify the process of using AI models for game development tasks, such as controlling scene objects, generating blueprints, running Python scripts, and more. The plugin currently supports models from organizations like OpenAI, Anthropic, XAI, Google Gemini, Meta AI, Deepseek, and Baidu. It provides features like API support, model control, generative AI capabilities, UI generation, project file management, and more. The plugin is still under development but offers a promising solution for integrating AI models into game development workflows.
LocalAGI
LocalAGI is a powerful, self-hostable AI Agent platform that allows you to design AI automations without writing code. It provides a complete drop-in replacement for OpenAI's Responses APIs with advanced agentic capabilities. With LocalAGI, you can create customizable AI assistants, automations, chat bots, and agents that run 100% locally, without the need for cloud services or API keys. The platform offers features like no-code agents, web-based interface, advanced agent teaming, connectors for various platforms, comprehensive REST API, short & long-term memory capabilities, planning & reasoning, periodic tasks scheduling, memory management, multimodal support, extensible custom actions, fully customizable models, observability, and more.
ai00_server
AI00 RWKV Server is an inference API server for the RWKV language model based upon the web-rwkv inference engine. It supports VULKAN parallel and concurrent batched inference and can run on all GPUs that support VULKAN. No need for Nvidia cards!!! AMD cards and even integrated graphics can be accelerated!!! No need for bulky pytorch, CUDA and other runtime environments, it's compact and ready to use out of the box! Compatible with OpenAI's ChatGPT API interface. 100% open source and commercially usable, under the MIT license. If you are looking for a fast, efficient, and easy-to-use LLM API server, then AI00 RWKV Server is your best choice. It can be used for various tasks, including chatbots, text generation, translation, and Q&A.
modelfusion
ModelFusion is an abstraction layer for integrating AI models into JavaScript and TypeScript applications, unifying the API for common operations such as text streaming, object generation, and tool usage. It provides features to support production environments, including observability hooks, logging, and automatic retries. You can use ModelFusion to build AI applications, chatbots, and agents. ModelFusion is a non-commercial open source project that is community-driven. You can use it with any supported provider. ModelFusion supports a wide range of models including text generation, image generation, vision, text-to-speech, speech-to-text, and embedding models. ModelFusion infers TypeScript types wherever possible and validates model responses. ModelFusion provides an observer framework and logging support. ModelFusion ensures seamless operation through automatic retries, throttling, and error handling mechanisms. ModelFusion is fully tree-shakeable, can be used in serverless environments, and only uses a minimal set of dependencies.
freeGPT
freeGPT provides free access to text and image generation models. It supports various models, including gpt3, gpt4, alpaca_7b, falcon_40b, prodia, and pollinations. The tool offers both asynchronous and non-asynchronous interfaces for text completion and image generation. It also features an interactive Discord bot that provides access to all the models in the repository. The tool is easy to use and can be integrated into various applications.
Acontext
Acontext is a context data platform designed for production AI agents, offering unified storage, built-in context management, and observability features. It helps agents scale from local demos to production without the need to rebuild context infrastructure. The platform provides solutions for challenges like scattered context data, long-running agents requiring context management, and tracking states from multi-modal agents. Acontext offers core features such as context storage, session management, disk storage, agent skills management, and sandbox for code execution and analysis. Users can connect to Acontext, install SDKs, initialize clients, store and retrieve messages, perform context engineering, and utilize agent storage tools. The platform also supports building agents using end-to-end scripts in Python and Typescript, with various templates available. Acontext's architecture includes client layer, backend with API and core components, infrastructure with PostgreSQL, S3, Redis, and RabbitMQ, and a web dashboard. Join the Acontext community on Discord and follow updates on GitHub.
connectonion
ConnectOnion is a simple, elegant open-source framework for production-ready AI agents. It provides a platform for creating and using AI agents with a focus on simplicity and efficiency. The framework allows users to easily add tools, debug agents, make them production-ready, and enable multi-agent capabilities. ConnectOnion offers a simple API, is production-ready with battle-tested models, and is open-source under the MIT license. It features a plugin system for adding reflection and reasoning capabilities, interactive debugging for easy troubleshooting, and no boilerplate code for seamless scaling from prototypes to production systems.
OpenMemory
OpenMemory is a cognitive memory engine for AI agents, providing real long-term memory capabilities beyond simple embeddings. It is self-hosted and supports Python + Node SDKs, with integrations for various tools like LangChain, CrewAI, AutoGen, and more. Users can ingest data from sources like GitHub, Notion, Google Drive, and others directly into memory. OpenMemory offers explainable traces for recalled information and supports multi-sector memory, temporal reasoning, decay engine, waypoint graph, and more. It aims to provide a true memory system rather than just a vector database with marketing copy, enabling users to build agents, copilots, journaling systems, and coding assistants that can remember and reason effectively.
lagent
Lagent is a lightweight open-source framework that allows users to efficiently build large language model(LLM)-based agents. It also provides some typical tools to augment LLM. The overview of our framework is shown below:
For similar tasks
acte
Acte is a framework designed to build GUI-like tools for AI Agents. It aims to address the issues of cognitive load and freedom degrees when interacting with multiple APIs in complex scenarios. By providing a graphical user interface (GUI) for Agents, Acte helps reduce cognitive load and constraints interaction, similar to how humans interact with computers through GUIs. The tool offers APIs for starting new sessions, executing actions, and displaying screens, accessible via HTTP requests or the SessionManager class.
pyloid
Pyloid is a Python backend version of Electron and Tauri, simplifying desktop application development. Built on QtWebEngine and PySide6, it offers seamless integration with Python features, enabling easy creation of powerful applications. It provides web-based GUI generation, system tray icon support, multi-window management, bridge API between Python and JavaScript, single/multi-instance application support, comprehensive desktop app features, clean code structure, live UI development experience, cross-platform support, integration with frontend libraries, window customization, direct utilization of PySide6 features, and detailed Numpy-style docstrings.
For similar jobs
promptflow
**Prompt flow** is a suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications, from ideation, prototyping, testing, evaluation to production deployment and monitoring. It makes prompt engineering much easier and enables you to build LLM apps with production quality.
deepeval
DeepEval is a simple-to-use, open-source LLM evaluation framework specialized for unit testing LLM outputs. It incorporates various metrics such as G-Eval, hallucination, answer relevancy, RAGAS, etc., and runs locally on your machine for evaluation. It provides a wide range of ready-to-use evaluation metrics, allows for creating custom metrics, integrates with any CI/CD environment, and enables benchmarking LLMs on popular benchmarks. DeepEval is designed for evaluating RAG and fine-tuning applications, helping users optimize hyperparameters, prevent prompt drifting, and transition from OpenAI to hosting their own Llama2 with confidence.
MegaDetector
MegaDetector is an AI model that identifies animals, people, and vehicles in camera trap images (which also makes it useful for eliminating blank images). This model is trained on several million images from a variety of ecosystems. MegaDetector is just one of many tools that aims to make conservation biologists more efficient with AI. If you want to learn about other ways to use AI to accelerate camera trap workflows, check out our of the field, affectionately titled "Everything I know about machine learning and camera traps".
leapfrogai
LeapfrogAI is a self-hosted AI platform designed to be deployed in air-gapped resource-constrained environments. It brings sophisticated AI solutions to these environments by hosting all the necessary components of an AI stack, including vector databases, model backends, API, and UI. LeapfrogAI's API closely matches that of OpenAI, allowing tools built for OpenAI/ChatGPT to function seamlessly with a LeapfrogAI backend. It provides several backends for various use cases, including llama-cpp-python, whisper, text-embeddings, and vllm. LeapfrogAI leverages Chainguard's apko to harden base python images, ensuring the latest supported Python versions are used by the other components of the stack. The LeapfrogAI SDK provides a standard set of protobuffs and python utilities for implementing backends and gRPC. LeapfrogAI offers UI options for common use-cases like chat, summarization, and transcription. It can be deployed and run locally via UDS and Kubernetes, built out using Zarf packages. LeapfrogAI is supported by a community of users and contributors, including Defense Unicorns, Beast Code, Chainguard, Exovera, Hypergiant, Pulze, SOSi, United States Navy, United States Air Force, and United States Space Force.
llava-docker
This Docker image for LLaVA (Large Language and Vision Assistant) provides a convenient way to run LLaVA locally or on RunPod. LLaVA is a powerful AI tool that combines natural language processing and computer vision capabilities. With this Docker image, you can easily access LLaVA's functionalities for various tasks, including image captioning, visual question answering, text summarization, and more. The image comes pre-installed with LLaVA v1.2.0, Torch 2.1.2, xformers 0.0.23.post1, and other necessary dependencies. You can customize the model used by setting the MODEL environment variable. The image also includes a Jupyter Lab environment for interactive development and exploration. Overall, this Docker image offers a comprehensive and user-friendly platform for leveraging LLaVA's capabilities.
carrot
The 'carrot' repository on GitHub provides a list of free and user-friendly ChatGPT mirror sites for easy access. The repository includes sponsored sites offering various GPT models and services. Users can find and share sites, report errors, and access stable and recommended sites for ChatGPT usage. The repository also includes a detailed list of ChatGPT sites, their features, and accessibility options, making it a valuable resource for ChatGPT users seeking free and unlimited GPT services.
TrustLLM
TrustLLM is a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. We then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. The document explains how to use the trustllm python package to help you assess the performance of your LLM in trustworthiness more quickly. For more details about TrustLLM, please refer to project website.
AI-YinMei
AI-YinMei is an AI virtual anchor Vtuber development tool (N card version). It supports fastgpt knowledge base chat dialogue, a complete set of solutions for LLM large language models: [fastgpt] + [one-api] + [Xinference], supports docking bilibili live broadcast barrage reply and entering live broadcast welcome speech, supports Microsoft edge-tts speech synthesis, supports Bert-VITS2 speech synthesis, supports GPT-SoVITS speech synthesis, supports expression control Vtuber Studio, supports painting stable-diffusion-webui output OBS live broadcast room, supports painting picture pornography public-NSFW-y-distinguish, supports search and image search service duckduckgo (requires magic Internet access), supports image search service Baidu image search (no magic Internet access), supports AI reply chat box [html plug-in], supports AI singing Auto-Convert-Music, supports playlist [html plug-in], supports dancing function, supports expression video playback, supports head touching action, supports gift smashing action, supports singing automatic start dancing function, chat and singing automatic cycle swing action, supports multi scene switching, background music switching, day and night automatic switching scene, supports open singing and painting, let AI automatically judge the content.