ailoy

ailoy

Ailoy is a developer-friendly library that simplifies building and deploying AI agents and LLM-based applications.

Stars: 107

Visit
 screenshot

Ailoy is a lightweight library for building AI applications such as agent systems or RAG pipelines with ease. It enables AI features effortlessly, supporting AI models locally or via cloud APIs, multi-turn conversation, system message customization, reasoning-based workflows, tool calling capabilities, and built-in vector store support. It also supports running native-equivalent functionality in web browsers using WASM. The library is in early development stages and provides examples in the `examples` directory for inspiration on building applications with Agents.

README:

Ailoy

document Python version Node version

ailoy-py version ailoy-node version ailoy-web version

Discord X (formerly Twitter) URL

Ailoy is a lightweight library for building AI applications — such as agent systems or RAG pipelines — with ease. It is designed to enable AI features effortlessly, one can just import and use.

[!WARNING] This library is in an early stage of development. APIs may change without notice.

[!TIP] We have a Discord channel! If you get stuck or have any questions, feel free to join and ask.

[!TIP] There are several interesting examples in the examples directory — take a look!

You might get inspired what kind of applications you can build with Agents.

Features

  • Run AI models either in local or via cloud APIs
  • Multi-turn conversation and system message customization
  • Support for reasoning-based workflows
  • Tool calling capabilities (including MCP integration)
  • Built-in vector store support (via Faiss or ChromaDB)
  • Supports running native-equivalent functionality in the web browsers(WASM)

For more details, please refer to the documentation.

Supported AI models

Currently, the following AI models are supported:

  • Language Models
    • Local Models
      • Qwen3

        Click to see details

        • Qwen/Qwen3-0.6B
        • Qwen/Qwen3-1.7B
        • Qwen/Qwen3-4B
        • Qwen/Qwen3-8B
        • Qwen/Qwen3-14B
        • Qwen/Qwen3-32B
        • Qwen/Qwen3-30B-A3B (MoE)
    • API Models
      • OpenAI
      • Gemini
      • Claude
      • Grok
  • Embedding Models
    • Local Models
      • BAAI/bge-m3

You can check out examples for a simple chatbot, tool usages and retrieval-augmented generation (RAG).

Requirements

For Agents with LLM APIs

You can create your own agent with Ailoy using external APIs such as OpenAI, Claude, Gemini or Grok.
Your system doesn't need to meet the hardware requirements with these APIs.

  • image Windows (x86_64)
  • image macOS (Apple Silicon)
  • image Linux (x86_64)
  • 🌐 Web Browsers

For Agents with AI running locally

To use Ailoy with local models, a compatible device is required.
If you can use a compatible device,

Ailoy supports Local AI executions on the following environments(details are below.):

  • image Windows (x86_64, with image Vulkan)
  • image macOS (Apple Silicon, with image Metal)
  • image Linux (x86_64, with image Vulkan)
  • 🌐 Modern Web Browsers (wasm32, with image WebGPU)
    • Chrome browsers for PC/Mac is recommended.
    • Supports up to Qwen3-8B

AI models typically consume a significant amount of memory.
The exact usage depends on the model size, but we recommend at least 8GB of GPU memory.
Running the Qwen 8B model requires at least 12GB of GPU memory.
On macOS, this refers to unified memory, as Apple Silicon uses a shared memory architecture.

Local AI requirement details

Windows

  • CPU: Intel Skylake or newer (and compatible AMD), x86_64 is required
  • GPU: At least 8GB of VRAM and support for Vulkan 1.3
    • NVIDIA/AMD/Intel graphic driver that supports Vulkan 1.3 or higher
  • OS: Windows 11 or Windows Server 2022 (earlier versions may work but are not officially tested)

macOS

  • Device: Apple Silicon with Metal support
  • Memory: At least 8GB of unified memory
  • OS: macOS 14 or newer

Linux

  • CPU: Intel Skylake or newer (and compatible AMD), x86_64 is required
  • GPU: At least 8GB of VRAM and support for Vulkan 1.3
    • NVIDIA/AMD/Intel graphic driver that supports Vulkan 1.3 or higher
  • OS: Debian 10 / Ubuntu 21.04 or newer (this means, os with glibc 2.28 or higher)

Web browser

  • Browser: Modern browsers with WebGPU support
  • GPU: WebGPU with "shader-f16" support
    • It may not work on mobile devices with Qualcomm APs.

Getting Started

Python

pip install ailoy-py
from ailoy import Runtime, Agent, LocalModel

rt = Runtime()
with Agent(rt, LocalModel("Qwen/Qwen3-0.6B")) as agent:
    for resp in agent.query("Hello world!"):
        resp.print()
rt.stop()

Node

npm install ailoy-node
import { startRuntime, defineAgent, LocalModel } from "ailoy-node";

(async () => {
  const rt = await startRuntime();
  const agent = await defineAgent(rt, LocalModel({ id: "Qwen/Qwen3-0.6B" }));
  for await (const resp of agent.query("Hello world!")) {
    agent.print(resp);
  }
  await agent.delete();
  await rt.stop();
})();

Web browser

npm install ailoy-web
import * as ai from "ailoy-web";

const rt = await ai.startRuntime();
const agent = await ai.defineAgent(
  rt,
  ai.LocalModel({ id: "Qwen/Qwen3-0.6B" })
);

(async () => {
  // Assume that `textarea` with id "answer" exists
  const textarea = document.getElementById("answer");

  for await (const resp of agent.query("Hello, world!")) {
    textarea.innerHTML += resp.content;
  }
})();

Build from source

Python Build

cd bindings/python
pip install -e .

For more details, refer to bindings/python/README.md.

Node.js Build

cd bindings/js-node
npm run build

For more details, refer to bindings/js-node/README.md.

Javascript Build for web browsers

cd bindings/js-web
npm run build

For more details, refer to bindings/js-web/README.md.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for ailoy

Similar Open Source Tools

For similar tasks

For similar jobs