island-ai
None
Stars: 134
island-ai is a TypeScript toolkit tailored for developers engaging with structured outputs from Large Language Models. It offers streamlined processes for handling, parsing, streaming, and leveraging AI-generated data across various applications. The toolkit includes packages like zod-stream for interfacing with LLM streams, stream-hooks for integrating streaming JSON data into React applications, and schema-stream for JSON streaming parsing based on Zod schemas. Additionally, related packages like @instructor-ai/instructor-js focus on data validation and retry mechanisms, enhancing the reliability of data processing workflows.
README:
> A TypeScript toolkit for building structured LLM data handling pipelines
Island AI is a collection of low-level utilities and high-level tools for handling structured data streams from LLMs. The packages range from basic JSON streaming parsers to complete LLM clients, giving you the flexibility to build custom solutions or use pre-built integrations.
A foundational streaming JSON parser that enables immediate data access through structured stubs.
Key Features:
- Streaming JSON parser with typed outputs
- Default value support
- Path completion tracking
- Nested object and array support
import { SchemaStream } from "schema-stream";
import { z } from "zod";
// Define complex nested schemas
const schema = z.object({
layer1: z.object({
layer2: z.object({
value: z.string(),
layer3: z.object({
layer4: z.object({
layer5: z.string()
})
})
})
}),
someArray: z.array(z.object({
someString: z.string(),
someNumber: z.number()
}))
});
// Get a readable stream of json (from an api or otherwise)
async function getSomeStreamOfJson(
jsonString: string
): Promise<{ body: ReadableStream }> {
const stream = new ReadableStream({
start(controller) {
const encoder = new TextEncoder()
const jsonBytes = encoder.encode(jsonString)
for (let i = 0; i < jsonBytes.length; ) {
const chunkSize = Math.floor(Math.random() * 5) + 2
const chunk = jsonBytes.slice(i, i + chunkSize)
controller.enqueue(chunk)
i += chunkSize
}
controller.close()
},
})
return { body: stream }
}
// Create parser with completion tracking
const parser = new SchemaStream(schema, {
onKeyComplete({ completedPaths }) {
console.log('Completed paths:', completedPaths);
}
});
// Get the readabale stream to parse
const readableStream = await getSomeStreamOfJson(
`{"someString": "Hello schema-stream", "someNumber": 42000000}`
)
// Parse streaming data
const stream = parser.parse();
readableStream.pipeThrough(stream);
// Get typed results
const reader = stream.readable.getReader();
const decoder = new TextDecoder()
let result = {}
let complete = false
while (true) {
const { value, done } = await reader.read();
complete = done
if (complete) break;
result = JSON.parse(decoder.decode(value));
// result is fully typed based on schema
}Extends schema-stream with OpenAI integration and Zod-specific features.
Key Features:
- OpenAI completion streaming
- Multiple response modes (TOOLS, FUNCTIONS, JSON, etc.)
- Schema validation during streaming
import { OAIStream } from "zod-stream";
import { withResponseModel } from "zod-stream";
import { z } from "zod";
// Define extraction schema
const ExtractionSchema = z.object({
users: z.array(z.object({
name: z.string(),
handle: z.string(),
twitter: z.string()
})).min(3),
location: z.string(),
budget: z.number()
});
// Configure OpenAI params with schema
const params = withResponseModel({
response_model: {
schema: ExtractionSchema,
name: "Extract"
},
params: {
messages: [{ role: "user", content: textBlock }],
model: "gpt-4"
},
mode: "TOOLS"
});
// Stream completions
const stream = OAIStream({
res: await oai.chat.completions.create({
...params,
stream: true
})
});
// Process results
const client = new ZodStream();
const extractionStream = await client.create({
completionPromise: () => stream,
response_model: {
schema: ExtractionSchema,
name: "Extract"
}
});
for await (const data of extractionStream) {
console.log('Progressive update:', data);
}React hooks for consuming streaming JSON data with Zod schema validation.
Key Features:
- Ready-to-use React hooks
- Automatic schema validation
- Progress tracking
- Error handling
import { useJsonStream } from "stream-hooks";
function DataViewer() {
const { loading, startStream, data, error } = useJsonStream({
schema: ExtractionSchema,
onReceive: (update) => {
console.log('Progressive update:', update);
},
});
return (
<div>
{loading && <div>Loading...</div>}
{error && <div>Error: {error.message}</div>}
{data && (
<pre>{JSON.stringify(data, null, 2)}</pre>
)}
<button onClick={() => startStream({
url: "/api/extract",
method: "POST",
body: { text: "..." }
})}>
Start Extraction
</button>
</div>
);
}Structured evaluation tools for assessing LLM outputs across multiple dimensions. Built with TypeScript and integrated with OpenAI and Instructor, it enables both automated evaluation and human-in-the-loop assessment workflows.
Key Features:
- 🎯 Model-Graded Evaluation: Leverage LLMs to assess response quality
- 📊 Accuracy Measurement: Compare outputs using semantic and lexical similarity
- 🔍 Context Validation: Evaluate responses against source materials
- ⚖️ Composite Assessment: Combine multiple evaluation types with custom weights
// Combine different evaluator types
const compositeEval = createWeightedEvaluator({
evaluators: {
entities: createContextEvaluator({ type: "entities-recall" }),
accuracy: createAccuracyEvaluator({
weights: {
factual: 0.9, // High weight on exact matches
semantic: 0.1 // Low weight on similar terms
}
}),
quality: createEvaluator({
client: oai,
model: "gpt-4-turbo",
evaluationDescription: "Rate quality"
})
},
weights: {
entities: 0.3,
accuracy: 0.4,
quality: 0.3
}
});
// Must provide all required fields for each evaluator type
await compositeEval({
data: [{
prompt: "Summarize the earnings call",
completion: "CEO Jane Smith announced 15% growth",
expectedCompletion: "The CEO reported strong growth",
groundTruth: "CEO discussed Q3 performance",
contexts: [
"CEO Jane Smith presented Q3 results",
"Company saw 15% growth in Q3 2023"
]
}]
});A universal LLM client that extends the OpenAI SDK to provide consistent interfaces across different providers that may not follow the OpenAI API specification.
Native API Support Status:
| Provider API | Status | Chat | Basic Stream | Functions/Tool calling | Function streaming | Notes |
|---|---|---|---|---|---|---|
| OpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | Direct SDK proxy |
| Anthropic | ✅ | ✅ | ✅ | ❌ | ❌ | Claude models |
| ✅ | ✅ | ✅ | ✅ | ❌ | Gemini models + context caching | |
| Azure | 🚧 | ✅ | ✅ | ❌ | ❌ | OpenAI model hosting |
| Cohere | ❌ | - | - | - | - | Not supported |
| AI21 | ❌ | - | - | - | - | Not supported |
Stream Types:
- Basic Stream: Simple text streaming
- Partial JSON Stream: Progressive JSON object construction during streaming
- Function Stream: Streaming function/tool calls and their results
OpenAI-Compatible Hosting Providers:
These providers use the OpenAI SDK format, so they work directly with the OpenAI client configuration:
| Provider | How to Use | Available Models |
|---|---|---|
| Together | Use OpenAI client with Together base URL | Mixtral, Llama, OpenChat, Yi, others |
| Anyscale | Use OpenAI client with Anyscale base URL | Mistral, Llama, others |
| Perplexity | Use OpenAI client with Perplexity base URL | pplx-* models |
| Replicate | Use OpenAI client with Replicate base URL | Various open models |
Key Features:
- OpenAI-compatible interface for non-OpenAI providers
- Support for major providers:
- OpenAI (direct SDK proxy)
- Anthropic (Claude models)
- Google (Gemini models)
- Together
- Microsoft/Azure
- Anyscale
- Streaming support across providers
- Function/tool calling compatibility
- Context caching for Gemini
- Structured output support
import { createLLMClient } from "llm-polyglot";
// Create provider-specific client
const anthropicClient = createLLMClient({
provider: "anthropic"
});
// Use consistent OpenAI-style API
const completion = await anthropicClient.chat.completions.create({
model: "claude-3-opus-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Extract data..." }]
});// Anthropic Streaming
const stream = await anthropicClient.chat.completions.create({
model: "claude-3-opus-20240229",
max_tokens: 1000,
stream: true,
messages: [{ role: "user", content: "Stream some content..." }]
});
let content = "";
for await (const chunk of stream) {
content += chunk.choices?.[0]?.delta?.content ?? "";
}
// Google/Gemini with Context Caching
const googleClient = createLLMClient({
provider: "google"
});
// Create a context cache
const cache = await googleClient.cacheManager.create({
model: "gemini-1.5-flash-8b",
messages: [{
role: "user",
content: "What is the capital of Montana?"
}],
ttlSeconds: 3600, // Cache for 1 hour
max_tokens: 1000
});
// Use cached context in new completion
const completion = await googleClient.chat.completions.create({
model: "gemini-1.5-flash-8b",
messages: [{
role: "user",
content: "What state is it in?"
}],
additionalProperties: {
cacheName: cache.name
},
max_tokens: 1000
});const completion = await anthropicClient.chat.completions.create({
model: "claude-3-opus-20240229",
max_tokens: 1000,
messages: [{
role: "user",
content: "Extract user information..."
}],
tool_choice: {
type: "function",
function: { name: "extract_user" }
},
tools: [{
type: "function",
function: {
name: "extract_user",
description: "Extract user information",
parameters: {
type: "object",
properties: {
name: { type: "string" },
age: { type: "number" }
},
required: ["name", "age"]
}
}
}]
});// Using with OpenAI-compatible providers:
const client = createLLMClient({
provider: "openai",
apiKey: "your_api_key",
baseURL: "https://api.together.xyz/v1", // or other provider URLs
});-
Anthropic (Claude)
- Full function/tool calling support
- Message streaming
- OpenAI-compatible responses
-
Google (Gemini)
- Context caching for token optimization
- Streaming support
- Function calling
- Optional OpenAI compatibility mode
- Grounding (i.e. Google Search) support
-
OpenAI
- Direct SDK proxy
- All native OpenAI features supported
The core Island AI packages provides more low-level utilities for building custom LLM clients and data handling pipelines (schema-stream, zod-stream, stream-hooks). For a complete, ready-to-use solution, check out Instructor, which composes some of these tools into a full-featured client.
When to use core packages:
- You need direct access to the HTTP stream for custom transport (e.g., not using SSE/WebSockets)
- You want to build a custom LLM client
- You need fine-grained control over streaming and parsing
- You're implementing server-side streaming with client-side parsing
- You need a structured evaluation tool
- You want to use different LLM providers that don't support the OpenAI SDK format
When to use Instructor:
- You want a complete solution for structured extraction
- You're using WebSocket-based streaming from server to client
- You're requests are only on the server
- You need the full async generator pattern for progressive object updates
- You want OpenAI SDK compatibility out of the box
For cases where you need direct control over the HTTP stream, you can use the core packages to build your own streaming endpoints:
import { OAIStream } from "zod-stream";
import { withResponseModel } from "zod-stream";
import OpenAI from "openai";
import { z } from "zod";
const oai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
organization: process.env.OPENAI_ORG_ID
});
// Define your schema
const schema = z.object({
content: z.string(),
users: z.array(z.object({
name: z.string(),
})),
});
// API Route Example (Next.js)
export async function POST(request: Request) {
const { messages } = await request.json();
// Configure OpenAI parameters with schema
const params = withResponseModel({
response_model: {
schema: schema,
name: "Users extraction and message"
},
params: {
messages,
model: "gpt-4",
},
mode: "TOOLS",
});
// Create streaming completion
const extractionStream = await oai.chat.completions.create({
...params,
stream: true,
});
// Return streaming response
return new Response(
OAIStream({ res: extractionStream })
);
}
// Client-side consumption
async function consumeStream() {
const response = await fetch('/api/extract', {
method: 'POST',
body: JSON.stringify({
messages: [{ role: 'user', content: 'Your prompt here' }]
})
});
const parser = new SchemaStream(schema);
const stream = parser.parse();
response.body
?.pipeThrough(stream)
.pipeTo(new WritableStream({
write(chunk) {
const data = JSON.parse(new TextDecoder().decode(chunk));
// Use partial data as it arrives
console.log('Partial data:', data);
}
}));
}
Instructor provides a high-level client that composes Island AI's core packages into a complete solution for structured extraction. It extends the OpenAI client with streaming and schema validation capabilities.
import Instructor from "@instructor-ai/instructor";
import OpenAI from "openai";
import { z } from "zod";
// Define your extraction schema
const ExtractionSchema = z.object({
users: z.array(
z.object({
name: z.string(),
handle: z.string(),
twitter: z.string()
})
).min(3),
location: z.string(),
budget: z.number()
});
type Extraction = Partial<z.infer<typeof ExtractionSchema>>;
// Initialize OpenAI client with Instructor
const oai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
organization: process.env.OPENAI_ORG_ID
});
const client = Instructor({
client: oai,
mode: "TOOLS"
});
// Stream completions with structured output
const extractionStream = await client.chat.completions.create({
messages: [{
role: "user",
content: "Your text content here..."
}],
model: "gpt-4",
response_model: {
schema: ExtractionSchema,
name: "Extract"
},
max_retries: 3,
stream: true,
stream_options: {
include_usage: true
}
});
// Process streaming results
let extraction: Extraction = {};
for await (const result of extractionStream) {
extraction = result;
console.log('Progressive update:', result);
}
console.log('Final extraction:', extraction);-
Instructor
- Provides a complete solution built on top of the OpenAI SDK
- Handles retries, validation, and streaming automatically
- Returns an async generator for progressive updates
- Ideal for WebSocket-based streaming from server to client
- Simpler integration when you don't need low-level control
-
Direct HTTP Streaming
- Gives you direct access to the HTTP stream
- Allows custom transport mechanisms
- Enables server-side streaming with client-side parsing
- More flexible for custom implementations
- Better for scenarios where you need to minimize server processing
We welcome contributions! Check out our issues labeled as good-first-issue or help-wanted.
For detailed documentation, visit https://island.hack.dance
MIT License - see LICENSE file for details.
For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for island-ai
Similar Open Source Tools
island-ai
island-ai is a TypeScript toolkit tailored for developers engaging with structured outputs from Large Language Models. It offers streamlined processes for handling, parsing, streaming, and leveraging AI-generated data across various applications. The toolkit includes packages like zod-stream for interfacing with LLM streams, stream-hooks for integrating streaming JSON data into React applications, and schema-stream for JSON streaming parsing based on Zod schemas. Additionally, related packages like @instructor-ai/instructor-js focus on data validation and retry mechanisms, enhancing the reliability of data processing workflows.
freeGPT
freeGPT provides free access to text and image generation models. It supports various models, including gpt3, gpt4, alpaca_7b, falcon_40b, prodia, and pollinations. The tool offers both asynchronous and non-asynchronous interfaces for text completion and image generation. It also features an interactive Discord bot that provides access to all the models in the repository. The tool is easy to use and can be integrated into various applications.
aiodocker
Aiodocker is a simple Docker HTTP API wrapper written with asyncio and aiohttp. It provides asynchronous bindings for interacting with Docker containers and images. Users can easily manage Docker resources using async functions and methods. The library offers features such as listing images and containers, creating and running containers, and accessing container logs. Aiodocker is designed to work seamlessly with Python's asyncio framework, making it suitable for building asynchronous Docker management applications.
CopilotKit
CopilotKit is an open-source framework for building, deploying, and operating fully custom AI Copilots, including in-app AI chatbots, AI agents, and AI Textareas. It provides a set of components and entry points that allow developers to easily integrate AI capabilities into their applications. CopilotKit is designed to be flexible and extensible, so developers can tailor it to their specific needs. It supports a variety of use cases, including providing app-aware AI chatbots that can interact with the application state and take action, drop-in replacements for textareas with AI-assisted text generation, and in-app agents that can access real-time application context and take action within the application.
acte
Acte is a framework designed to build GUI-like tools for AI Agents. It aims to address the issues of cognitive load and freedom degrees when interacting with multiple APIs in complex scenarios. By providing a graphical user interface (GUI) for Agents, Acte helps reduce cognitive load and constraints interaction, similar to how humans interact with computers through GUIs. The tool offers APIs for starting new sessions, executing actions, and displaying screens, accessible via HTTP requests or the SessionManager class.
herc.ai
Herc.ai is a powerful library for interacting with the Herc.ai API. It offers free access to users and supports all languages. Users can benefit from Herc.ai's features unlimitedly with a one-time subscription and API key. The tool provides functionalities for question answering and text-to-image generation, with support for various models and customization options. Herc.ai can be easily integrated into CLI, CommonJS, TypeScript, and supports beta models for advanced usage. Developed by FiveSoBes and Luppux Development.
orch
orch is a library for building language model powered applications and agents for the Rust programming language. It can be used for tasks such as text generation, streaming text generation, structured data generation, and embedding generation. The library provides functionalities for executing various language model tasks and can be integrated into different applications and contexts. It offers flexibility for developers to create language model-powered features and applications in Rust.
zenu
ZeNu is a high-performance deep learning framework implemented in pure Rust, featuring a pure Rust implementation for safety and performance, GPU performance comparable to PyTorch with CUDA support, a simple and intuitive API, and a modular design for easy extension. It supports various layers like Linear, Convolution 2D, LSTM, and optimizers such as SGD and Adam. ZeNu also provides device support for CPU and CUDA (NVIDIA GPU) with CUDA 12.3 and cuDNN 9. The project structure includes main library, automatic differentiation engine, neural network layers, matrix operations, optimization algorithms, CUDA implementation, and other support crates. Users can find detailed implementations like MNIST classification, CIFAR10 classification, and ResNet implementation in the examples directory. Contributions to ZeNu are welcome under the MIT License.
x
Ant Design X is a tool for crafting AI-driven interfaces effortlessly. It is built on the best practices of enterprise-level AI products, offering flexible and diverse atomic components for various AI dialogue scenarios. The tool provides out-of-the-box model integration with inference services compatible with OpenAI standards. It also enables efficient management of conversation data flows, supports rich template options, complete TypeScript support, and advanced theme customization. Ant Design X is designed to enhance development efficiency and deliver exceptional AI interaction experiences.
venom
Venom is a high-performance system developed with JavaScript to create a bot for WhatsApp, support for creating any interaction, such as customer service, media sending, sentence recognition based on artificial intelligence and all types of design architecture for WhatsApp.
go-anthropic
Go-anthropic is an unofficial API wrapper for Anthropic Claude in Go. It supports completions, streaming completions, messages, streaming messages, vision, and tool use. Users can interact with the Anthropic Claude API to generate text completions, analyze messages, process images, and utilize specific tools for various tasks.
aio-pika
Aio-pika is a wrapper around aiormq for asyncio and humans. It provides a completely asynchronous API, object-oriented API, transparent auto-reconnects with complete state recovery, Python 3.7+ compatibility, transparent publisher confirms support, transactions support, and complete type-hints coverage.
agents-flex
Agents-Flex is a LLM Application Framework like LangChain base on Java. It provides a set of tools and components for building LLM applications, including LLM Visit, Prompt and Prompt Template Loader, Function Calling Definer, Invoker and Running, Memory, Embedding, Vector Storage, Resource Loaders, Document, Splitter, Loader, Parser, LLMs Chain, and Agents Chain.
wenxin-starter
WenXin-Starter is a spring-boot-starter for Baidu's "Wenxin Qianfan WENXINWORKSHOP" large model, which can help you quickly access Baidu's AI capabilities. It fully integrates the official API documentation of Wenxin Qianfan. Supports text-to-image generation, built-in dialogue memory, and supports streaming return of dialogue. Supports QPS control of a single model and supports queuing mechanism. Plugins will be added soon.
agentops
AgentOps is a toolkit for evaluating and developing robust and reliable AI agents. It provides benchmarks, observability, and replay analytics to help developers build better agents. AgentOps is open beta and can be signed up for here. Key features of AgentOps include: - Session replays in 3 lines of code: Initialize the AgentOps client and automatically get analytics on every LLM call. - Time travel debugging: (coming soon!) - Agent Arena: (coming soon!) - Callback handlers: AgentOps works seamlessly with applications built using Langchain and LlamaIndex.
aiohttp
aiohttp is an async http client/server framework that supports both client and server side of HTTP protocol. It also supports both client and server Web-Sockets out-of-the-box and avoids Callback Hell. aiohttp provides a Web-server with middleware and pluggable routing.
For similar tasks
island-ai
island-ai is a TypeScript toolkit tailored for developers engaging with structured outputs from Large Language Models. It offers streamlined processes for handling, parsing, streaming, and leveraging AI-generated data across various applications. The toolkit includes packages like zod-stream for interfacing with LLM streams, stream-hooks for integrating streaming JSON data into React applications, and schema-stream for JSON streaming parsing based on Zod schemas. Additionally, related packages like @instructor-ai/instructor-js focus on data validation and retry mechanisms, enhancing the reliability of data processing workflows.
n8n-docs
n8n is an extendable workflow automation tool that enables you to connect anything to everything. It is open-source and can be self-hosted or used as a service. n8n provides a visual interface for creating workflows, which can be used to automate tasks such as data integration, data transformation, and data analysis. n8n also includes a library of pre-built nodes that can be used to connect to a variety of applications and services. This makes it easy to create complex workflows without having to write any code.
hash
HASH is a self-building, open-source database which grows, structures and checks itself. With it, we're creating a platform for decision-making, which helps you integrate, understand and use data in a variety of different ways.
ezdata
Ezdata is a data processing and task scheduling system developed based on Python backend and Vue3 frontend. It supports managing multiple data sources, abstracting various data sources into a unified data model, integrating chatgpt for data question and answer functionality, enabling low-code data integration and visualization processing, scheduling single and dag tasks, and integrating a low-code data visualization dashboard system.
buildel
Buildel is an AI automation platform that empowers users to create versatile workflows without writing code. It supports multiple providers and interfaces, offers pre-built use cases, and allows users to bring their own API keys. Ideal for AI-powered document retrieval, conversational interfaces, and data integration. Users can get started at app.buildel.ai or run Buildel locally with Node.js, Elixir/Erlang, Docker, Git, and JQ installed. Join the community on Discord for support and discussions.
obot
Obot is an open source AI agent platform that allows users to build agents for various use cases such as copilots, assistants, and autonomous workflows. It offers integration with leading LLM providers, built-in RAG for data, easy integration with custom web services and APIs, and OAuth 2.0 authentication.
partial-json-parser-js
Partial JSON Parser is a lightweight and customizable library for parsing partial JSON strings. It allows users to parse incomplete JSON data and stream it to the user. The library provides options to specify what types of partialness are allowed during parsing, such as strings, objects, arrays, special values, and more. It helps handle malformed JSON and returns the parsed JavaScript value. Partial JSON Parser is implemented purely in JavaScript and offers both commonjs and esm builds.
draive
draive is an open-source Python library designed to simplify and accelerate the development of LLM-based applications. It offers abstract building blocks for connecting functionalities with large language models, flexible integration with various AI solutions, and a user-friendly framework for building scalable data processing pipelines. The library follows a function-oriented design, allowing users to represent complex programs as simple functions. It also provides tools for measuring and debugging functionalities, ensuring type safety and efficient asynchronous operations for modern Python apps.
For similar jobs
sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.
teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.
ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.
classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.
chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.
BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students
uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.
griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.