sparkle
An agent focused LLM package for Laravel
Stars: 56
Sparkle is a tool that streamlines the process of building AI-driven features in applications using Large Language Models (LLMs). It guides users through creating and managing agents, defining tools, and interacting with LLM providers like OpenAI. Sparkle allows customization of LLM provider settings, model configurations, and provides a seamless integration with Sparkle Server for exposing agents via an OpenAI-compatible chat API endpoint.
README:
Sparkle provides a streamlined way to build AI-driven features in your application using Large Language Models (LLMs). This guide will walk you through creating and managing agents, defining tools, and interacting with various LLM providers, such as OpenAI.
Before you begin, ensure you have installed the necessary packages and configured your application to use Sparkle.
You can install Sparkle via Composer:
composer require echolabs/sparkleAfter installation, you can publish the configuration file:
php artisan vendor:publish --provider="EchoLabs\Sparkle\SparkleServiceProvider"This will create a config/sparkle.php file where you can configure your default LLM provider, model, and other settings. The provider also automatically registers the routes for Sparkle Server.
The configuration file allows you to set your default LLM provider and model:
return [
'providers' => [
'openai' => [
'api_key' => env('OPENAI_API_KEY'),
],
],
];Make sure to set the necessary API keys and environment variables for your chosen provider(s).
Agents are the core of Sparkle. An agent encapsulates the logic for interacting with an LLM, including the prompt, options, and any tools it might use.
To define an agent, create a class in your App\Agents directory:
<?php
declare(strict_types=1);
namespace App\Agents;
use EchoLabs\Sparkle\Facades\Agent;
use EchoLabs\Sparkle\Facades\Tool;
class MultiToolAgent
{
public function __invoke()
{
return Agent::provider('openai')
->using('gpt-4')
->withOptions([
'top_p' => 1,
'temperature' => 0.8,
'max_tokens' => 2048,
])
->withPrompt($this->prompt())
->withTools($this->tools());
}
private function prompt(): string
{
$prompt = "MODEL ADOPTS ROLE of [PERSONA: Nyx the Cthulhu]! \r\n";
$prompt .= "Nyx is the cutest, most friendly, Cthulhu around. \r\n";
$prompt .= 'The current date and time is '.now()->toDateTimeString();
return $prompt;
}
/** @return array<int, Tool> */
public function tools(): array
{
return [
Tool::as('search')
->for('useful when you need to search for current events')
->withParameter('query', 'the search query string', 'string', true)
->using(function (string $query): string {
// simulate API request
sleep(3);
return 'The tigers game is at 3pm eastern in Detroit';
}),
Tool::as('weather')
->for('useful when you need to search for current weather conditions')
->withParameter('city', 'The city that you want the weather for', 'string', true)
->withParameter('datetime', 'the datetime for the weather conditions. format 2022-08-14 20:24:38', 'string', true)
->using(function (string $city, string $datetime): string {
// simulate API request
sleep(3);
return 'The weather will be 75° and sunny';
}),
];
}
}- Provider: The LLM provider, such as OpenAI.
- Model: The specific model, like GPT-4.
-
Options: Configuration options that modify the behavior of the LLM. (
max_tokens,temperature) - Prompt: The initial text prompt that sets the context for the model.
- Tools: Custom logic or external APIs that the agent can call.
- Memory: Memory holds the conversation history.
<?php
(new MultiToolAgent)()
->run();
// or as a stream
(new MultiToolAgent)()
->stream();Agents use the ArrayBuffer memory driver by default. You can access the underlying memory provider via $agent->memory() which allows you to add messages ex. $agent->memory()->addMessage(Message::user('Who are you?')).
<?php
use EchoLabs\Sparkle\MemoryProviders\ArrayBuffer;
$searchTool = Tool::as('search')
->for('useful when you need to search for current events')
->withParameter('query', 'the search query string', 'string', true)
->using(function (string $query): string {
// simulate API request
sleep(3);
return 'The tigers game is at 3pm eastern in Detroit';
});
$agent = Agent::provider('openai')
->using('gpt-4')
->withOptions([
'top_p' => 1,
'temperature' => 0.8,
'max_tokens' => 2048,
])
->withPrompt(view('prompts.nova'))
->withTools([$searchTool])
->withMemory(new ArrayBuffer([
Message::user('What time is the Tigers game today?'),
]));
// The tigers game is scheduled for today at 3pm in Detroit
dd($agent->run()->content);For example, if you wanted to create an Eloquent memory driver you just need to implement the MemoryProvider contract.
<?php
declare(strict_types=1);
namespace EchoLabs\Sparkle\Contracts;
use EchoLabs\Sparkle\Message;
interface MemoryProvider
{
public function addMessage(Message $message): self;
/** @param array<int, Message> $messages */
public function addMessages(array $messages): self;
/** @return array<int, Message> */
public function messages(): array;
}Tools in Sparkle represent discrete units of functionality that an agent can use. A tool could be anything from a database query to an external API call.
Tools are defined using the Tool class, allowing you to specify parameters and the logic that executes when the tool is called.
<?php
Tool::as('search')
->for('useful when you need to search for current events')
->withParameter('query', 'The search query string', 'string')
->using(function (string $query): string {
// Simulate an API call or other logic
return 'The event is at 3pm eastern';
});-
as(string $name): Sets the tool's name. -
for(string $description): Provides a description of the tool, guiding the LLM on when to use it. -
withParameter(string $name, string $description, string $type = 'string', bool $required = true): Defines a parameter for the tool. -
using(Closure|Invokeable $fn): Specifies the logic executed when the tool is invoked.
Sparkle Server provides an easy way to expose your agents via an OpenAI-compatible chat API endpoint, supporting both streaming and non-streaming responses. This feature allows seamless integration with tools like Open Web UI or other platforms that expect an OpenAI-like API for interacting with LLMs.
To get started with Sparkle Server, you'll need to register your agents within your application. This is typically done within a service provider.
Agents are registered with the SparkleServer facade in a service provider. Here's an example of how to register an agent:
<?php
namespace App\Providers;
use App\Agents\MultiToolAgent;
use EchoLabs\Sparkle\Facades\SparkleServer;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
public function register(): void
{
SparkleServer::register(
'agent-with-tools',
new MultiToolAgent
);
}
}In this example, we're registering an agent called agent-with-tools using the MultiToolAgent class. The name provided will be used as the model identifier when making API requests.
Once your agents are registered, you can interact with them through a Sparkle Server endpoint. The server is compatible with the OpenAI chat completion API, making it easy to integrate with existing tools and workflows. My personal favorite tool for this is Open WebUI.
To send a message to an agent, you can use a POST request to the /sparkle/openai/v1/chat/completions endpoint. Below is an example of such a request:
curl -X "POST" "http://localhost:8001/sparkle/openai/v1/chat/completions" \
-H 'Accept: application/json' \
-H 'Content-Type: application/json; charset=utf-8' \
-d $'{
"model": "agent-with-tools",
"stream": false,
"messages": [
{
"content": "What time is the Tigers game today? Do I need a coat?",
"role": "user"
}
]
}'The server will respond with a JSON object containing the assistant's message, similar to the OpenAI API response format:
{
"id": "01J6APGE5YS8S04NN7N8REPPR6",
"object": "chat.completion",
"created": 1724788521,
"model": "agent-with-tools",
"usage": [],
"choices": [
{
"index": 0,
"delta": {
"role": "assistant",
"content": "The Tigers game is today at 3pm Eastern in Detroit. The weather will be 75° and sunny, so you shouldn't need a coat."
},
"finish_reason": "stop"
}
]
}For applications that require real-time data, Sparkle Server also supports streaming responses. Here's how you would make a request that streams the response:
curl -X "POST" "http://localhost:8001/sparkle/openai/v1/chat/completions" \
-H 'Accept: application/json' \
-H 'Content-Type: application/json; charset=utf-8' \
-d $'{
"model": "agent-with-tools",
"stream": true,
"messages": [
{
"content": "What time is the Tigers game today? Do I need a coat?",
"role": "user"
}
]
}'The server will stream back the response in chunks, allowing you to process the response as it's generated:
data: {"object":"chat.completion.chunk","model":"gpt-4","choices":[{"delta":{"role":"assistant","content":"The"}}]}
data: {"object":"chat.completion.chunk","model":"gpt-4","choices":[{"delta":{"role":"assistant","content":" Tigers"}}]}
data: {"object":"chat.completion.chunk","model":"gpt-4","choices":[{"delta":{"role":"assistant","content":" game"}}]}
...
data: {"object":"chat.completion.chunk","model":"gpt-4","choices":[{"delta":{"role":"assistant","content":"."}}]}
data: [DONE]
To list all the agents registered with Sparkle Server, you can send a GET request to the /sparkle/openai/v1/models endpoint:
curl "http://localhost:8001/sparkle/openai/v1/models" \
-H 'Accept: application/json' \
-H 'Content-Type: application/json; charset=utf-8'The server will return a list of all registered agents:
{
"object": "list",
"data": [
{
"id": "agent-with-tools",
"object": "model"
}
]
}For Tasks:
Click tags to check more tools for each tasksFor Jobs:
Alternative AI tools for sparkle
Similar Open Source Tools
sparkle
Sparkle is a tool that streamlines the process of building AI-driven features in applications using Large Language Models (LLMs). It guides users through creating and managing agents, defining tools, and interacting with LLM providers like OpenAI. Sparkle allows customization of LLM provider settings, model configurations, and provides a seamless integration with Sparkle Server for exposing agents via an OpenAI-compatible chat API endpoint.
clarifai-python
The Clarifai Python SDK offers a comprehensive set of tools to integrate Clarifai's AI platform to leverage computer vision capabilities like classification , detection ,segementation and natural language capabilities like classification , summarisation , generation , Q&A ,etc into your applications. With just a few lines of code, you can leverage cutting-edge artificial intelligence to unlock valuable insights from visual and textual content.
json-repair
JSON Repair is a toolkit designed to address JSON anomalies that can arise from Large Language Models (LLMs). It offers a comprehensive solution for repairing JSON strings, ensuring accuracy and reliability in your data processing. With its user-friendly interface and extensive capabilities, JSON Repair empowers developers to seamlessly integrate JSON repair into their workflows.
instructor
Instructor is a Python library that makes it a breeze to work with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API to manage validation, retries, and streaming responses. Get ready to supercharge your LLM workflows!
redis-vl-python
The Python Redis Vector Library (RedisVL) is a tailor-made client for AI applications leveraging Redis. It enhances applications with Redis' speed, flexibility, and reliability, incorporating capabilities like vector-based semantic search, full-text search, and geo-spatial search. The library bridges the gap between the emerging AI-native developer ecosystem and the capabilities of Redis by providing a lightweight, elegant, and intuitive interface. It abstracts the features of Redis into a grammar that is more aligned to the needs of today's AI/ML Engineers or Data Scientists.
client-python
The Mistral Python Client is a tool inspired by cohere-python that allows users to interact with the Mistral AI API. It provides functionalities to access and utilize the AI capabilities offered by Mistral. Users can easily install the client using pip and manage dependencies using poetry. The client includes examples demonstrating how to use the API for various tasks, such as chat interactions. To get started, users need to obtain a Mistral API Key and set it as an environment variable. Overall, the Mistral Python Client simplifies the integration of Mistral AI services into Python applications.
parea-sdk-py
Parea AI provides a SDK to evaluate & monitor AI applications. It allows users to test, evaluate, and monitor their AI models by defining and running experiments. The SDK also enables logging and observability for AI applications, as well as deploying prompts to facilitate collaboration between engineers and subject-matter experts. Users can automatically log calls to OpenAI and Anthropic, create hierarchical traces of their applications, and deploy prompts for integration into their applications.
redisvl
Redis Vector Library (RedisVL) is a Python client library for building AI applications on top of Redis. It provides a high-level interface for managing vector indexes, performing vector search, and integrating with popular embedding models and providers. RedisVL is designed to make it easy for developers to build and deploy AI applications that leverage the speed, flexibility, and reliability of Redis.
Lumos
Lumos is a Chrome extension powered by a local LLM co-pilot for browsing the web. It allows users to summarize long threads, news articles, and technical documentation. Users can ask questions about reviews and product pages. The tool requires a local Ollama server for LLM inference and embedding database. Lumos supports multimodal models and file attachments for processing text and image content. It also provides options to customize models, hosts, and content parsers. The extension can be easily accessed through keyboard shortcuts and offers tools for automatic invocation based on prompts.
SimplerLLM
SimplerLLM is an open-source Python library that simplifies interactions with Large Language Models (LLMs) for researchers and beginners. It provides a unified interface for different LLM providers, tools for enhancing language model capabilities, and easy development of AI-powered tools and apps. The library offers features like unified LLM interface, generic text loader, RapidAPI connector, SERP integration, prompt template builder, and more. Users can easily set up environment variables, create LLM instances, use tools like SERP, generic text loader, calling RapidAPI APIs, and prompt template builder. Additionally, the library includes chunking functions to split texts into manageable chunks based on different criteria. Future updates will bring more tools, interactions with local LLMs, prompt optimization, response evaluation, GPT Trainer, document chunker, advanced document loader, integration with more providers, Simple RAG with SimplerVectors, integration with vector databases, agent builder, and LLM server.
lmstudio.js
lmstudio.js is a pre-release alpha client SDK for LM Studio, allowing users to use local LLMs in JS/TS/Node. It is currently undergoing rapid development with breaking changes expected. Users can follow LM Studio's announcements on Twitter and Discord. The SDK provides API usage for loading models, predicting text, setting up the local LLM server, and more. It supports features like custom loading progress tracking, model unloading, structured output prediction, and cancellation of predictions. Users can interact with LM Studio through the CLI tool 'lms' and perform tasks like text completion, conversation, and getting prediction statistics.
syncode
SynCode is a novel framework for the grammar-guided generation of Large Language Models (LLMs) that ensures syntactically valid output with respect to defined Context-Free Grammar (CFG) rules. It supports general-purpose programming languages like Python, Go, SQL, JSON, and more, allowing users to define custom grammars using EBNF syntax. The tool compares favorably to other constrained decoders and offers features like fast grammar-guided generation, compatibility with HuggingFace Language Models, and the ability to work with various decoding strategies.
langrila
Langrila is a library that provides an easy way to use API-based LLM (Large Language Models) with an emphasis on simple architecture for readability. It supports various AI models for chat and embedding tasks, as well as retrieval functionalities using Qdrant, Chroma, and Usearch. Langrila also includes modules for function calling, conversation memory management, and prompt templates. It enforces coding policies for simplicity, responsibility independence, and minimum module implementation. The library requires Python version 3.10 to 3.13 and additional dependencies like OpenAI, Gemini, Qdrant, Chroma, and Usearch for specific functionalities.
sparrow
Sparrow is an innovative open-source solution for efficient data extraction and processing from various documents and images. It seamlessly handles forms, invoices, receipts, and other unstructured data sources. Sparrow stands out with its modular architecture, offering independent services and pipelines all optimized for robust performance. One of the critical functionalities of Sparrow - pluggable architecture. You can easily integrate and run data extraction pipelines using tools and frameworks like LlamaIndex, Haystack, or Unstructured. Sparrow enables local LLM data extraction pipelines through Ollama or Apple MLX. With Sparrow solution you get API, which helps to process and transform your data into structured output, ready to be integrated with custom workflows. Sparrow Agents - with Sparrow you can build independent LLM agents, and use API to invoke them from your system. **List of available agents:** * **llamaindex** - RAG pipeline with LlamaIndex for PDF processing * **vllamaindex** - RAG pipeline with LLamaIndex multimodal for image processing * **vprocessor** - RAG pipeline with OCR and LlamaIndex for image processing * **haystack** - RAG pipeline with Haystack for PDF processing * **fcall** - Function call pipeline * **unstructured-light** - RAG pipeline with Unstructured and LangChain, supports PDF and image processing * **unstructured** - RAG pipeline with Weaviate vector DB query, Unstructured and LangChain, supports PDF and image processing * **instructor** - RAG pipeline with Unstructured and Instructor libraries, supports PDF and image processing. Works great for JSON response generation
lightllm
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework known for its lightweight design, scalability, and high-speed performance. It offers features like tri-process asynchronous collaboration, Nopad for efficient attention operations, dynamic batch scheduling, FlashAttention integration, tensor parallelism, Token Attention for zero memory waste, and Int8KV Cache. The tool supports various models like BLOOM, LLaMA, StarCoder, Qwen-7b, ChatGLM2-6b, Baichuan-7b, Baichuan2-7b, Baichuan2-13b, InternLM-7b, Yi-34b, Qwen-VL, Llava-7b, Mixtral, Stablelm, and MiniCPM. Users can deploy and query models using the provided server launch commands and interact with multimodal models like QWen-VL and Llava using specific queries and images.
aiavatarkit
AIAvatarKit is a tool for building AI-based conversational avatars quickly. It supports various platforms like VRChat and cluster, along with real-world devices. The tool is extensible, allowing unlimited capabilities based on user needs. It requires VOICEVOX API, Google or Azure Speech Services API keys, and Python 3.10. Users can start conversations out of the box and enjoy seamless interactions with the avatars.
For similar tasks
Flowise
Flowise is a tool that allows users to build customized LLM flows with a drag-and-drop UI. It is open-source and self-hostable, and it supports various deployments, including AWS, Azure, Digital Ocean, GCP, Railway, Render, HuggingFace Spaces, Elestio, Sealos, and RepoCloud. Flowise has three different modules in a single mono repository: server, ui, and components. The server module is a Node backend that serves API logics, the ui module is a React frontend, and the components module contains third-party node integrations. Flowise supports different environment variables to configure your instance, and you can specify these variables in the .env file inside the packages/server folder.
nlux
nlux is an open-source Javascript and React JS library that makes it super simple to integrate powerful large language models (LLMs) like ChatGPT into your web app or website. With just a few lines of code, you can add conversational AI capabilities and interact with your favourite LLM.
generative-ai-go
The Google AI Go SDK enables developers to use Google's state-of-the-art generative AI models (like Gemini) to build AI-powered features and applications. It supports use cases like generating text from text-only input, generating text from text-and-images input (multimodal), building multi-turn conversations (chat), and embedding.
awesome-langchain-zh
The awesome-langchain-zh repository is a collection of resources related to LangChain, a framework for building AI applications using large language models (LLMs). The repository includes sections on the LangChain framework itself, other language ports of LangChain, tools for low-code development, services, agents, templates, platforms, open-source projects related to knowledge management and chatbots, as well as learning resources such as notebooks, videos, and articles. It also covers other LLM frameworks and provides additional resources for exploring and working with LLMs. The repository serves as a comprehensive guide for developers and AI enthusiasts interested in leveraging LangChain and LLMs for various applications.
Large-Language-Model-Notebooks-Course
This practical free hands-on course focuses on Large Language models and their applications, providing a hands-on experience using models from OpenAI and the Hugging Face library. The course is divided into three major sections: Techniques and Libraries, Projects, and Enterprise Solutions. It covers topics such as Chatbots, Code Generation, Vector databases, LangChain, Fine Tuning, PEFT Fine Tuning, Soft Prompt tuning, LoRA, QLoRA, Evaluate Models, Knowledge Distillation, and more. Each section contains chapters with lessons supported by notebooks and articles. The course aims to help users build projects and explore enterprise solutions using Large Language Models.
ai-chatbot
Next.js AI Chatbot is an open-source app template for building AI chatbots using Next.js, Vercel AI SDK, OpenAI, and Vercel KV. It includes features like Next.js App Router, React Server Components, Vercel AI SDK for streaming chat UI, support for various AI models, Tailwind CSS styling, Radix UI for headless components, chat history management, rate limiting, session storage with Vercel KV, and authentication with NextAuth.js. The template allows easy deployment to Vercel and customization of AI model providers.
awesome-local-llms
The 'awesome-local-llms' repository is a curated list of open-source tools for local Large Language Model (LLM) inference, covering both proprietary and open weights LLMs. The repository categorizes these tools into LLM inference backend engines, LLM front end UIs, and all-in-one desktop applications. It collects GitHub repository metrics as proxies for popularity and active maintenance. Contributions are encouraged, and users can suggest additional open-source repositories through the Issues section or by running a provided script to update the README and make a pull request. The repository aims to provide a comprehensive resource for exploring and utilizing local LLM tools.
Awesome-AI-Data-Guided-Projects
A curated list of data science & AI guided projects to start building your portfolio. The repository contains guided projects covering various topics such as large language models, time series analysis, computer vision, natural language processing (NLP), and data science. Each project provides detailed instructions on how to implement specific tasks using different tools and technologies.
For similar jobs
sweep
Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.
teams-ai
The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.
ai-guide
This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.
classifai
Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.
chatbot-ui
Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.
BricksLLM
BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students
uAgents
uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.
griptape
Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.