unify

unify

Notion for AI Observability ๐Ÿ“Š

Stars: 309

Visit
 screenshot

The Unify Python Package provides access to the Unify REST API, allowing users to query Large Language Models (LLMs) from any Python 3.7.1+ application. It includes Synchronous and Asynchronous clients with Streaming responses support. Users can easily use any endpoint with a single key, route to the best endpoint for optimal throughput, cost, or latency, and customize prompts to interact with the models. The package also supports dynamic routing to automatically direct requests to the top-performing provider. Additionally, users can enable streaming responses and interact with the models asynchronously for handling multiple user requests simultaneously.

README:


Static Badge X (formerly Twitter) Follow Static Badge

Fully hackable LLMOps. Build custom interfaces for: logging, evals, guardrails, labelling, tracing, agents, human-in-the-loop, hyperparam sweeps, and anything else you can think of โœจ

Just unify.log your data, and add an interface using the four building blocks:

  1. tables ๐Ÿ”ข
  2. views ๐Ÿ”
  3. plots ๐Ÿ“Š
  4. editor ๐Ÿ•น๏ธ (coming soon)

Every LLM product has unique and changing requirements, as do the users. Your infra should reflect this!

We've tried to make Unify as (a) simple, (b) modular and (c) hackable as possible, so you can quickly probe, analyze, and iterate on the data that's important for you, your product and your users โšก

Quickstart

Sign up, pip install unifyai, run your first eval โฌ‡๏ธ, and then check out the logs in your first interface ๐Ÿ“Š

import unify
from random import randint, choice

# initialize project
unify.activate("Maths Assistant")

# build agent
client = unify.Unify("o3-mini@openai", traced=True)
client.set_system_message(
    "You are a helpful maths assistant, "
    "tasked with adding and subtracting integers."
)

# add test cases
qs = [
    f"{randint(0, 100)} {choice(['+', '-'])} {randint(0, 100)}"
    for i in range(10)
]

# define evaluator
@unify.traced
def evaluate_response(question: str, response: str) -> float:
    correct_answer = eval(question)
    try:
        response_int = int(
            "".join(
                [
                    c for c in response.split(" ")[-1]
                    if c.isdigit()
                ]
            ),
        )
        return float(correct_answer == response_int)
    except ValueError:
        return 0.

# define evaluation
@unify.traced
def evaluate(q: str):
    response = client.copy().generate(q)
    score = evaluate_response(q, response)
    unify.log(
        question=q,
        response=response,
        score=score
    )

# execute + log your evaluation
with unify.Experiment():
    unify.map(evaluate, qs)

Check out our Quickstart Video for a guided walkthrough.

Focus on your product, not the LLM ๐ŸŽฏ

Despite all of the hype, abstractions, and jargon, the process for building quality LLM apps is pretty simple.

create simplest possible agent ๐Ÿค–
while True:
    create/expand unit tests (evals) ๐Ÿ—‚๏ธ
    while run(tests) failing: ๐Ÿงช
        Analyze failures, understand the root cause ๐Ÿ”
        Vary system prompt, in-context examples, tools etc. to rectify ๐Ÿ”€
    Beta test with users, find more failures ๐Ÿšฆ

We've tried to strip away all of the excessive LLM jargon, so you can focus on your product, your users, and the data you care about, and nothing else ๐Ÿ“ˆ

Unify takes inspiration from:

Whether you're technical or non-technical, we hope Unify can help you to rapidly build top-notch LLM apps, and to remain fully focused on your product (not the LLM).

Learn More

Check out our docs, and if you have any questions feel free to reach out to us on discord ๐Ÿ‘พ

Unify is under active development ๐Ÿšง, feedback in all shapes/sizes is also very welcome! ๐Ÿ™

Happy prompting! ๐Ÿง‘โ€๐Ÿ’ป

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for unify

Similar Open Source Tools

For similar tasks

For similar jobs