langfuse

langfuse

🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Stars: 5950

Visit
 screenshot

Langfuse is a powerful tool that helps you develop, monitor, and test your LLM applications. With Langfuse, you can: * **Develop:** Instrument your app and start ingesting traces to Langfuse, inspect and debug complex logs, and manage, version, and deploy prompts from within Langfuse. * **Monitor:** Track metrics (cost, latency, quality) and gain insights from dashboards & data exports, collect and calculate scores for your LLM completions, run model-based evaluations, collect user feedback, and manually score observations in Langfuse. * **Test:** Track and test app behaviour before deploying a new version, test expected in and output pairs and benchmark performance before deploying, and track versions and releases in your application. Langfuse is easy to get started with and offers a generous free tier. You can sign up for Langfuse Cloud or deploy Langfuse locally or on your own infrastructure. Langfuse also offers a variety of integrations to make it easy to connect to your LLM applications.

README:

Langfuse GitHub Banner

Langfuse: Open Source LLM Engineering Platform

LLM Observability, Prompt Management, LLM Evaluations,
Datasets, LLM Metrics, and Prompt Playground

Docs · Report Bug · Feature Request · Changelog · Roadmap · Discord
Langfuse uses Github Discussions for Support and Feature Requests.
We're hiring. Join us in Product Engineering and Developer Relations.

MIT License Y Combinator W23 Docker Image langfuse Python package on PyPi langfuse npm package

Langfuse Overview

Unmute video for voice-over

https://github.com/langfuse/langfuse/assets/2834609/a94062e9-c782-4ee9-af59-dee6370149a8

Develop

Monitor

Test

  • Experiments: Track and test app behaviour before deploying a new version

Get started

Langfuse Cloud

Managed deployment by the Langfuse team, generous free-tier (hobby plan), no credit card required.

» Langfuse Cloud

Self-Hosting Open Source LLM Observability with Langfuse

Localhost (docker)

# Clone repository
git clone https://github.com/langfuse/langfuse.git
cd langfuse

# Run server and database
docker compose up -d

→ Learn more about deploying locally

Self-host (docker)

Langfuse is simple to self-host and keep updated. It currently requires only a single docker container and a postgres database. → Self Hosting Instructions

Templated deployments: Railway, GCP, AWS, Azure, Kubernetes and others

Get Started

API Keys

You need a Langfuse public and secret key to get started. Sign up here and find them in your project settings.

Ingesting Data · Instrumenting Your Application · LLM Observability with Langfuse

Note: We recommend using our fully async, typed SDKs that allow you to instrument any LLM application with any underlying model. They are available in Python (Decorators) & JS/TS. The SDKs will always be the most fully featured and stable way to ingest data into Langfuse.

See the → Quickstart to integrate Langfuse.

LLM Observability Integrations

Integration Supports Description
SDK Python, JS/TS Manual instrumentation using the SDKs for full flexibility.
OpenAI Python, JS/TS Automated instrumentation using drop-in replacement of OpenAI SDK.
Langchain Python, JS/TS Automated instrumentation by passing callback handler to Langchain application.
LlamaIndex Python Automated instrumentation via LlamaIndex callback system.
Haystack Python Automated instrumentation via Haystack content tracing system.
LiteLLM Python, JS/TS (proxy only) Use any LLM as a drop in replacement for GPT. Use Azure, OpenAI, Cohere, Anthropic, Ollama, VLLM, Sagemaker, HuggingFace, Replicate (100+ LLMs).
Vercel AI SDK JS/TS TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js.
API Directly call the public API. OpenAPI spec available.

Packages integrated with Langfuse:

Name Description
Instructor Library to get structured LLM outputs (JSON, Pydantic)
Dify Open source LLM app development platform with no-code builder.
Ollama Easily run open source LLMs on your own machine.
Mirascope Python toolkit for building LLM applications.
Flowise JS/TS no-code builder for customized LLM flows.
Langflow Python-based UI for LangChain, designed with react-flow to provide an effortless way to experiment and prototype flows.

Questions and feedback

Ideas and roadmap

Support and feedback

In order of preference the best way to communicate with us:

Contributing to Langfuse

  • Vote on Ideas
  • Raise and comment on Issues
  • Open a PR - see CONTRIBUTING.md for details on how to setup a development environment.

License

This repository is MIT licensed, except for the ee folders. See LICENSE and docs for more details.

Misc

GET API to export your data

GET routes to use data in downstream applications (e.g. embedded analytics). You can also access them conveniently via the SDKs (docs).

Security & Privacy

We take data security and privacy seriously. Please refer to our Security and Privacy page for more information.

Telemetry

By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).

This helps us to:

  1. Understand how Langfuse is used and improve the most relevant features.
  2. Track overall usage for internal and external (e.g. fundraising) reporting.

None of the data is shared with third parties and does not include any sensitive information. We want to be super transparent about this and you can find the exact data we collect here.

You can opt-out by setting TELEMETRY_ENABLED=false.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for langfuse

Similar Open Source Tools

For similar tasks

For similar jobs