latitude-llm

latitude-llm

Latitude is the open-source prompt engineering platform to build, evaluate, and refine your prompts with AI

Stars: 3891

Visit
 screenshot

Latitude is an open-source prompt engineering platform that helps developers and product teams build AI features with confidence. It simplifies prompt management, aids in testing AI responses, and provides detailed analytics on request performance. Latitude offers collaborative prompt management, support for advanced features, version control, API and SDKs for integration, observability, evaluations in batch or real-time, and is community-driven. It can be deployed on Latitude Cloud for a managed solution or self-hosted for control and customization.

README:

Latitude — Open Source AI Engineering Platform

Open-Source AI Engineering Platform

Observability and evaluations first, then an eval-driven reliability loop to continuously improve prompts.

Docs  ·   Slack  ·   X

Latitude demo — observability, evals, and prompt management

🌈 Why Latitude?

Latitude is an open-source platform for building and operating LLM features in production.

Most teams adopt Latitude in stages: start by instrumenting your existing LLM calls to get observability and evaluation coverage, then move into a reliability loop that turns production failures into repeatable fixes.

Start with observability + evaluations:

  • Observability → capture prompts, inputs/outputs, tool calls, and latency/token usage/cost from real traffic
  • Prompt playground → reproduce runs, iterate with real inputs, version changes, and publish to the AI Gateway
  • Datasets → curate real examples for batch testing and regression suites
  • Evaluations → built-in evals, LLM-as-judge, and human-in-the-loop scoring
  • Experiments → compare models/providers and prompt versions with measurable results

Grow into the reliability loop:

  • Annotations → turn human judgment into a signal you can track and optimize
  • Issue discovery → cluster failures into recurring issues and failure modes
  • Automatic evals → convert issues into continuous tests that guard releases
  • Prompt optimizer (GEPA) → search prompt variations against your eval suite and reduce recurring failures

Latitude Telemetry works with most model providers and frameworks out of the box, and can be extended for custom integrations. See the full integration list (including OTLP ingest).

📚 Table Of Contents

⚡ Quick start

Latitude is available as a managed cloud product or as a self-hosted deployment:

  1. Latitude Cloud: fully managed.
  2. Latitude Self-Hosted: run the open-source distribution on your own infrastructure.

Choose the option that best fits your needs and follow the corresponding instructions below.

Latitude Cloud

To get started with Latitude, follow these steps:

  1. Sign up → Create an account at latitude.so and create a project.
  2. Instrument → Add the telemetry SDK (recommended) or export OTLP traces to compatible backend.
  3. Evaluate → Create datasets and evals to measure quality and catch regressions.
  4. Manage + ship → Version prompts/agents, publish changes, and deploy via the gateway.
  5. Optimize → Use eval-driven optimization to reduce recurring failures.

For more details on each step, see our documentation or join the community.

Latitude Self-Hosted

Follow the instructions in the self-hosted guide to get started with Latitude Self-Hosted.

After setting up Latitude Self-Hosted, you can follow the same steps as in the Latitude Cloud guide to create, test, evaluate, and deploy your prompts.

👥 Community

The Latitude community is on Slack, where you can ask questions, share feedback, and show what you're building.

🤝 Contributing

Contributions are welcome. For an overview of the repo and its architecture, see the contributor guide.

If you want to help, join the Slack community, open an issue, or submit a pull request.

📄 License

Latitude is licensed under the LGPL-3.0.

Alternatively, we offer a more permissive commercial license for those who need it. Please contact us at [email protected] for more information.

🔗 Links

Made with love by the Latitude Team

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for latitude-llm

Similar Open Source Tools

For similar tasks

For similar jobs