Awesome-LLMOps

Awesome-LLMOps

An awesome & curated list of best LLMOps tools for developers

Stars: 3758

Visit
 screenshot

README:

Awesome LLMOps

discord invitation link

An awesome & curated list of the best LLMOps tools for developers.

Contribute

Contributions are most welcome, please adhere to the contribution guidelines.

Table of Contents

Model

Large Language Model

Project Details Repository
Alpaca Code and documentation to train Stanford's Alpaca models, and generate the data. GitHub Badge
BELLE A 7B Large Language Model fine-tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca. GitHub Badge
Bloom BigScience Large Open-science Open-access Multilingual Language Model GitHub Badge
dolly Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform GitHub Badge
Falcon 40B Falcon-40B-Instruct is a 40B parameters causal decoder-only model built by TII based on Falcon-40B and finetuned on a mixture of Baize. It is made available under the Apache 2.0 license.
FastChat (Vicuna) An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and FastChat-T5. GitHub Badge
Gemma Gemma is a family of lightweight, open models built from the research and technology that Google used to create the Gemini models.
GLM-6B (ChatGLM) An Open Bilingual Pre-Trained Model, quantization of ChatGLM-130B, can run on consumer-level GPUs. GitHub Badge
ChatGLM2-6B ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B. GitHub Badge
GLM-130B (ChatGLM) An Open Bilingual Pre-Trained Model (ICLR 2023) GitHub Badge
GPT-NeoX An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library. GitHub Badge
Luotuo A Chinese LLM, Based on LLaMA and fine tune by Stanford Alpaca, Alpaca LoRA, Japanese-Alpaca-LoRA. GitHub Badge
Mixtral-8x7B-v0.1 The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts.
StableLM StableLM: Stability AI Language Models GitHub Badge

⬆ back to ToC

CV Foundation Model

Project Details Repository
disco-diffusion A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations. GitHub Badge
midjourney Midjourney is an independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.
segment-anything (SAM) produces high quality object masks from input prompts such as points or boxes, and it can be used to generate masks for all objects in an image. GitHub Badge
stable-diffusion A latent text-to-image diffusion model GitHub Badge
stable-diffusion v2 High-Resolution Image Synthesis with Latent Diffusion Models GitHub Badge

⬆ back to ToC

Audio Foundation Model

Project Details Repository
bark Bark is a transformer-based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. GitHub Badge
whisper Robust Speech Recognition via Large-Scale Weak Supervision GitHub Badge

Serving

Large Model Serving

Project Details Repository
Alpaca-LoRA-Serve Alpaca-LoRA as Chatbot service GitHub Badge
CTranslate2 fast inference engine for Transformer models in C++ GitHub Badge
Clip-as-a-service serving the OpenAI CLIP model GitHub Badge
DeepSpeed-MII MII makes low-latency and high-throughput inference possible, powered by DeepSpeed. GitHub Badge
Faster Whisper fast inference engine for whisper in C++ using CTranslate2. GitHub Badge
FlexGen Running large language models on a single GPU for throughput-oriented scenarios. GitHub Badge
Flowise Drag & drop UI to build your customized LLM flow using LangchainJS. GitHub Badge
llama.cpp Port of Facebook's LLaMA model in C/C++ GitHub Badge
Infinity Rest API server for serving text-embeddings GitHub Badge
Modelz-LLM OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others) GitHub Badge
Ollama Serve Llama 2 and other large language models locally from command line or through a browser interface. GitHub Badge
TensorRT-LLM Inference engine for TensorRT on Nvidia GPUs GitHub Badge
text-generation-inference Large Language Model Text Generation Inference GitHub Badge
text-embeddings-inference Inference for text-embedding models GitHub Badge
tokenizers 💥 Fast State-of-the-Art Tokenizers optimized for Research and Production GitHub Badge
vllm A high-throughput and memory-efficient inference and serving engine for LLMs. GitHub stars
whisper.cpp Port of OpenAI's Whisper model in C/C++ GitHub Badge
x-stable-diffusion Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. GitHub Badge

⬆ back to ToC

Frameworks/Servers for Serving

Project Details Repository
BentoML The Unified Model Serving Framework GitHub Badge
Jina Build multimodal AI services via cloud native technologies · Model Serving · Generative AI · Neural Search · Cloud Native GitHub Badge
Mosec A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface. GitHub Badge
TFServing A flexible, high-performance serving system for machine learning models. GitHub Badge
Torchserve Serve, optimize and scale PyTorch models in production GitHub Badge
Triton Server (TRTIS) The Triton Inference Server provides an optimized cloud and edge inferencing solution. GitHub Badge
langchain-serve Serverless LLM apps on Production with Jina AI Cloud GitHub Badge
lanarky FastAPI framework to build production-grade LLM applications GitHub Badge
ray-llm LLMs on Ray - RayLLM GitHub Badge
Xinference Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop. GitHub Badge

⬆ back to ToC

Security

Frameworks for LLM security

Project Details Repository
Plexiglass A Python Machine Learning Pentesting Toolbox for Adversarial Attacks. Works with LLMs, DNNs, and other machine learning algorithms. GitHub Badge

⬆ back to ToC

Observability

Project Details Repository
Azure OpenAI Logger "Batteries included" logging solution for your Azure OpenAI instance. GitHub Badge
Deepchecks Tests for Continuous Validation of ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort. GitHub Badge
Evidently An open-source framework to evaluate, test and monitor ML and LLM-powered systems. GitHub Badge
Fiddler AI Evaluate, monitor, analyze, and improve machine learning and generative models from pre-production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity. GitHub Badge
Giskard Testing framework dedicated to ML models, from tabular to LLMs. Detect risks of biases, performance issues and errors in 4 lines of code. GitHub Badge
Great Expectations Always know what to expect from your data. GitHub Badge
whylogs The open standard for data logging GitHub Badge

⬆ back to ToC

LLMOps

Project Details Repository
agenta The LLMOps platform to build robust LLM apps. Easily experiment and evaluate different prompts, models, and workflows to build robust apps. GitHub Badge
AI studio A Reliable Open Source AI studio to build core infrastructure stack for your LLM Applications. It allows you to gain visibility, make your application reliable, and prepare it for production with features such as caching, rate limiting, exponential retry, model fallback, and more. GitHub Badge
Arize-Phoenix ML observability for LLMs, vision, language, and tabular models. GitHub Badge
BudgetML Deploy a ML inference service on a budget in less than 10 lines of code. GitHub Badge
deeplake Stream large multimodal datasets to achieve near 100% GPU utilization. Query, visualize, & version control data. Access data w/o the need to recompute the embeddings for the model finetuning. GitHub Badge
Dify Open-source framework aims to enable developers (and even non-developers) to quickly build useful applications based on large language models, ensuring they are visual, operable, and improvable. GitHub Badge
Dstack Cost-effective LLM development in any cloud (AWS, GCP, Azure, Lambda, etc). GitHub Badge
Embedchain Framework to create ChatGPT like bots over your dataset. GitHub Badge
Evidently An open-source framework to evaluate, test and monitor ML and LLM-powered systems. GitHub Badge
Fiddler AI Evaluate, monitor, analyze, and improve MLOps and LLMOps from pre-production to production.
Glide Cloud-Native LLM Routing Engine. Improve LLM app resilience and speed. GitHub Badge
gotoHuman Bring a human into the loop in your LLM-based and agentic workflows. Prompt users to approve actions, select next steps, or review and validate generated results.
GPTCache Creating semantic cache to store responses from LLM queries. GitHub Badge
GPUStack An open-source GPU cluster manager for running and managing LLMs GitHub Badge
Haystack Quickly compose applications with LLM Agents, semantic search, question-answering and more. GitHub Badge
Helicone Open-source LLM observability platform for logging, monitoring, and debugging AI applications. Simple 1-line integration to get started. GitHub Badge
Izlo Prompt management tools for teams. Store, improve, test, and deploy your prompts in one unified workspace.
Keywords AI A unified DevOps platform for AI software. Keywords AI makes it easy for developers to build LLM applications.
langchain Building applications with LLMs through composability GitHub Badge
LangFlow An effortless way to experiment and prototype LangChain flows with drag-and-drop components and a chat interface. GitHub Badge
Langfuse Open Source LLM Engineering Platform: Traces, evals, prompt management and metrics to debug and improve your LLM application. GitHub Badge
LangKit Out-of-the-box LLM telemetry collection library that extracts features and profiles prompts, responses and metadata about how your LLM is performing over time to find problems at scale. GitHub Badge
LiteLLM 🚅 A simple & light 100 line package to standardize LLM API calls across OpenAI, Azure, Cohere, Anthropic, Replicate API Endpoints GitHub Badge
Literal AI Multi-modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and collect human feedback.
LlamaIndex Provides a central interface to connect your LLMs with external data. GitHub Badge
LLMApp LLM App is a Python library that helps you build real-time LLM-enabled data pipelines with few lines of code. GitHub Badge
LLMFlows LLMFlows is a framework for building simple, explicit, and transparent LLM applications such as chatbots, question-answering systems, and agents. GitHub Badge
LLMonitor Observability and monitoring for AI apps and agents. Debug agents with powerful tracing and logging. Usage analytics and dive deep into the history of your requests. Developer friendly modules with plug-and-play integration into LangChain. GitHub Badge
magentic Seamlessly integrate LLMs as Python functions. Use type annotations to specify structured output. Mix LLM queries and function calling with regular Python code to create complex LLM-powered functionality. GitHub Badge
Manag.ai Your all-in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease.
Mirascope Intuitive convenience tooling for lightning-fast, efficient development and ensuring quality in LLM-based applications GitHub Badge
OpenLIT OpenLIT is an OpenTelemetry-native GenAI and LLM Application Observability tool and provides OpenTelmetry Auto-instrumentation for monitoring LLMs, VectorDBs and Frameworks. It provides valuable insights into token & cost usage, user interaction, and performance related metrics. GitHub Badge
Opik Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle. GitHub Badge
Parea AI Platform and SDK for AI Engineers providing tools for LLM evaluation, observability, and a version-controlled enhanced prompt playground. GitHub Badge
Pezzo 🕹️ Pezzo is the open-source LLMOps platform built for developers and teams. In just two lines of code, you can seamlessly troubleshoot your AI operations, collaborate and manage your prompts in one place, and instantly deploy changes to any environment. GitHub Badge
PromptHub Full stack prompt management tool designed to be usable by technical and non-technical team members. Test, version, collaborate, deploy, and monitor, all from one place.
promptfoo Open-source tool for testing & evaluating prompt quality. Create test cases, automatically check output quality and catch regressions, and reduce evaluation cost. GitHub Badge
PromptFoundry The simple prompt engineering and evaluation tool designed for developers building AI applications. GitHub Badge
PromptLayer 🍰 Prompt Engineering platform. Collaborate, test, evaluate, and monitor your LLM applications Github Badge
PromptMage Open-source tool to simplify the process of creating and managing LLM workflows and prompts as a self-hosted solution. GitHub Badge
Prompteams Prompt management system. Version, test, collaborate, and retrieve prompts through real-time APIs. Have GitHub style with repos, branches, and commits (and commit history).
prompttools Open-source tools for testing and experimenting with prompts. The core idea is to enable developers to evaluate prompts using familiar interfaces like code and notebooks. In just a few lines of codes, you can test your prompts and parameters across different models (whether you are using OpenAI, Anthropic, or LLaMA models). You can even evaluate the retrieval accuracy of vector databases. GitHub Badge
TreeScale All In One Dev Platform For LLM Apps. Deploy LLM-enhanced APIs seamlessly using tools for prompt optimization, semantic querying, version management, statistical evaluation, and performance tracking. As a part of the developer friendly API implementation TreeScale offers Elastic LLM product, which makes a unified API Endpoint for all major LLM providers and open source models.
TrueFoundry Deploy LLMOps tools like Vector DBs, Embedding server etc on your own Kubernetes (EKS,AKS,GKE,On-prem) Infra including deploying, Fine-tuning, tracking Prompts and serving Open Source LLM Models with full Data Security and Optimal GPU Management. Train and Launch your LLM Application at Production scale with best Software Engineering practices.
ReliableGPT 💪 Handle OpenAI Errors (overloaded OpenAI servers, rotated keys, or context window errors) for your production LLM Applications. GitHub Badge
Portkey Control Panel with an observability suite & an AI gateway — to ship fast, reliable, and cost-efficient apps.
Vellum An AI product development platform to experiment with, evaluate, and deploy advanced LLM apps.
Weights & Biases (Prompts) A suite of LLMOps tools within the developer-first W&B MLOps platform. Utilize W&B Prompts for visualizing and inspecting LLM execution flow, tracking inputs and outputs, viewing intermediate results, securely managing prompts and LLM chain configurations.
Wordware A web-hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks.
xTuring Build and control your personal LLMs with fast and efficient fine-tuning. GitHub Badge
ZenML Open-source framework for orchestrating, experimenting and deploying production-grade ML solutions, with built-in langchain & llama_index integrations. GitHub Badge

⬆ back to ToC

Search

Vector search

Project Details Repository
AquilaDB An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search. GitHub Badge
Awadb AI Native database for embedding vectors GitHub Badge
Chroma the open source embedding database GitHub Badge
Infinity The AI-native database built for LLM applications, providing incredibly fast vector and full-text search GitHub Badge
Lancedb Developer-friendly, serverless vector database for AI applications. Easily add long-term memory to your LLM apps! GitHub Badge
Marqo Tensor search for humans. GitHub Badge
Milvus Vector database for scalable similarity search and AI applications. GitHub Badge
Pinecone The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles.
pgvector Open-source vector similarity search for Postgres. GitHub Badge
pgvecto.rs Vector database plugin for Postgres, written in Rust, specifically designed for LLM. GitHub Badge
Qdrant Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud GitHub Badge
txtai Build AI-powered semantic search applications GitHub Badge
Vald A Highly Scalable Distributed Vector Search Engine GitHub Badge
Vearch A distributed system for embedding-based vector retrieval GitHub Badge
VectorDB A Python vector database you just need - no more, no less. GitHub Badge
Vellum A managed service for ingesting documents and performing hybrid semantic/keyword search across them. Comes with out-of-box support for OCR, text chunking, embedding model experimentation, metadata filtering, and production-grade APIs.
Weaviate Weaviate is an open source vector search engine that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients. GitHub Badge

⬆ back to ToC

Code AI

Project Details Repository
CodeGeeX CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023) GitHub Badge
CodeGen CodeGen is an open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex. GitHub Badge
CodeT5 Open Code LLMs for Code Understanding and Generation. GitHub Badge
Continue ⏩ the open-source autopilot for software development—bring the power of ChatGPT to VS Code GitHub Badge
fauxpilot An open-source alternative to GitHub Copilot server GitHub Badge
tabby Self-hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot. GitHub Badge

Training

IDEs and Workspaces

Project Details Repository
code server Run VS Code on any machine anywhere and access it in the browser. GitHub Badge
conda OS-agnostic, system-level binary package manager and ecosystem. GitHub Badge
Docker Moby is an open-source project created by Docker to enable and accelerate software containerization. GitHub Badge
envd 🏕️ Reproducible development environment for AI/ML. GitHub Badge
Jupyter Notebooks The Jupyter notebook is a web-based notebook environment for interactive computing. GitHub Badge
Kurtosis A build, packaging, and run system for ephemeral multi-container environments. GitHub Badge
Wordware A web-hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks.

⬆ back to ToC

Foundation Model Fine Tuning

Project Details Repository
alpaca-lora Instruct-tune LLaMA on consumer hardware GitHub Badge
finetuning-scheduler A PyTorch Lightning extension that accelerates and enhances foundation model experimentation with flexible fine-tuning schedules. GitHub Badge
Flyflow Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost GitHub Badge
LMFlow An Extensible Toolkit for Finetuning and Inference of Large Foundation Models GitHub Badge
Lora Using Low-rank adaptation to quickly fine-tune diffusion models. GitHub Badge
peft State-of-the-art Parameter-Efficient Fine-Tuning. GitHub Badge
p-tuning-v2 An optimized prompt tuning strategy achieving comparable performance to fine-tuning on small/medium-sized models and sequence tagging challenges. (ACL 2022) GitHub Badge
QLoRA Efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. GitHub Badge
TRL Train transformer language models with reinforcement learning. GitHub Badge

⬆ back to ToC

Frameworks for Training

Project Details Repository
Accelerate 🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision. GitHub Badge
Apache MXNet Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler. GitHub Badge
axolotl A tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures. GitHub Badge
Caffe A fast open framework for deep learning. GitHub Badge
Candle Minimalist ML framework for Rust . GitHub Badge
ColossalAI An integrated large-scale model training system with efficient parallelization techniques. GitHub Badge
DeepSpeed DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective. GitHub Badge
Horovod Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet. GitHub Badge
Jax Autograd and XLA for high-performance machine learning research. GitHub Badge
Kedro Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code. GitHub Badge
Keras Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. GitHub Badge
LightGBM A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks. GitHub Badge
MegEngine MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto-differentiation. GitHub Badge
metric-learn Metric Learning Algorithms in Python. GitHub Badge
MindSpore MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios. GitHub Badge
Oneflow OneFlow is a performance-centered and open-source deep learning framework. GitHub Badge
PaddlePaddle Machine Learning Framework from Industrial Practice. GitHub Badge
PyTorch Tensors and Dynamic neural networks in Python with strong GPU acceleration. GitHub Badge
PyTorch Lightning Deep learning framework to train, deploy, and ship AI products Lightning fast. GitHub Badge
XGBoost Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library. GitHub Badge
scikit-learn Machine Learning in Python. GitHub Badge
TensorFlow An Open Source Machine Learning Framework for Everyone. GitHub Badge
VectorFlow A minimalist neural network library optimized for sparse data and single machine environments. GitHub Badge

⬆ back to ToC

Experiment Tracking

Project Details Repository
Aim an easy-to-use and performant open-source experiment tracker. GitHub Badge
ClearML Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management GitHub Badge
Comet Comet is an MLOps platform that offers experiment tracking, model production management, a model registry, and full data lineage from training straight through to production. Comet plays nicely with all your favorite tools, so you don't have to change your existing workflow. Comet Opik to confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle! GitHub Badge
Guild AI Experiment tracking, ML developer tools. GitHub Badge
MLRun Machine Learning automation and tracking. GitHub Badge
Kedro-Viz Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project. GitHub Badge
LabNotebook LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments. GitHub Badge
Sacred Sacred is a tool to help you configure, organize, log and reproduce experiments. GitHub Badge
Weights & Biases A developer first, lightweight, user-friendly experiment tracking and visualization tool for machine learning projects, streamlining collaboration and simplifying MLOps. W&B excels at tracking LLM-powered applications, featuring W&B Prompts for LLM execution flow visualization, input and output monitoring, and secure management of prompts and LLM chain configurations. GitHub Badge

⬆ back to ToC

Visualization

Project Details Repository
Fiddler AI Rich dashboards, reports, and UMAP to perform root cause analysis, pinpoint problem areas, like correctness, safety, and privacy issues, and improve LLM outcomes.
Maniford A model-agnostic visual debugging tool for machine learning. GitHub Badge
netron Visualizer for neural network, deep learning, and machine learning models. GitHub Badge
OpenOps Bring multiple data streams into one dashboard. GitHub Badge
TensorBoard TensorFlow's Visualization Toolkit. GitHub Badge
TensorSpace Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js. GitHub Badge
dtreeviz A python library for decision tree visualization and model interpretation. GitHub Badge
Zetane Viewer ML models and internal tensors 3D visualizer. GitHub Badge
Zeno AI evaluation platform for interactively exploring data and model outputs. GitHub Badge

Model Editing

Project Details Repository
FastEdit FastEdit aims to assist developers with injecting fresh and customized knowledge into large language models efficiently using one single command. GitHub Badge

⬆ back to ToC

Data

Data Management

Project Details Repository
ArtiVC A version control system to manage large files. Lake is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size. GitHub Badge
Dolt Git for Data. GitHub Badge
DVC Data Version Control - Git for Data & Models - ML Experiments Management. GitHub Badge
Delta-Lake Storage layer that brings scalable, ACID transactions to Apache Spark and other engines. GitHub Badge
Pachyderm Pachyderm is a version control system for data. GitHub Badge
Quilt A self-organizing data hub for S3. GitHub Badge

⬆ back to ToC

Data Storage

Project Details Repository
JuiceFS A distributed POSIX file system built on top of Redis and S3. GitHub Badge
LakeFS Git-like capabilities for your object storage. GitHub Badge
Lance Modern columnar data format for ML implemented in Rust. GitHub Badge

⬆ back to ToC

Data Tracking

Project Details Repository
Piperider A CLI tool that allows you to build data profiles and write assertion tests for easily evaluating and tracking your data's reliability over time. GitHub Badge
LUX A Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process. GitHub Badge

⬆ back to ToC

Feature Engineering

Project Details Repository
Featureform The Virtual Feature Store. Turn your existing data infrastructure into a feature store. GitHub Badge
FeatureTools An open source python framework for automated feature engineering GitHub Badge

⬆ back to ToC

Data/Feature enrichment

Project Details Repository
Upgini Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features GitHub Badge
Feast An open source feature store for machine learning. GitHub Badge
distilabel ⚗️ distilabel is a framework for synthetic data and AI feedback for AI engineers that require high-quality outputs, full data ownership, and overall efficiency. GitHub Badge

⬆ back to ToC

Large Scale Deployment

ML Platforms

Project Details Repository
Comet Comet is an MLOps platform that offers experiment tracking, model production management, a model registry, and full data lineage from training straight through to production. Comet plays nicely with all your favorite tools, so you don't have to change your existing workflow. Comet Opik to confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle! GitHub Badge
ClearML Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management. GitHub Badge
Hopsworks Hopsworks is a MLOps platform for training and operating large and small ML systems, including fine-tuning and serving LLMs. Hopsworks includes both a feature store and vector database for RAG. GitHub Badge
OpenLLM An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease. GitHub Badge
MLflow Open source platform for the machine learning lifecycle. GitHub Badge
MLRun An open MLOps platform for quickly building and managing continuous ML applications across their lifecycle. GitHub Badge
ModelFox ModelFox is a platform for managing and deploying machine learning models. GitHub Badge
Kserve Standardized Serverless ML Inference Platform on Kubernetes GitHub Badge
Kubeflow Machine Learning Toolkit for Kubernetes. GitHub Badge
PAI Resource scheduling and cluster management for AI. GitHub Badge
Polyaxon Machine Learning Management & Orchestration Platform. GitHub Badge
Primehub An effortless infrastructure for machine learning built on the top of Kubernetes. GitHub Badge
OpenModelZ One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or even a single machine). GitHub Badge
Seldon-core An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models GitHub Badge
Starwhale An MLOps/LLMOps platform for model building, evaluation, and fine-tuning. GitHub Badge
TrueFoundry A PaaS to deploy, Fine-tune and serve LLM Models on a company’s own Infrastructure with Data Security and Optimal GPU and Cost Management. Launch your LLM Application at Production scale with best DevSecOps practices.
Weights & Biases A lightweight and flexible platform for machine learning experiment tracking, dataset versioning, and model management, enhancing collaboration and streamlining MLOps workflows. W&B excels at tracking LLM-powered applications, featuring W&B Prompts for LLM execution flow visualization, input and output monitoring, and secure management of prompts and LLM chain configurations. GitHub Badge

⬆ back to ToC

Workflow

Project Details Repository
Airflow A platform to programmatically author, schedule and monitor workflows. GitHub Badge
aqueduct An Open-Source Platform for Production Data Science GitHub Badge
Argo Workflows Workflow engine for Kubernetes. GitHub Badge
Flyte Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale. GitHub Badge
Hamilton A lightweight framework to represent ML/language model pipelines as a series of python functions. GitHub Badge
Kubeflow Pipelines Machine Learning Pipelines for Kubeflow. GitHub Badge
LangFlow An effortless way to experiment and prototype LangChain flows with drag-and-drop components and a chat interface. GitHub Badge
Metaflow Build and manage real-life data science projects with ease! GitHub Badge
Ploomber The fastest way to build data pipelines. Develop iteratively, deploy anywhere. GitHub Badge
Prefect The easiest way to automate your data. GitHub Badge
VDP An open-source unstructured data ETL tool to streamline the end-to-end unstructured data processing pipeline. GitHub Badge
ZenML MLOps framework to create reproducible pipelines. GitHub Badge

⬆ back to ToC

Scheduling

Project Details Repository
Kueue Kubernetes-native Job Queueing. GitHub Badge
PAI Resource scheduling and cluster management for AI (Open-sourced by Microsoft). GitHub Badge
Slurm A Highly Scalable Workload Manager. GitHub Badge
Volcano A Cloud Native Batch System (Project under CNCF). GitHub Badge
Yunikorn Light-weight, universal resource scheduler for container orchestrator systems. GitHub Badge

⬆ back to ToC

Model Management

Project Details Repository
Comet Comet is an MLOps platform that offers Model Production Management, a Model Registry, and full model lineage from training straight through to production. Use Comet for model reproducibility, model debugging, model versioning, model visibility, model auditing, model governance, and model monitoring. GitHub Badge
dvc ML Experiments Management - Data Version Control - Git for Data & Models GitHub Badge
ModelDB Open Source ML Model Versioning, Metadata, and Experiment Management GitHub Badge
MLEM A tool to package, serve, and deploy any ML model on any platform. GitHub Badge
ormb Docker for Your ML/DL Models Based on OCI Artifacts GitHub Badge

⬆ back to ToC

Performance

ML Compiler

Project Details Repository
ONNX-MLIR Compiler technology to transform a valid Open Neural Network Exchange (ONNX) graph into code that implements the graph with minimum runtime support. GitHub Badge
bitsandbytes Accessible large language models via k-bit quantization for PyTorch. GitHub Badge
TVM Open deep learning compiler stack for cpu, gpu and specialized accelerators GitHub Badge

⬆ back to ToC

Profiling

Project Details Repository
octoml-profile octoml-profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware with state-of-the-art ML acceleration technology. GitHub Badge
scalene a high-performance, high-precision CPU, GPU, and memory profiler for Python GitHub Badge

⬆ back to ToC

AutoML

Project Details Repository
Archai a platform for Neural Network Search (NAS) that allows you to generate efficient deep networks for your applications. GitHub Badge
autoai A framework to find the best performing AI/ML model for any AI problem. GitHub Badge
AutoGL An autoML framework & toolkit for machine learning on graphs GitHub Badge
AutoGluon AutoML for Image, Text, and Tabular Data. GitHub Badge
automl-gs Provide an input CSV and a target field to predict, generate a model + code to run it. GitHub Badge
autokeras AutoML library for deep learning. GitHub Badge
Auto-PyTorch Automatic architecture search and hyperparameter optimization for PyTorch. GitHub Badge
auto-sklearn an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator. GitHub Badge
Dragonfly An open source python library for scalable Bayesian optimisation. GitHub Badge
Determined scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods. GitHub Badge
DEvol (DeepEvolution) a basic proof of concept for genetic architecture search in Keras. GitHub Badge
EvalML An open source python library for AutoML. GitHub Badge
FEDOT AutoML framework for the design of composite pipelines. GitHub Badge
FLAML Fast and lightweight AutoML (paper). GitHub Badge
Goptuna A hyperparameter optimization framework, inspired by Optuna. GitHub Badge
HpBandSter a framework for distributed hyperparameter optimization. GitHub Badge
HPOlib2 a library for hyperparameter optimization and black box optimization benchmarks. GitHub Badge
Hyperband open source code for tuning hyperparams with Hyperband. GitHub Badge
Hypernets A General Automated Machine Learning Framework. GitHub Badge
Hyperopt Distributed Asynchronous Hyperparameter Optimization in Python. GitHub Badge
hyperunity A toolset for black-box hyperparameter optimisation. GitHub Badge
Intelli A framework to connect a flow of ML models by applying graph theory. GitHub Badge
Katib Katib is a Kubernetes-native project for automated machine learning (AutoML). GitHub Badge
Keras Tuner Hyperparameter tuning for humans. GitHub Badge
learn2learn PyTorch Meta-learning Framework for Researchers. GitHub Badge
Ludwig a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code. GitHub Badge
MOE a global, black box optimization engine for real world metric optimization by Yelp. GitHub Badge
Model Search a framework that implements AutoML algorithms for model architecture search at scale. GitHub Badge
NASGym a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS). GitHub Badge
NNI An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning. GitHub Badge
Optuna A hyperparameter optimization framework. GitHub Badge
Pycaret An open-source, low-code machine learning library in Python that automates machine learning workflows. GitHub Badge
Ray Tune Scalable Hyperparameter Tuning. GitHub Badge
REMBO Bayesian optimization in high-dimensions via random embedding. GitHub Badge
RoBO a Robust Bayesian Optimization framework. GitHub Badge
scikit-optimize(skopt) Sequential model-based optimization with a scipy.optimize interface. GitHub Badge
Spearmint a software package to perform Bayesian optimization. GitHub Badge
TPOT one of the very first AutoML methods and open-source software packages. GitHub Badge
Torchmeta A Meta-Learning library for PyTorch. GitHub Badge
Vegas an AutoML algorithm tool chain by Huawei Noah's Arb Lab. GitHub Badge

⬆ back to ToC

Optimizations

Project Details Repository
FeatherCNN FeatherCNN is a high performance inference engine for convolutional neural networks. GitHub Badge
Forward A library for high performance deep learning inference on NVIDIA GPUs. GitHub Badge
NCNN ncnn is a high-performance neural network inference framework optimized for the mobile platform. GitHub Badge
PocketFlow use AutoML to do model compression. GitHub Badge
TensorFlow Model Optimization A suite of tools that users, both novice and advanced, can use to optimize machine learning models for deployment and execution. GitHub Badge
TNN A uniform deep learning inference framework for mobile, desktop and server. GitHub Badge
optimum-tpu Google TPU optimizations for transformers models GitHub Badge

⬆ back to ToC

Federated ML

Project Details Repository
EasyFL An Easy-to-use Federated Learning Platform GitHub Badge
FATE An Industrial Grade Federated Learning Framework GitHub Badge
FedML The federated learning and analytics library enabling secure and collaborative machine learning on decentralized data anywhere at any scale. Supporting large-scale cross-silo federated learning, cross-device federated learning on smartphones/IoTs, and research simulation. GitHub Badge
Flower A Friendly Federated Learning Framework GitHub Badge
Harmonia Harmonia is an open-source project aiming at developing systems/infrastructures and libraries to ease the adoption of federated learning (abbreviated to FL) for researches and production usage. GitHub Badge
TensorFlow Federated A framework for implementing federated learning GitHub Badge

⬆ back to ToC

Awesome Lists

Project Details Repository
Awesome Argo A curated list of awesome projects and resources related to Argo GitHub Badge
Awesome AutoDL Automated Deep Learning: Neural Architecture Search Is Not the End (a curated list of AutoDL resources and an in-depth analysis) GitHub Badge
Awesome AutoML Curating a list of AutoML-related research, tools, projects and other resources GitHub Badge
Awesome AutoML Papers A curated list of automated machine learning papers, articles, tutorials, slides and projects GitHub Badge
Awesome-Code-LLM 👨‍💻 An awesome and curated list of best code-LLM for research. GitHub Badge
Awesome Federated Learning Systems A curated list of Federated Learning Systems related academic papers, articles, tutorials, slides and projects. GitHub Badge
Awesome Federated Learning A curated list of federated learning publications, re-organized from Arxiv (mostly) GitHub Badge
awesome-federated-learningacc All materials you need for Federated Learning: blogs, videos, papers, and softwares, etc. GitHub Badge
Awesome Open MLOps This is the Fuzzy Labs guide to the universe of free and open source MLOps tools. GitHub Badge
Awesome Production Machine Learning A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning GitHub Badge
Awesome Tensor Compilers A list of awesome compiler projects and papers for tensor computation and deep learning. GitHub Badge
kelvins/awesome-mlops A curated list of awesome MLOps tools. GitHub Badge
visenger/awesome-mlops Machine Learning Operations - An awesome list of references for MLOps GitHub Badge
currentslab/awesome-vector-search A curated list of awesome vector search framework/engine, library, cloud service and research papers to vector similarity search. GitHub Badge
pleisto/flappy Production-Ready LLM Agent SDK for Every Developer GitHub Badge

⬆ back to ToC

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for Awesome-LLMOps

Similar Open Source Tools

For similar tasks

No tools available

For similar jobs

No tools available