oumi

Easily fine-tune, evaluate and deploy gpt-oss, Qwen3, DeepSeek-R1, or any open source LLM / VLM!

Stars: 8452

Visit

Oumi is an open-source platform for building state-of-the-art foundation models, offering tools for data preparation, training, evaluation, and deployment. It supports training and fine-tuning models with various parameters, working with text and multimodal models, synthesizing and curating training data, deploying models efficiently, evaluating models comprehensively, and running on different platforms. Oumi provides a consistent API, reliability, and flexibility for research purposes.

README:

Everything you need to build state-of-the-art foundation models, end-to-end.

🔥 News

[2025/09] Oumi v0.4.0 released with DeepSpeed support, a Hugging Face Hub cache management tool, KTO/Vision DPO trainer support
[2025/08] Training and inference support for OpenAI's gpt-oss-20b and gpt-oss-120b: recipes here
[2025/08] Aug 14 Webinar - OpenAI's gpt-oss: Separating the Substance from the Hype.
[2025/08] Oumi v0.3.0 released with model quantization (AWQ), an improved LLM-as-a-Judge API, and Adaptive Inference
[2025/07] Recipe for Qwen3 235B
[2025/07] July 24 webinar: "Training a State-of-the-art Agent LLM with Oumi + Lambda"
[2025/06] Oumi v0.2.0 released with support for GRPO fine-tuning, a plethora of new model support, and much more
[2025/06] Announcement of Data Curation for Vision Language Models (DCVLR) competition at NeurIPS2025
[2025/06] Recipes for training, inference, and eval with the newly released Falcon-H1 and Falcon-E models
[2025/05] Support and recipes for InternVL3 1B
[2025/04] Added support for training and inference with Llama 4 models: Scout (17B activated, 109B total) and Maverick (17B activated, 400B total) variants, including full fine-tuning, LoRA, and QLoRA configurations
[2025/04] Recipes for Qwen3 model family
[2025/04] Introducing HallOumi: a State-of-the-Art Claim-Verification Model (technical overview)
[2025/04] Oumi now supports two new Vision-Language models: Phi4 and Qwen 2.5

🔎 About

Oumi is a fully open-source platform that streamlines the entire lifecycle of foundation models - from data preparation and training to evaluation and deployment. Whether you're developing on a laptop, launching large scale experiments on a cluster, or deploying models in production, Oumi provides the tools and workflows you need.

With Oumi, you can:

🚀 Train and fine-tune models from 10M to 405B parameters using state-of-the-art techniques (SFT, LoRA, QLoRA, GRPO, and more)
🤖 Work with both text and multimodal models (Llama, DeepSeek, Qwen, Phi, and others)
🔄 Synthesize and curate training data with LLM judges
⚡️ Deploy models efficiently with popular inference engines (vLLM, SGLang)
📊 Evaluate models comprehensively across standard benchmarks
🌎 Run anywhere - from laptops to clusters to clouds (AWS, Azure, GCP, Lambda, and more)
🔌 Integrate with both open models and commercial APIs (OpenAI, Anthropic, Vertex AI, Together, Parasail, ...)

All with one consistent API, production-grade reliability, and all the flexibility you need for research.

Learn more at oumi.ai, or jump right in with the quickstart guide.

🚀 Getting Started

Notebook	Try in Colab	Goal
🎯 Getting Started: A Tour		Quick tour of core features: training, evaluation, inference, and job management
🔧 Model Finetuning Guide		End-to-end guide to LoRA tuning with data prep, training, and evaluation
📚 Model Distillation		Guide to distilling large models into smaller, efficient ones
📋 Model Evaluation		Comprehensive model evaluation using Oumi's evaluation framework
☁️ Remote Training		Launch and monitor training jobs on cloud (AWS, Azure, GCP, Lambda, etc.) platforms
📈 LLM-as-a-Judge		Filter and curate training data with built-in judges

🔧 Usage

Installation

Installing oumi in your environment is straightforward:

# Install the package (CPU & NPU only)
pip install oumi  # For local development & testing

# OR, with GPU support (Requires Nvidia or AMD GPU)
pip install oumi[gpu]  # For GPU training

# To get the latest version, install from the source
pip install git+https://github.com/oumi-ai/oumi.git

For more advanced installation options, see the installation guide.

Oumi CLI

You can quickly use the oumi command to train, evaluate, and infer models using one of the existing recipes:

# Training
oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml

# Evaluation
oumi evaluate -c configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml

# Inference
oumi infer -c configs/recipes/smollm/inference/135m_infer.yaml --interactive

For more advanced options, see the training, evaluation, inference, and llm-as-a-judge guides.

Running Jobs Remotely

You can run jobs remotely on cloud platforms (AWS, Azure, GCP, Lambda, etc.) using the oumi launch command:

# GCP
oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml

# AWS
oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml --resources.cloud aws

# Azure
oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml --resources.cloud azure

# Lambda
oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml --resources.cloud lambda

Note: Oumi is in beta and under active development. The core features are stable, but some advanced features might change as the platform improves.

💻 Why use Oumi?

If you need a comprehensive platform for training, evaluating, or deploying models, Oumi is a great choice.

Here are some of the key features that make Oumi stand out:

🔧 Zero Boilerplate: Get started in minutes with ready-to-use recipes for popular models and workflows. No need to write training loops or data pipelines.
🏢 Enterprise-Grade: Built and validated by teams training models at scale
🎯 Research Ready: Perfect for ML research with easily reproducible experiments, and flexible interfaces for customizing each component.
🌐 Broad Model Support: Works with most popular model architectures - from tiny models to the largest ones, text-only to multimodal.
🚀 SOTA Performance: Native support for distributed training techniques (FSDP, DeepSpeed, DDP) and optimized inference engines (vLLM, SGLang).
🤝 Community First: 100% open source with an active community. No vendor lock-in, no strings attached.

📚 Examples & Recipes

Explore the growing collection of ready-to-use configurations for state-of-the-art models and training workflows:

Note: These configurations are not an exhaustive list of what's supported, simply examples to get you started. You can find a more exhaustive list of supported models, and datasets (supervised fine-tuning, pre-training, preference tuning, and vision-language finetuning) in the oumi documentation.

Qwen Family

Model	Example Configurations
Qwen3 30B A3B	LoRA • Inference • Evaluation
Qwen3 32B	LoRA • Inference • Evaluation
Qwen3 14B	LoRA • Inference • Evaluation
Qwen3 8B	FFT • Inference • Evaluation
Qwen3 4B	FFT • Inference • Evaluation
Qwen3 1.7B	FFT • Inference • Evaluation
Qwen3 0.6B	FFT • Inference • Evaluation
QwQ 32B	FFT • LoRA • QLoRA • Inference • Evaluation
Qwen2.5-VL 3B	SFT • LoRA• Inference (vLLM) • Inference
Qwen2-VL 2B	SFT • LoRA • Inference (vLLM) • Inference (SGLang) • Inference • Evaluation

🐋 DeepSeek R1 Family

Model	Example Configurations
DeepSeek R1 671B	Inference (Together AI)
Distilled Llama 8B	FFT • LoRA • QLoRA • Inference • Evaluation
Distilled Llama 70B	FFT • LoRA • QLoRA • Inference • Evaluation
Distilled Qwen 1.5B	FFT • LoRA • Inference • Evaluation
Distilled Qwen 32B	LoRA • Inference • Evaluation

🦙 Llama Family

Model	Example Configurations
Llama 4 Scout Instruct 17B	FFT • LoRA • QLoRA • Inference (vLLM) • Inference • Inference (Together.ai)
Llama 4 Scout 17B	FFT
Llama 3.1 8B	FFT • LoRA • QLoRA • Pre-training • Inference (vLLM) • Inference • Evaluation
Llama 3.1 70B	FFT • LoRA • QLoRA • Inference • Evaluation
Llama 3.1 405B	FFT • LoRA • QLoRA
Llama 3.2 1B	FFT • LoRA • QLoRA • Inference (vLLM) • Inference (SGLang) • Inference • Evaluation
Llama 3.2 3B	FFT • LoRA • QLoRA • Inference (vLLM) • Inference (SGLang) • Inference • Evaluation
Llama 3.3 70B	FFT • LoRA • QLoRA • Inference (vLLM) • Inference • Evaluation
Llama 3.2 Vision 11B	SFT • Inference (vLLM) • Inference (SGLang) • Evaluation

🦅 Falcon family

Model	Example Configurations
Falcon-H1	FFT • Inference • Evaluation
Falcon-E (BitNet)	FFT • DPO • Evaluation

🎨 Vision Models

Model	Example Configurations
Llama 3.2 Vision 11B	SFT • LoRA • Inference (vLLM) • Inference (SGLang) • Evaluation
LLaVA 7B	SFT • Inference (vLLM) • Inference
Phi3 Vision 4.2B	SFT • LoRA • Inference (vLLM)
Phi4 Vision 5.6B	SFT • LoRA • Inference (vLLM) • Inference
Qwen2-VL 2B	SFT • LoRA • Inference (vLLM) • Inference (SGLang) • Inference • Evaluation
Qwen2.5-VL 3B	SFT • LoRA• Inference (vLLM) • Inference
SmolVLM-Instruct 2B	SFT • LoRA

🔍 Even more options

This section lists all the language models that can be used with Oumi. Thanks to the integration with the 🤗 Transformers library, you can easily use any of these models for training, evaluation, or inference.

Models prefixed with a checkmark (✅) have been thoroughly tested and validated by the Oumi community, with ready-to-use recipes available in the configs/recipes directory.

📋 Click to see more supported models

Instruct Models

Model	Size	Paper	HF Hub	License	Open ¹
✅ SmolLM-Instruct	135M/360M/1.7B	Blog	Hub	Apache 2.0	✅
✅ DeepSeek R1 Family	1.5B/8B/32B/70B/671B	Blog	Hub	MIT	❌
✅ Llama 3.1 Instruct	8B/70B/405B	Paper	Hub	License	❌
✅ Llama 3.2 Instruct	1B/3B	Paper	Hub	License	❌
✅ Llama 3.3 Instruct	70B	Paper	Hub	License	❌
✅ Phi-3.5-Instruct	4B/14B	Paper	Hub	License	❌
✅ Qwen3	0.6B-32B	Paper	Hub	License	❌
Qwen2.5-Instruct	0.5B-70B	Paper	Hub	License	❌
OLMo 2 Instruct	7B	Paper	Hub	Apache 2.0	✅
MPT-Instruct	7B	Blog	Hub	Apache 2.0	✅
Command R	35B/104B	Blog	Hub	License	❌
Granite-3.1-Instruct	2B/8B	Paper	Hub	Apache 2.0	❌
Gemma 2 Instruct	2B/9B	Blog	Hub	License	❌
DBRX-Instruct	130B MoE	Blog	Hub	Apache 2.0	❌
Falcon-Instruct	7B/40B	Paper	Hub	Apache 2.0	❌
✅ Llama 4 Scout Instruct	17B (Activated) 109B (Total)	Paper	Hub	License	❌
✅ Llama 4 Maverick Instruct	17B (Activated) 400B (Total)	Paper	Hub	License	❌

Vision-Language Models

Model	Size	Paper	HF Hub	License	Open
✅ Llama 3.2 Vision	11B	Paper	Hub	License	❌
✅ LLaVA-1.5	7B	Paper	Hub	License	❌
✅ Phi-3 Vision	4.2B	Paper	Hub	License	❌
✅ BLIP-2	3.6B	Paper	Hub	MIT	❌
✅ Qwen2-VL	2B	Blog	Hub	License	❌
✅ SmolVLM-Instruct	2B	Blog	Hub	Apache 2.0	✅

Base Models

Model	Size	Paper	HF Hub	License	Open
✅ SmolLM2	135M/360M/1.7B	Blog	Hub	Apache 2.0	✅
✅ Llama 3.2	1B/3B	Paper	Hub	License	❌
✅ Llama 3.1	8B/70B/405B	Paper	Hub	License	❌
✅ GPT-2	124M-1.5B	Paper	Hub	MIT	✅
DeepSeek V2	7B/13B	Blog	Hub	License	❌
Gemma2	2B/9B	Blog	Hub	License	❌
GPT-J	6B	Blog	Hub	Apache 2.0	✅
GPT-NeoX	20B	Paper	Hub	Apache 2.0	✅
Mistral	7B	Paper	Hub	Apache 2.0	❌
Mixtral	8x7B/8x22B	Blog	Hub	Apache 2.0	❌
MPT	7B	Blog	Hub	Apache 2.0	✅
OLMo	1B/7B	Paper	Hub	Apache 2.0	✅
✅ Llama 4 Scout	17B (Activated) 109B (Total)	Paper	Hub	License	❌

Reasoning Models

Model	Size	Paper	HF Hub	License	Open
✅ gpt-oss	20B/120B	Paper	Hub	Apache 2.0	❌
✅ Qwen3	0.6B-32B	Paper	Hub	License	❌
Qwen QwQ	32B	Blog	Hub	License	❌

Code Models

Model	Size	Paper	HF Hub	License	Open
✅ Qwen2.5 Coder	0.5B-32B	Blog	Hub	License	❌
DeepSeek Coder	1.3B-33B	Paper	Hub	License	❌
StarCoder 2	3B/7B/15B	Paper	Hub	License	✅

Math Models

Model	Size	Paper	HF Hub	License	Open
DeepSeek Math	7B	Paper	Hub	License	❌

📖 Documentation

To learn more about all the platform's capabilities, see the Oumi documentation.

🤝 Join the Community!

Oumi is a community-first effort. Whether you are a developer, a researcher, or a non-technical user, all contributions are very welcome!

To contribute to the oumi repository, please check the CONTRIBUTING.md for guidance on how to contribute to send your first Pull Request.
Make sure to join our Discord community to get help, share your experiences, and contribute to the project!
If you are interested in joining one of the community's open-science efforts, check out our open collaboration page.

🙏 Acknowledgements

Oumi makes use of several libraries and tools from the open-source community. We would like to acknowledge and deeply thank the contributors of these projects! ✨ 🌟 💫

📝 Citation

If you find Oumi useful in your research, please consider citing it:

@software{oumi2025,
  author = {Oumi Community},
  title = {Oumi: an Open, End-to-end Platform for Building Large Foundation Models},
  month = {January},
  year = {2025},
  url = {https://github.com/oumi-ai/oumi}
}

📜 License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

Open models are defined as models with fully open weights, training code, and data, and a permissive license. See Open Source Definitions for more information. ↩

For Tasks:

Click tags to check more tools for each tasks

train models fine-tune models curate training data deploy models evaluate models

For Jobs:

machine learning engineer data scientist research scientist ai engineer nlp engineer

Alternative AI tools for oumi

Similar Open Source Tools

oumi

github

: 8.5k

dora

Dataflow-oriented robotic application (dora-rs) is a framework that makes creation of robotic applications fast and simple. Building a robotic application can be summed up as bringing together hardwares, algorithms, and AI models, and make them communicate with each others. At dora-rs, we try to: make integration of hardware and software easy by supporting Python, C, C++, and also ROS2. make communication low latency by using zero-copy Arrow messages. dora-rs is still experimental and you might experience bugs, but we're working very hard to make it stable as possible.

github

: 2.5k

Awesome-LLM-Tabular

This repository is a curated list of research papers that explore the integration of Large Language Model (LLM) technology with tabular data. It aims to provide a comprehensive resource for researchers and practitioners interested in this emerging field. The repository includes papers on a wide range of topics, including table-to-text generation, table question answering, and tabular data classification. It also includes a section on related datasets and resources.

github

: 335

visionOS-examples

visionOS-examples is a repository containing accelerators for Spatial Computing. It includes examples such as Local Large Language Model, Chat Apple Vision Pro, WebSockets, Anchor To Head, Hand Tracking, Battery Life, Countdown, Plane Detection, Timer Vision, and PencilKit for visionOS. The repository showcases various functionalities and features for Apple Vision Pro, offering tools for developers to enhance their visionOS apps with capabilities like hand tracking, plane detection, and real-time cryptocurrency prices.

github

: 223

TRACE

TRACE is a temporal grounding video model that utilizes causal event modeling to capture videos' inherent structure. It presents a task-interleaved video LLM model tailored for sequential encoding/decoding of timestamps, salient scores, and textual captions. The project includes various model checkpoints for different stages and fine-tuning on specific datasets. It provides evaluation codes for different tasks like VTG, MVBench, and VideoMME. The repository also offers annotation files and links to raw videos preparation projects. Users can train the model on different tasks and evaluate the performance based on metrics like CIDER, METEOR, SODA_c, F1, mAP, Hit@1, etc. TRACE has been enhanced with trace-retrieval and trace-uni models, showing improved performance on dense video captioning and general video understanding tasks.

github

: 54

nx

Nx is a build system optimized for monorepos, featuring AI-powered architectural awareness and advanced CI capabilities. It provides faster task scheduling, caching, and more for existing workspaces. Nx Cloud enhances CI by offering remote caching, task distribution, automated e2e test splitting, and task flakiness detection. The tool aims to scale monorepos efficiently and improve developer productivity.

github

: 26.9k

chat-master

ChatMASTER is a self-built backend conversation service based on AI large model APIs, supporting synchronous and streaming responses with perfect printer effects. It supports switching between mainstream models such as DeepSeek, Kimi, Doubao, OpenAI, Claude3, Yiyan, Tongyi, Xinghuo, ChatGLM, Shusheng, and more. It also supports loading local models and knowledge bases using Ollama and Langchain, as well as online API interfaces like Coze and Gitee AI. The project includes Java server-side, web-side, mobile-side, and management background configuration. It provides various assistant types for prompt output and allows creating custom assistant templates in the management background. The project uses technologies like Spring Boot, Spring Security + JWT, Mybatis-Plus, Lombok, Mysql & Redis, with easy-to-understand code and comprehensive permission control using JWT authentication system for multi-terminal support.

github

: 99

PocketFlow

Pocket Flow is a 100-line minimalist LLM framework designed for (Multi-)Agents, Workflow, RAG, etc. It provides a core abstraction for LLM projects by focusing on computation and communication through a graph structure and shared store. The framework aims to support the development of LLM Agents, such as Cursor AI, by offering a minimal and low-level approach that is well-suited for understanding and usage. Users can install Pocket Flow via pip or by copying the source code, and detailed documentation is available on the project website.

github

: 1.8k

phoenix

Phoenix is a tool that provides MLOps and LLMOps insights at lightning speed with zero-config observability. It offers a notebook-first experience for monitoring models and LLM Applications by providing LLM Traces, LLM Evals, Embedding Analysis, RAG Analysis, and Structured Data Analysis. Users can trace through the execution of LLM Applications, evaluate generative models, explore embedding point-clouds, visualize generative application's search and retrieval process, and statistically analyze structured data. Phoenix is designed to help users troubleshoot problems related to retrieval, tool execution, relevance, toxicity, drift, and performance degradation.

github

: 6.9k

web-builder

Web Builder is a low-code front-end framework based on Material for Angular, offering a rich component library for excellent digital innovation experience. It allows rapid construction of modern responsive UI, multi-theme, multi-language web pages through drag-and-drop visual configuration. The framework includes a beautiful admin theme, complete front-end solutions, and AI integration in the Pro version for optimizing copy, creating components, and generating pages with a single sentence.

github

: 494

awesome-llm-webapps

This repository is a curated list of open-source, actively maintained web applications that leverage large language models (LLMs) for various use cases, including chatbots, natural language interfaces, assistants, and question answering systems. The projects are evaluated based on key criteria such as licensing, maintenance status, complexity, and features, to help users select the most suitable starting point for their LLM-based applications. The repository welcomes contributions and encourages users to submit projects that meet the criteria or suggest improvements to the existing list.

github

: 173

Awesome-LLMOps

github

: 4.3k

cs-books

CS Books is a curated collection of computer science resources organized by topics and real-world applications. It provides a dual academic/practical focus for students, researchers, and industry professionals. The repository contains a variety of books covering topics such as computer architecture, computer programming, artificial intelligence, data science, cloud computing, edge computing, embedded systems, signal processing, automotive, cybersecurity, game development, healthcare, and robotics. Each section includes a curated list of books with reference links to their Google Drive folders, allowing users to access valuable resources in these fields.

github

: 52

fastapi

智元 Fast API is a one-stop API management system that unifies various LLM APIs in terms of format, standards, and management, achieving the ultimate in functionality, performance, and user experience. It supports various models from companies like OpenAI, Azure, Baidu, Keda Xunfei, Alibaba Cloud, Zhifu AI, Google, DeepSeek, 360 Brain, and Midjourney. The project provides user and admin portals for preview, supports cluster deployment, multi-site deployment, and cross-zone deployment. It also offers Docker deployment, a public API site for registration, and screenshots of the admin and user portals. The API interface is similar to OpenAI's interface, and the project is open source with repositories for API, web, admin, and SDK on GitHub and Gitee.

github

: 245

LLamaTuner

LLamaTuner is a repository for the Efficient Finetuning of Quantized LLMs project, focusing on building and sharing instruction-following Chinese baichuan-7b/LLaMA/Pythia/GLM model tuning methods. The project enables training on a single Nvidia RTX-2080TI and RTX-3090 for multi-round chatbot training. It utilizes bitsandbytes for quantization and is integrated with Huggingface's PEFT and transformers libraries. The repository supports various models, training approaches, and datasets for supervised fine-tuning, LoRA, QLoRA, and more. It also provides tools for data preprocessing and offers models in the Hugging Face model hub for inference and finetuning. The project is licensed under Apache 2.0 and acknowledges contributions from various open-source contributors.

github

: 586

llm-book

The 'llm-book' repository is dedicated to the introduction of large-scale language models, focusing on natural language processing tasks. The code is designed to run on Google Colaboratory and utilizes datasets and models available on the Hugging Face Hub. Note that as of July 28, 2023, there are issues with the MARC-ja dataset links, but an alternative notebook using the WRIME Japanese sentiment analysis dataset has been added. The repository covers various chapters on topics such as Transformers, fine-tuning language models, entity recognition, summarization, document embedding, question answering, and more.

github

: 291

For similar tasks

oumi

github

: 8.5k

ai-on-gke

This repository contains assets related to AI/ML workloads on Google Kubernetes Engine (GKE). Run optimized AI/ML workloads with Google Kubernetes Engine (GKE) platform orchestration capabilities. A robust AI/ML platform considers the following layers: Infrastructure orchestration that support GPUs and TPUs for training and serving workloads at scale Flexible integration with distributed computing and data processing frameworks Support for multiple teams on the same infrastructure to maximize utilization of resources

github

: 280

ray

Ray is a unified framework for scaling AI and Python applications. It consists of a core distributed runtime and a set of AI libraries for simplifying ML compute, including Data, Train, Tune, RLlib, and Serve. Ray runs on any machine, cluster, cloud provider, and Kubernetes, and features a growing ecosystem of community integrations. With Ray, you can seamlessly scale the same code from a laptop to a cluster, making it easy to meet the compute-intensive demands of modern ML workloads.

github

: 38.8k

labelbox-python

Labelbox is a data-centric AI platform for enterprises to develop, optimize, and use AI to solve problems and power new products and services. Enterprises use Labelbox to curate data, generate high-quality human feedback data for computer vision and LLMs, evaluate model performance, and automate tasks by combining AI and human-centric workflows. The academic & research community uses Labelbox for cutting-edge AI research.

github

: 135

djl

Deep Java Library (DJL) is an open-source, high-level, engine-agnostic Java framework for deep learning. It is designed to be easy to get started with and simple to use for Java developers. DJL provides a native Java development experience and allows users to integrate machine learning and deep learning models with their Java applications. The framework is deep learning engine agnostic, enabling users to switch engines at any point for optimal performance. DJL's ergonomic API interface guides users with best practices to accomplish deep learning tasks, such as running inference and training neural networks.

github

: 4.1k

mlflow

MLflow is a platform to streamline machine learning development, including tracking experiments, packaging code into reproducible runs, and sharing and deploying models. MLflow offers a set of lightweight APIs that can be used with any existing machine learning application or library (TensorFlow, PyTorch, XGBoost, etc), wherever you currently run ML code (e.g. in notebooks, standalone applications or the cloud). MLflow's current components are: * `MLflow Tracking `_: An API to log parameters, code, and results in machine learning experiments and compare them using an interactive UI. * `MLflow Projects `_: A code packaging format for reproducible runs using Conda and Docker, so you can share your ML code with others. * `MLflow Models `_: A model packaging format and tools that let you easily deploy the same model (from any ML library) to batch and real-time scoring on platforms such as Docker, Apache Spark, Azure ML and AWS SageMaker. * `MLflow Model Registry `_: A centralized model store, set of APIs, and UI, to collaboratively manage the full lifecycle of MLflow Models.

github

: 21.9k

tt-metal

TT-NN is a python & C++ Neural Network OP library. It provides a low-level programming model, TT-Metalium, enabling kernel development for Tenstorrent hardware.

github

: 1.1k

burn

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

github

: 10.2k

For similar jobs

sweep

Sweep is an AI junior developer that turns bugs and feature requests into code changes. It automatically handles developer experience improvements like adding type hints and improving test coverage.

github

: 7.1k

teams-ai

The Teams AI Library is a software development kit (SDK) that helps developers create bots that can interact with Teams and Microsoft 365 applications. It is built on top of the Bot Framework SDK and simplifies the process of developing bots that interact with Teams' artificial intelligence capabilities. The SDK is available for JavaScript/TypeScript, .NET, and Python.

github

: 502

ai-guide

This guide is dedicated to Large Language Models (LLMs) that you can run on your home computer. It assumes your PC is a lower-end, non-gaming setup.

github

: 159

classifai

Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence. Tap into leading cloud-based services like OpenAI, Microsoft Azure AI, Google Gemini and IBM Watson to augment your WordPress-powered websites. Publish content faster while improving SEO performance and increasing audience engagement. ClassifAI integrates Artificial Intelligence and Machine Learning technologies to lighten your workload and eliminate tedious tasks, giving you more time to create original content that matters.

github

: 620

chatbot-ui

Chatbot UI is an open-source AI chat app that allows users to create and deploy their own AI chatbots. It is easy to use and can be customized to fit any need. Chatbot UI is perfect for businesses, developers, and anyone who wants to create a chatbot.

github

: 27.7k

BricksLLM

BricksLLM is a cloud native AI gateway written in Go. Currently, it provides native support for OpenAI, Anthropic, Azure OpenAI and vLLM. BricksLLM aims to provide enterprise level infrastructure that can power any LLM production use cases. Here are some use cases for BricksLLM: * Set LLM usage limits for users on different pricing tiers * Track LLM usage on a per user and per organization basis * Block or redact requests containing PIIs * Improve LLM reliability with failovers, retries and caching * Distribute API keys with rate limits and cost limits for internal development/production use cases * Distribute API keys with rate limits and cost limits for students

github

: 953

uAgents

uAgents is a Python library developed by Fetch.ai that allows for the creation of autonomous AI agents. These agents can perform various tasks on a schedule or take action on various events. uAgents are easy to create and manage, and they are connected to a fast-growing network of other uAgents. They are also secure, with cryptographically secured messages and wallets.

github

: 1.3k

griptape

Griptape is a modular Python framework for building AI-powered applications that securely connect to your enterprise data and APIs. It offers developers the ability to maintain control and flexibility at every step. Griptape's core components include Structures (Agents, Pipelines, and Workflows), Tasks, Tools, Memory (Conversation Memory, Task Memory, and Meta Memory), Drivers (Prompt and Embedding Drivers, Vector Store Drivers, Image Generation Drivers, Image Query Drivers, SQL Drivers, Web Scraper Drivers, and Conversation Memory Drivers), Engines (Query Engines, Extraction Engines, Summary Engines, Image Generation Engines, and Image Query Engines), and additional components (Rulesets, Loaders, Artifacts, Chunkers, and Tokenizers). Griptape enables developers to create AI-powered applications with ease and efficiency.

github

: 2.2k