metaflow

metaflow

Open Source Platform for developing, scaling and deploying serious ML, AI, and data science systems

Stars: 8114

Visit
 screenshot

Metaflow is a user-friendly library designed to assist scientists and engineers in developing and managing real-world data science projects. Initially created at Netflix, Metaflow aimed to enhance the productivity of data scientists working on diverse projects ranging from traditional statistics to cutting-edge deep learning. For further information, refer to Metaflow's website and documentation.

README:

Metaflow_Logo_Horizontal_FullColor_Ribbon_Dark_RGB

Metaflow

Metaflow is a human-friendly library that helps scientists and engineers build and manage real-life data science projects. Metaflow was originally developed at Netflix to boost productivity of data scientists who work on a wide variety of projects from classical statistics to state-of-the-art deep learning.

For more information, see Metaflow's website and documentation.

From prototype to production (and back)

Metaflow provides a simple, friendly API that covers foundational needs of ML, AI, and data science projects:

  1. Rapid local prototyping, support for notebooks, and built-in experiment tracking and versioning.
  2. Horizontal and vertical scalability to the cloud, utilizing both CPUs and GPUs, and fast data access.
  3. Managing dependencies and one-click deployments to highly available production orchestrators.

Getting started

Getting up and running is easy. If you don't know where to start, Metaflow sandbox will have you running and exploring Metaflow in seconds.

Installing Metaflow in your Python environment

To install Metaflow in your local environment, you can install from PyPi:

pip install metaflow

Alternatively, you can also install from conda-forge:

conda install -c conda-forge metaflow

If you are eager to try out Metaflow in practice, you can start with the tutorial. After the tutorial, you can learn more about how Metaflow works here.

Deploying infrastructure for Metaflow in your cloud

While you can get started with Metaflow easily on your laptop, the main benefits of Metaflow lie in its ability to scale out to external compute clusters and to deploy to production-grade workflow orchestrators. To benefit from these features, follow this guide to configure Metaflow and the infrastructure behind it appropriately.

An active community of thousands of data scientists and ML engineers discussing the ins-and-outs of applied machine learning.

Get in touch

There are several ways to get in touch with us:

Contributing

We welcome contributions to Metaflow. Please see our contribution guide for more details.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for metaflow

Similar Open Source Tools

For similar tasks

For similar jobs