beta9

beta9

Ultrafast serverless GPU inference, sandboxes, and background jobs

Stars: 1298

Visit
 screenshot

Beta9 is an open-source platform for running scalable serverless GPU workloads across cloud providers. It allows users to scale out workloads to thousands of GPU or CPU containers, achieve ultrafast cold-start for custom ML models, automatically scale to zero to pay for only what is used, utilize flexible distributed storage, distribute workloads across multiple cloud providers, and easily deploy task queues and functions using simple Python abstractions. The platform is designed for launching remote serverless containers quickly, featuring a custom, lazy loading image format backed by S3/FUSE, a fast redis-based container scheduling engine, content-addressed storage for caching images and files, and a custom runc container runtime.

README:

Logo Logo

Run AI Workloads at Scale

Colab ⭐ Star the Repo Documentation Join Slack Twitter AGPL

Beam is a fast, open-source runtime for serverless AI workloads. It gives you a Pythonic interface to deploy and scale AI applications with zero infrastructure overhead.

Watch the demo

✨ Features

  • Fast Image Builds: Launch containers in under a second using a custom container runtime
  • Parallelization and Concurrency: Fan out workloads to 100s of containers
  • First-Class Developer Experience: Hot-reloading, webhooks, and scheduled jobs
  • Scale-to-Zero: Workloads are serverless by default
  • Volume Storage: Mount distributed storage volumes
  • GPU Support: Run on our cloud (4090s, H100s, and more) or bring your own GPUs

📦 Installation

pip install beam-client

⚡️ Quickstart

  1. Create an account here
  2. Follow our Getting Started Guide

Creating a sandbox

Spin up isolated containers to run LLM-generated code:

from beam import Image, Sandbox


sandbox = Sandbox(image=Image()).create()
response = sandbox.process.run_code("print('I am running remotely')")

print(response.result)

Deploy a serverless inference endpoint

Create an autoscaling endpoint for your custom model:

from beam import Image, endpoint
from beam import QueueDepthAutoscaler

@endpoint(
    image=Image(python_version="python3.11"),
    gpu="A10G",
    cpu=2,
    memory="16Gi",
    autoscaler=QueueDepthAutoscaler(max_containers=5, tasks_per_container=30)
)
def handler():
    return {"label": "cat", "confidence": 0.97}

Run background tasks

Schedule resilient background tasks (or replace your Celery queue) by adding a simple decorator:

from beam import Image, TaskPolicy, schema, task_queue


class Input(schema.Schema):
    image_url = schema.String()


@task_queue(
    name="image-processor",
    image=Image(python_version="python3.11"),
    cpu=1,
    memory=1024,
    inputs=Input,
    task_policy=TaskPolicy(max_retries=3),
)
def my_background_task(input: Input, *, context):
    image_url = input.image_url
    print(f"Processing image: {image_url}")
    return {"image_url": image_url}


if __name__ == "__main__":
    # Invoke a background task from your app (without deploying it)
    my_background_task.put(image_url="https://example.com/image.jpg")

    # You can also deploy this behind a versioned endpoint with:
    # beam deploy app.py:my_background_task --name image-processor

Self-Hosting vs Cloud

Beta9 is the open-source engine powering Beam, our fully-managed cloud platform. You can self-host Beta9 for free or choose managed cloud hosting through Beam.

👋 Contributing

We welcome contributions big or small. These are the most helpful things for us:

❤️ Thanks to Our Contributors

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for beta9

Similar Open Source Tools

For similar tasks

For similar jobs