kalavai-client

kalavai-client

A platform to crowdsource AI computation

Stars: 72

Visit
 screenshot

Kalavai is an open-source platform that transforms everyday devices into an AI supercomputer by aggregating resources from multiple machines. It facilitates matchmaking of resources for large AI projects, making AI hardware accessible and affordable. Users can create local and public pools, connect with the community's resources, and share computing power. The platform aims to be a management layer for research groups and organizations, enabling users to unlock the power of existing hardware without needing a devops team. Kalavai CLI tool helps manage both versions of the platform.

README:

alt text

GitHub Release GitHub download count GitHub contributors GitHub License GitHub Repo stars Discord Signup

⭐ Kalavai is open-source - Support us by leaving a star ⭐

Kalavai: the first platform to crowdsource AI computation

Run large AI projects that you couldn't with your own hardware alone

Kalavai is an open source platform that turns everyday devices into your very own AI supercomputer. We help you aggregate resources from multiple machines: home desktops, gaming laptops, work computers, cloud VMs... When you need to go beyond, Kalavai facilitates matchmaking of resources so anyone in our community can tap into a larger pool of devices by inspiring others to join your cause.

Kalavai - The first platform to crowdsource AI computation | Product Hunt

Overview of Kalavai

What can Kalavai do?

Kalavai's goal is to make AI hardware accessible and affordable to all. We do it in two ways:

  1. The open source version can be used to pool any devices, for commercial and non-commercial purposes. This is perfect as a management layer for research groups and organisations that already have hardware lying around and wish to unlock its power, without requiring a devops team. This AI pools are free, secure and totally private.

  2. Our managed version acts as a platform to crowdsource AI computing between its members, facilitating users to connect with the community's resources. Think Reddit, but instead of memes, users share resources with inspiring projects.

Both versions can be managed using our free kalavai CLI tool.

Kalavai is at a very early stage of its development. We encourage people to use it and give us feedback! Although we are trying to minimise breaking changes, these may occur until we have a stable version (v1.0).

Want to know more?

News updates

Getting started

The kalavai CLI is the main tool to interact with the Kalavai platform, to create and manage both local and public pools. Let's go over its installation

Requirements

  • A laptop, desktop or Virtual Machine
  • Admin / privileged access (eg. sudo access in linux or Administrator in Windows)
  • Running Windows or Linux (see more details in our compatibility matrix)

Linux

Run the following command on your terminal:

curl -sfL https://raw.githubusercontent.com/kalavai-net/kalavai-client/main/assets/install_client.sh | bash -

Windows

For Windows machines complete WSL configuration first before continuing. You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11 to use the commands below. If you are on earlier versions please see the manual install page.

  1. Open a PowerShell with administrative permissions (Run as Administrator)

  2. Install WSL2:

wsl --install -d Ubuntu-24.04
  1. Make sure to enable systemd by editing (or creating if required) a file /etc/wsl.conf
[boot]
systemd=true
  1. Restart the WSL instance by exiting and logging back in:
exit
wsl --shutdown
wsl -d Ubuntu-24.04
  1. Inside WSL, install Kalavai:
curl -sfL https://raw.githubusercontent.com/kalavai-net/kalavai-client/main/assets/install_client.sh | bash -

Note: you must keep the WSL console window open to continue to share resources with an AI pool. If you restart your machine or close the console, you will need to resume kalavai as follows:

kalavai pool resume

Known issue: if the above resume command hangs or fails, try to run the pause command before and then reattempt resuming:

kalavai pool pause
kalavai pool resume

Createa a local pool

Kalavai is free to use, no caps, for both commercial and non-commercial purposes. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you wish to join computers in different locations / networks, check managed kalavai.

1. Start a seed node

Simply use the CLI to start your seed node:

kalavai pool start <pool-name>

Now you are ready to add worker nodes to this seed. To do so, generate a joining token:

$ kalavai pool token

Join token: <token>

2. Add worker nodes

Increase the power of your AI pool by inviting others to join.

Copy the joining token. On the worker node, run:

kalavai pool join <token>

Public pools: crowdsource community resources

Our public platform expands local pools in two key aspects:

  • Worker nodes no longer have to be in the same local network
  • Users can tap into community resources: inspire others in the community to join their projects with their resources

To get started, you need is a free account on our platform.

A) Tap into community resources

Create a new pool, using a public location provided by Kalavai:

# Authenticate with your kalavai account
kalavai login

# Get available public locations
kalavai location list

┏━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓  
┃ VPN ┃ location    ┃ subnet        ┃          
┡━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ 0   │ uk_london_1 │ 100.10.0.0/16 │
└─────┴─────────────┴───────────────┘

# Create and publish your pool
kalavai pool start <pool-name> --location uk_london_1

If all goes well, your pool will be created and published on the Public Seeds section of our platform

Public seeds

Note: to be able to publish pools your account needs to have sufficient karma points. Earn karma by sharing your resources with others.

B) Share resources with inspiring community projects

Have idle computing resources? Wish to be part of exciting public projects? Want to give back to the community? Earn social credit (both literally and metaphorically) by sharing your computer with others within the community.

All you need is a public joining key. Get them in our platform, on the list of published pools. Press Join and follow the instructions

alt text

What next?

Within a pool, you can monitor the nodes connected and the resources available:

# Get all connected nodes
kalavai node list

# Get all resources within the pool (CPUs, RAM, GPUs...)
kalavai pool resources

Enough already, let's run stuff!

Check our examples to put your new AI pool to good use!

Compatibility matrix

If your system is not currently supported, open an issue and request it. We are expanding this list constantly.

OS compatibility

Currently compatible and tested OS:

  • Ubuntu (22.04, 24.04)
  • Pop! OS 22.04
  • Windows 10+ (using WSL2)

Currently compatible (untested. Interested in testing them?):

  • Debian-based linux
  • Fedora
  • RedHat
  • Any distro capable of installing .deb and .rpm packages.

Currently not compatible:

  • MacOS

Hardware compatibility:

  • amd64 or x86_64 CPU architecture
  • (optional) NVIDIA GPU
  • AMD and Intel GPUs are currently not supported (yet!)

Roadmap

  • [x] Kalavai client on Linux
  • [x] [TEMPLATE] Distributed LLM deployment
  • [x] Kalavai client on Windows (with WSL2)
  • [x] Public pools
  • [ ] [TEMPLATE] Distributed LLM fine tuning
  • [ ] Kalavai client on Mac
  • [x] Ray cluster support

Anything missing here? Give us a shout in the discussion board

Contribute

Star History

Star History Chart

Build from source

Requirements

Python version <= 3.10.

On Ubuntu:

virtualenv -p python3 env
source env/bin/activate
sudo apt install python3-tk python3-dev rpm squashfs-tools ruby-dev build-essential gcc -y
sudo gem i fpm -f
pip install -e .

Build

Run the build process with:

bash build.sh

This will produce two main assets:

  • dist/kalavai as the linux executable CLI application
  • packages/kalavai-cli-* for all compatible package installables.

Unit tests

To run the unit tests, use:

python -m unittest

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for kalavai-client

Similar Open Source Tools

For similar tasks

For similar jobs