vearch

vearch

Distributed vector search for AI-native applications

Stars: 1989

Visit
 screenshot

Vearch is a cloud-native distributed vector database designed for efficient similarity search of embedding vectors in AI applications. It supports hybrid search with vector search and scalar filtering, offers fast vector retrieval from millions of objects in milliseconds, and ensures scalability and reliability through replication and elastic scaling out. Users can deploy Vearch cluster on Kubernetes, add charts from the repository or locally, start with Docker-compose, or compile from source code. The tool includes components like Master for schema management, Router for RESTful API, and PartitionServer for hosting document partitions with raft-based replication. Vearch can be used for building visual search systems for indexing images and offers a Python SDK for easy installation and usage. The tool is suitable for AI developers and researchers looking for efficient vector search capabilities in their applications.

README:

简体中文 | English

License: Apache-2.0 Build Status Go Report Card Gitter

Overview

Vearch is a cloud-native distributed vector database for efficient similarity search of embedding vectors in your AI applications.

Key features

  • Hybrid search: Both vector search and scalar filtering.

  • Performance: Fast vector retrieval - search from millions of objects in milliseconds.

  • Scalability & Reliability: Replication and elastic scaling out.

Document

Restful APIs

OpenAPIs

SDK

Usage cases

Use Vearch as a memory backend

Real world Demos

  • VisualSearch: Vearch can be leveraged to build a complete visual search system to index billions of images. The image retrieval plugin for object detection and feature extraction is also required.

Quick start

Deploy vearch cluster on k8s

Add charts through the repo

$ helm repo add vearch https://vearch.github.io/vearch-helm
$ helm repo update && helm install my-release vearch/vearch

Add charts from local

$ git clone https://github.com/vearch/vearch-helm.git && cd vearch-helm
$ helm install my-release ./charts -f ./charts/values.yaml

Start by docker-compose

standalone mode

$ cd cloud
$ cp ../config/config.toml .
$ docker-compose --profile standalone up -d

cluster mode

$ cd cloud
$ cp ../config/config_cluster.toml .
$ docker-compose --profile cluster up -d

Deploy by docker: Quickly start with vearch docker image, please see DeployByDocker

Compile by source code: Quickly compile the source codes, please see SourceCompileDeployment

Components

Vearch Architecture

arc

Master: Responsible for schema mananagement, cluster-level metadata, and resource coordination.

Router: Provides RESTful API: upsert, delete, search and query; request routing, and result merging.

PartitionServer (PS): Hosts document partitions with raft-based replication. Gamma is the core vector search engine implemented based on faiss. It provides the ability of storing, indexing and retrieving the vectors and scalars.

Reference

Reference to cite when you use Vearch in a research paper:

@misc{li2019design,
      title={The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform},
      author={Jie Li and Haifeng Liu and Chuanghua Gui and Jianyu Chen and Zhenyun Ni and Ning Wang},
      year={2019},
      eprint={1908.07389},
      archivePrefix={arXiv},
      primaryClass={cs.IR}
}

Community

You can report bugs or ask questions in the issues page of the repository.

For public discussion of Vearch or for questions, you can also send email to [email protected].

Our slack : https://vearchwrokspace.slack.com

Known Users

Welcome to register the company name in this issue: https://github.com/vearch/vearch/issues/230 (in order of registration)

Users

License

Licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for vearch

Similar Open Source Tools

For similar tasks

For similar jobs