denser-retriever

denser-retriever

An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.

Stars: 119

Visit
 screenshot

Denser Retriever is an enterprise-grade AI retriever designed to streamline AI integration into applications, combining keyword-based searches, vector databases, and machine learning rerankers using xgboost. It provides state-of-the-art accuracy on MTEB Retrieval benchmarking and supports various heterogeneous retrievers for end-to-end applications like chatbots and semantic search.

README:

denser logo Denser Retriever

Python Version Dependencies Status

Code style: ruff Security: bandit Pre-commit Semantic Versions License Coverage Report

An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.

๐Ÿ“ Description

Denser Retriever combines multiple search technologies into a single platform. It utilizes gradient boosting ( xgboost) machine learning technique to combine:

  • Keyword-based searches that focus on fetching precisely what the query mentions.
  • Vector databases that are great for finding a wide range of potentially relevant answers.
  • Machine Learning rerankers that fine-tune the results to ensure the most relevant answers top the list.

Our experiments on MTEB datasets show that the combination of keyword search, vector search and a reranker via an xgboost model (denoted as ES+VS+RR_n) can significantly improve the vector search (VS) baseline.

mteb_ndcg_plot

๐Ÿš€ Features

The initial release of Denser Retriever provides the following features.

  • Supporting heterogeneous retrievers such as keyword search, vector search, and ML model reranking
  • Leveraging xgboost ML technique to effectively combine heterogeneous retrievers
  • State-of-the-art accuracy on MTEB Retrieval benchmarking
  • Demonstrating how to use Denser retriever to power an end-to-end applications such as chatbot and semantic search

๐Ÿ“ฆ Installation

We recommend installing Python via Anaconda, as we have received feedback about issues with Numpy installation when using the installer from https://www.python.org/downloads/. We are working on providing a solution to this problem. To install Denser Retriever, you can run:

Pip

pip install git+https://github.com/denser-org/denser-retriever.git#main

Poetry

poetry add git+https://github.com/denser-org/denser-retriever.git#main

๐Ÿ“ƒ Documentation

The official documentation is hosted on retriever.denser.ai. Click here to get started.

๐Ÿ‘จ๐Ÿผโ€๐Ÿ’ป Development

You can start developing Denser Retriever on your local machine.

See DEVELOPMENT.md for more details.

๐Ÿ›ก License

License

This project is licensed under the terms of the MIT license. See LICENSE for more details.

๐Ÿ“ƒ Citation

@misc{denser-retriever,
  author = {denser-org},
  title = {An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/denser-org/denser-retriever}}
}

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for denser-retriever

Similar Open Source Tools

For similar tasks

For similar jobs