swirl-search

swirl-search

SWIRL AI Connect: AI infrastructure software that powers your Search & Retrieval Augmented Generation (RAG) applications. Simplify and enhance your AI pipelines with seamless integration of large language models (LLMs) and data sources.

Stars: 1705

Visit
 screenshot

Swirl is an open-source software that allows users to simultaneously search multiple content sources and receive AI-ranked results. It connects to various data sources, including databases, public data services, and enterprise sources, and utilizes AI and LLMs to generate insights and answers based on the user's data. Swirl is easy to use, requiring only the download of a YML file, starting in Docker, and searching with Swirl. Users can add credentials to preloaded SearchProviders to access more sources. Swirl also offers integration with ChatGPT as a configured AI model. It adapts and distributes user queries to anything with a search API, re-ranking the unified results using Large Language Models without extracting or indexing anything. Swirl includes five Google Programmable Search Engines (PSEs) to get users up and running quickly. Key features of Swirl include Microsoft 365 integration, SearchProvider configurations, query adaptation, synchronous or asynchronous search federation, optional subscribe feature, pipelining of Processor stages, results stored in SQLite3 or PostgreSQL, built-in Query Transformation support, matching on word stems and handling of stopwords, duplicate detection, re-ranking of unified results using Cosine Vector Similarity, result mixers, page through all results requested, sample data sets, optional spell correction, optional search/result expiration service, easily extensible Connector and Mixer objects, and a welcoming community for collaboration and support.

README:

Whitepaper

Swirl

SWIRL AI Connect

Bring AI to the Data, not the Data to the AI

SWIRL AI Connect is advanced AI infrastructure software. It supports enhanced Retrieval Augmented Generation (RAG) capabilities, powerful analytics, and SWIRL Co-Pilot. SWIRL harnesses AI for business, enabling organizations to make better decisions and take more effective and timely actions.

Start Searching · Slack · Key Features · Contribute · Documentation · Connectors


License: Apache 2.0 GitHub Release

Website SWIRL Slack

Test and Build Pipeline

Get your AI up and running in minutes, not months. SWIRL AI Connect is an open-source AI Connect platform that streamlines the integration of advanced AI technologies into business operations. It supports powerful features like Retrieval-Augmented Generation (RAG), Analytics, and Co-Pilot, enabling enhanced decision-making with AI and boosting enterprise AI transformation.

SWIRL operated without needing to move data into a vector database or undergo ETL processes. This approach not only enhances security but also speeds up the deployment. As a private cloud AI provider, SWIRL operates entirely within your private cloud infrastructure, running locally inside the firewall to ensure maximum data security and compliance.

Bringing AI to the Data Newsletter

Join SWIRL Community Slack

Why SWIRL AI Connect?

  • Instant AI Deployment: Swiftly deploy AI-driven enterprise software within your private cloud environment. SWIRL AI Connect integrates seamlessly, offering built-in security measures like data compliance and firewall protections, ensuring secure AI connectivity and granular access control.

  • Easy and Fast Retrieval Augmented Generation(RAG): SWIRL AI Connect simplifies the use of Retrieval-Augmented Generation (RAG). Our platform eliminates the need for external vector databases, LangChain, or LlamaIndex, making implementing AI RAG tools directly on your data easier.

  • No Data Movement: Operate directly on local data without the hassles of ETL processes, re-indexing, or data movement. SWIRL AI Connect enhances data security by allowing the data to remain in place and run securely inside your firewall.

  • Boost Productivity with AI: Enhance team efficiency and streamline workflows with advanced analytics and Co-Pilot features. SWIRL AI Connect helps you find information faster and make smarter decisions, accelerating enterprise AI transformation and boosting productivity.

SWIRL AI Connect enables you to perform Unified Search and bring in a secure AI Co-Pilot.

SWIRL Unified Search: SWIRL Unified Search offers a secure and powerful integrated search solution that enables users to query across all enterprise data sources seamlessly. This scalable unified search platform is designed for large enterprises, startups, and small teams, allowing for comprehensive searches across cloud services, on-premise systems, and data silos without compromising security. By implementing SWIRL Unified Search, businesses can enhance productivity, improve data accessibility, and make more informed decisions by harnessing the full potential of their data landscape.

SWIRL Co-Pilot: SWIRL Co-Pilot acts as an intelligent assistant, leveraging advanced AI to provide context-aware insights and support to business users. Securely integrated within your enterprise systems, SWIRL Co-Pilot helps streamline workflows, automate tasks, and deliver personalized recommendations, significantly boosting operational efficiency. Users benefit from real-time decision support, reduced manual workload, and a more intuitive interaction with their data, enabling them to focus on strategic activities that drive business growth.

Schedule Your Free Demo of SWIRL Co-Pilot

SWIRL Co-Pilot

SWIRL's Ranking in Action

SWIRL leverages the specific context of your enterprise data to deliver highly relevant search results tailored to business needs. While general search engines like Google offer broad capabilities, SWIRL excels in the precise and secure handling of enterprise-specific queries, providing actionable insights that enhance decision-making and business efficiency.

SWIRL vs Google Ranking

SWIRL AI Connect Features

Features 1 Features 2

Based on SWIRL AI Connect

SWIRL AI Co-pilot and SWIRL DB

🔎 How Swirl Works

SWIRL AI Connect offers a straightforward no-code setup to easily integrate AI capabilities into your enterprise. It connects directly to various enterprise and data applications—like Teams, Snowflake, Databricks, and Google Drive—enabling you to search, fetch, and build an AI-based knowledge base. Utilize SWIRL’s Co-Pilot and Retrieval-Augmented Generation (RAG) to enhance productivity without the need for extracting or indexing any data.

  1. Connect: Easily link SWIRL AI Connect to your data sources—be it databases, document stores, or cloud services. Simply add your authentication details to start.

  2. Query: Interact with SWIRL AI Connect using natural language. Ask questions or input commands to immediately harness the power of AI in your workflows.

  3. Get Results: Benefit from SWIRL AI Connect’s advanced search capabilities combined with generative AI. It quickly delivers accurate and contextually augmented responses by distributing queries across connected platforms that have a search API—ranging from search engines and databases to noSQL engines and SaaS services.


🔌 List of Connectors

GitHub Connectors

Full list of connectors is available here.

For Enterprise Support on Connectors Contact the Swirl Team at: [email protected]


🔥 Try Swirl Now In Docker

Prerequisites

Start Swirl in Docker

Warning Make sure the Docker app is running before proceeding!

curl https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.yaml -o docker-compose.yaml
  • Optional: To enable Swirl's Real-Time Retrieval Augmented Generation (RAG) in Docker, run the following commands from the Console using a valid OpenAI API key:
export MSAL_CB_PORT=8000
export MSAL_HOST=localhost
export OPENAI_API_KEY=‘<your-OpenAI-API-key>

🔑 Check out OpenAI's YouTube video if you don't have an OpenAI API Key.

  • In MacOS or Linux, run the following command from the Console:
docker-compose pull && docker-compose up
  • In Windows, run the following command from PowerShell:
docker compose up

After a few minutes the following or similar should appear:

  • Open this URL with a browser: http://localhost:8000 (or http://localhost:8000/galaxy)

  • If the search page appears, click Log Out at the top, right. The Swirl login page will appear.

  • Enter the username admin and password password, then click Login.

  • Enter a search in the search box and press the Search button. Ranked results appear in just a few seconds:

The most recent Search object will be displayed at the top. Click on the result_url link to view the full JSON Response.

Notes 📝

Warning The Docker version of Swirl does not retain any data or configuration when shut down!

🔑 Swirl includes five (5) Google Programmable Search Engines (PSEs) to get you up and running right away. The credentials for these are shared with the Swirl Community.

🔑 Using Swirl with Microsoft 365 requires installation and approval by an authorized company Administrator. For more information, please review the M365 Guide or contact us.

Next Steps 👇

Video Tutorial

Guide to Run SWIRL in Docker in 60 seconds.


🌟 Key Features

Feature
📌 Microsoft 365 integration and OAUTH2 support
🔍 SearchProvider configurations for all included Connectors. They can be organized with the active, default and tags properties.
✏️ Adaptation of the query for each provider such as rewriting NOT term to -term, removing NOTted terms from providers that don't support NOT, and passing down the AND, + and OR operators.
Synchronous or asynchronous search federation via APIs
🛎️ Optional subscribe feature to continuously monitor any search for new results
🛠️ Pipelining of Processor stages for real-time adaptation and transformation of queries, responses and results
🗄️ Results stored in SQLite3 or PostgreSQL for post-processing, consumption and/or analytics
➡️ Built-in Query Transformation support, including re-writing and replacement
📖 Matching on word stems and handling of stopwords via NLTK
🚫 Duplicate detection on field or by configurable Cosine Similarity threshold
🔄 Re-ranking of unified results using Cosine Vector Similarity based on spaCy's large language model and NLTK
🎚️ Result mixers order results by relevancy, date or round-robin (stack) format, with optional filtering of just new items in subscribe mode
📄 Page through all results requested, re-run, re-score and update searches using URLs provided with each result set
📁 Sample data sets for use with SQLite3 and PostgreSQL
✒️ Optional spell correction using TextBlob
Optional search/result expiration service to limit storage use
🔌 Easily extensible Connector and Mixer objects

👩‍💻 Contributing to Swirl

Do you have a brilliant idea or improvement for SWIRL? We're all ears, and thrilled you're here to help!

🔗 Get Started in 3 Easy Steps:

  1. Connect with Fellow Enthusiasts - Jump into the Swirl Slack Community and share your ideas. You'll find a welcoming group of Swirl enthusiasts and team members eager to assist and collaborate.
  2. Branch It Out - Always branch off from the develop branch with a descriptive name that encapsulates your idea or fix.
  3. Start Your Contribution - Ready to get your hands dirty? Make sure all contributions come through a GitHub pull request. We roughly follow the Gitflow branching model, so all changes destined for the next release should be made to the develop branch.

📚 First time contributing on GitHub? No worries, the GitHub documentation has you covered with a great guide on contributing to projects.

💡 Every contribution, big or small, makes a difference. Join us in shaping the future of Swirl!


☁ Use the Swirl Cloud

For information about Swirl as a managed service, please contact us!


📖 Documentation

🔗 SWIRL Documentation


👷‍♂️ Need Help? We're Here for You!

At Swirl, every user matters to us. Whether you're a beginner finding your way or an expert with feedback, we're here to support, listen, and help. Don't hesitate to reach out to us.

  • Join the SWIRL Community Slack: Dive into our SWIRL Community on Slack - to discuss anything related to SWIRL.

  • Direct Support: For any questions, suggestions, or even a simple hello, drop us an email at [email protected]. We cherish every message and promise to get back to you promptly!

  • Request A Connector (Enterprise Support) Want to see a new connector quickly and fast. Contact the Swirl Team at: [email protected]

For Tasks:

Click tags to check more tools for each tasks

For Jobs:

Alternative AI tools for swirl-search

Similar Open Source Tools

For similar tasks

For similar jobs