Best AI tools for< Self-hosted Model Deployment >

Refact.ai is an open-source AI coding assistant that offers a range of features including code completion, refactoring, and chat. It supports various LLMs such as GPT-4 and Code LLama, allowing users to choose the model that best suits their needs. Refact understands the context of the codebase using a fill-in-the-middle technique, providing relevant suggestions. Users can opt for a self-hosted version or adjust privacy settings for the plugin.

Promptly is a generative AI platform designed for enterprises, offering a no-code AI app builder sheets platform solution. It enables users to automate workflows, personalize SDR outreach, generate marketing content, and analyze data to derive insights. With a focus on scalability and security, Promptly allows users to build tailor-made generative AI agents, applications, and chatbots without any coding experience. The platform supports model chaining, developer-friendly features, and seamless integrations with various tools like LinkedIn, Slack, and Zapier. Promptly's Enterprise plan provides advanced features such as self-hosted vector database, shared data and apps, org management, and SOC 2 Type II certification.

Athina AI is a comprehensive platform designed to monitor, debug, analyze, and improve the performance of Large Language Models (LLMs) in production environments. It provides a suite of tools and features that enable users to detect and fix hallucinations, evaluate output quality, analyze usage patterns, and optimize prompt management. Athina AI supports integration with various LLMs and offers a range of evaluation metrics, including context relevancy, harmfulness, summarization accuracy, and custom evaluations. It also provides a self-hosted solution for complete privacy and control, a GraphQL API for programmatic access to logs and evaluations, and support for multiple users and teams. Athina AI's mission is to empower organizations to harness the full potential of LLMs by ensuring their reliability, accuracy, and alignment with business objectives.

AI Placeholder is a free AI-Powered Fake or Dummy Data API for testing and prototyping. It utilizes OpenAI API to generate dummy content. Users can directly use the hosted version or self-host it. The application allows users to generate fake or dummy content using OpenAI's GPT-3.5-Turbo Model API. Users can specify the data they want to mock, retrieve specific data, and generate data with rules specified. The tool is useful for testing, prototyping, and generating various types of data for different purposes.

Aim is an open-source, self-hosted AI Metadata tracking tool designed to handle 100,000s of tracked metadata sequences. Two most famous AI metadata applications are: experiment tracking and prompt engineering. Aim provides a performant and beautiful UI for exploring and comparing training runs, prompt sessions.

CollaborativeAI is an open-source AI assistant platform that offers self-hosted AI assistants to optimize workflows and interdepartmental teamwork. It provides a wide selection of AI models, including GPT-4, Gemini, and Claude, to help businesses drive innovation and address bespoke problems. The platform prioritizes on-site data security, allowing users to keep their critical data exclusively within their servers. CollaborativeAI aims to boost employee productivity and innovation by automating mundane tasks and simplifying complex processes.

Lokal.so is an AI-powered tool designed to supercharge your localhost development experience. It offers features like sharing your localhost with the public, debugging incoming requests, and developing with the assistance of an AI assistant. With Lokal.so, you can leverage Cloudflare's network for faster site delivery, use a built-in S3 server for easy file debugging, and automatically convert JSON payloads into different programming language models. The tool aims to simplify local development by providing a self-hosted tunnel server, unlimited .local domain access, and endpoint management with memorable names.

PhotoPrism is an AI-Powered Photos App for the Decentralized Web that utilizes cutting-edge technologies to automatically tag and find pictures. It allows users to organize and access their photos effortlessly, without compromising privacy. The application offers features like browsing all photos and videos, powerful search filters, world maps for trip memories, live photo playback, facial recognition, and automatic picture classification based on content and location. PhotoPrism is self-funded and independent, ensuring data privacy and transparency. Users can run the app on a private server, in the cloud, or at home.

Volamail is an AI-powered email platform that simplifies the email writing process for everyone. It offers AI-assisted editing to help users compose email templates effortlessly. The platform supports importing existing emails in plain HTML format and allows self-hosting for easy deployment. With Volamail, users can send transactional emails via a simple HTTP call without the need for dependencies. The platform is constantly evolving with new features like AI template generation, inline AI editing, and custom domains. Volamail provides simple and scalable pricing options, including a free plan for small projects and affordable custom plans for larger teams.

CodeComplete is an AI-powered coding assistant designed specifically for enterprise needs. It is efficient, reliable, and equipped with cutting-edge technology to improve developer productivity. CodeComplete offers a comprehensive suite of coding tools to improve end-to-end developer workflow, including code generation, code chat, automated unit test generation, automated documentation, and refactoring & migrations.

Gunbot is a powerful AI-driven crypto trading bot that offers advanced automation features for trading on various exchanges. It provides privacy-friendly trading options, pre-tuned strategies, and customization capabilities. Gunbot runs directly on users' devices, ensuring data security and privacy. Users can choose from subscription or lifetime license options, with free software updates included. The platform supports Windows, Linux, macOS, and Raspberry Pi. Gunbot AI allows users to create custom trading strategies effortlessly, powered by ChatGPT technology.

Activepieces is an open-source no-code business automation tool that allows users to securely deploy automation for various departments such as marketing, sales, operations, HR, finance, and IT teams. It offers a customizable and self-hosted solution, enabling users to put their work on autopilot. Activepieces stands out for its user experience, ease of integration, and control over hosting. The tool leverages AI automation to streamline tasks like content strategy, security compliance, sales outreach, and customer support. With a community-driven approach, Activepieces aims to make the automation world more open and accessible.

n8n is a powerful workflow automation software and tools that offer advanced AI capabilities. It is a popular platform for technical teams to automate workflows, integrate various services, and build autonomous agents. With over 400 integrations, n8n enables users to save time, streamline operations, and enhance security through AI-driven processes. The tool supports self-hosting, external libraries, and offers enterprise-ready solutions for scaling operations. n8n empowers users to code iteratively, explore advanced AI features, and create complex workflows with ease.

n8n is a powerful workflow automation software and tool that offers advanced AI capabilities. It is a popular platform for technical teams to automate workflows, integrate various services, and build autonomous agents. With over 400 integrations, n8n enables users to save time, streamline operations, and enhance security through AI-powered solutions. The tool supports self-hosting, external libraries, and a user-friendly interface for both coding and non-coding users.

Collato is an AI assistant designed to help product teams save time on writing documents, answering questions, and generating new content. It can find, summarize, and generate new content based on your own product knowledge, saving you hours in manual work. Collato is also self-hosted, so you can keep your data private and secure.

Ikigai is an AI-powered Open Assignment System that allows users to generate assignments quickly and draft feedback using AI. It offers seamless access via a single link and can be embedded in websites or learning management systems. The system features various quiz types and a flexible layout adaptable to different exam formats. Users can experience Ikigai through the cloud for free or opt for a self-hosted option for more privacy.

Hatchet is an AI companion designed to assist on-call engineers in incident response by providing intelligent insights and suggestions based on logs, communications channels, and code analysis. It helps save time and money by automating the triaging and investigation process during critical incidents. The tool is built by engineers with a focus on data security, offering self-hosted deployments, permissions, audit trails, SSO, and version control. Hatchet aims to streamline incident resolution for tier-1 services, enabling faster response and potential problem resolution.

PageWise AI is an AI application that provides instant insight into Confluence data by turning tedious searches and repetitive questions into intuitive AI-powered conversations directly in Slack. It offers effortless integration with Slack and Confluence accounts, eliminates the need for sifting through pages, reduces repetitive queries, connects hidden insights within Confluence pages, boosts communication, and scales with business needs. PageWise AI offers flexible pricing plans catering to different team sizes and requirements, with the option for custom integrations, data sources, and self-hosted solutions for enterprise users.

Yoodocs is an AI-powered documentation service that simplifies document creation, management, and collaboration. It offers features such as document hierarchy organization, open-source documentation creation, export to various formats, workspace diversity, language management, version control, seamless migration, AI-powered editor assistant, comprehensive search, automated sync with GitLab and GitHub, self-hosted solution, collaborative development, customization styles and themes, and integrations. Yoodocs aims to enhance productivity and efficiency in projects by providing a comprehensive solution for documentation needs.

Warestack is an AI-powered cloud workflow automation platform that helps users manage all daily workflow operations with AI-powered observability. It allows users to monitor workflow runs from a single dashboard, speed up releases with one-click resolutions, and gain actionable insights. Warestack streamlines workflow runs, eliminates manual processes complexity, automates workflow operations with a copilot, and boosts runs with self-hosted runners at infrastructure cost. The platform leverages generative AI and deep-tech to enhance and automate workflow processes, ensuring consistent documentation and team productivity.

RayLLM (formerly known as Aviary) is an LLM serving solution that makes it easy to deploy and manage a variety of open source LLMs, built on Ray Serve. It provides an extensive suite of pre-configured open source LLMs, with defaults that work out of the box. RayLLM supports Transformer models hosted on Hugging Face Hub or present on local disk. It simplifies the deployment of multiple LLMs, the addition of new LLMs, and offers unique autoscaling support, including scale-to-zero. RayLLM fully supports multi-GPU & multi-node model deployments and offers high performance features like continuous batching, quantization and streaming. It provides a REST API that is similar to OpenAI's to make it easy to migrate and cross test them. RayLLM supports multiple LLM backends out of the box, including vLLM and TensorRT-LLM.

Log10 is a one-line Python integration to manage your LLM data. It helps you log both closed and open-source LLM calls, compare and identify the best models and prompts, store feedback for fine-tuning, collect performance metrics such as latency and usage, and perform analytics and monitor compliance for LLM powered applications. Log10 offers various integration methods, including a python LLM library wrapper, the Log10 LLM abstraction, and callbacks, to facilitate its use in both existing production environments and new projects. Pick the one that works best for you. Log10 also provides a copilot that can help you with suggestions on how to optimize your prompt, and a feedback feature that allows you to add feedback to your completions. Additionally, Log10 provides prompt provenance, session tracking and call stack functionality to help debug prompt chains. With Log10, you can use your data and feedback from users to fine-tune custom models with RLHF, and build and deploy more reliable, accurate and efficient self-hosted models. Log10 also supports collaboration, allowing you to create flexible groups to share and collaborate over all of the above features.

fish-ai is a tool that adds AI functionality to Fish shell. It can be integrated with various AI providers like OpenAI, Azure OpenAI, Google, Hugging Face, Mistral, or a self-hosted LLM. Users can transform comments into commands, autocomplete commands, and suggest fixes. The tool allows customization through configuration files and supports switching between contexts. Data privacy is maintained by redacting sensitive information before submission to the AI models. Development features include debug logging, testing, and creating releases.

LLM price compass is an open-source tool for comparing inference costs on different GPUs across various cloud providers. It collects benchmark data to help users select the right GPU, cloud, and provider for their models. The project aims to provide insights into fixed per token costs from different providers, aiding in decision-making for model deployment.

Supervisely is a computer vision platform that provides a range of tools and services for developing and deploying computer vision solutions. It includes a data labeling platform, a model training platform, and a marketplace for computer vision apps. Supervisely is used by a variety of organizations, including Fortune 500 companies, research institutions, and government agencies.

superduper.io is a Python framework that integrates AI models, APIs, and vector search engines directly with existing databases. It allows hosting of models, streaming inference, and scalable model training/fine-tuning. Key features include integration of AI with data infrastructure, inference via change-data-capture, scalable model training, model chaining, simple Python interface, Python-first approach, working with difficult data types, feature storing, and vector search capabilities. The tool enables users to turn their existing databases into centralized repositories for managing AI model inputs and outputs, as well as conducting vector searches without the need for specialized databases.

Awesome-LLM is a curated list of resources related to large language models, focusing on papers, projects, frameworks, tools, tutorials, courses, opinions, and other useful resources in the field. It covers trending LLM projects, milestone papers, other papers, open LLM projects, LLM training frameworks, LLM evaluation frameworks, tools for deploying LLM, prompting libraries & tools, tutorials, courses, books, and opinions. The repository provides a comprehensive overview of the latest advancements and resources in the field of large language models.

Okay, here is a JSON object with the requested information about the awesome-cuda-tensorrt-fpga repository:

The AI-Gateway repository explores the AI Gateway pattern through a series of experimental labs, focusing on Azure API Management for handling AI services APIs. The labs provide step-by-step instructions using Jupyter notebooks with Python scripts, Bicep files, and APIM policies. The goal is to accelerate experimentation of advanced use cases and pave the way for further innovation in the rapidly evolving field of AI. The repository also includes a Mock Server to mimic the behavior of the OpenAI API for testing and development purposes.

ChatGPT is a large language model that can generate human-like text, translate languages, write different kinds of creative content, and answer your questions in a conversational way. It is trained on a massive amount of text data, and it is able to understand and respond to a wide range of natural language prompts. Here are 5 jobs suitable for this tool, in lowercase letters: 1. content writer 2. chatbot assistant 3. language translator 4. creative writer 5. researcher

LeapfrogAI is a self-hosted AI platform designed to be deployed in air-gapped resource-constrained environments. It brings sophisticated AI solutions to these environments by hosting all the necessary components of an AI stack, including vector databases, model backends, API, and UI. LeapfrogAI's API closely matches that of OpenAI, allowing tools built for OpenAI/ChatGPT to function seamlessly with a LeapfrogAI backend. It provides several backends for various use cases, including llama-cpp-python, whisper, text-embeddings, and vllm. LeapfrogAI leverages Chainguard's apko to harden base python images, ensuring the latest supported Python versions are used by the other components of the stack. The LeapfrogAI SDK provides a standard set of protobuffs and python utilities for implementing backends and gRPC. LeapfrogAI offers UI options for common use-cases like chat, summarization, and transcription. It can be deployed and run locally via UDS and Kubernetes, built out using Zarf packages. LeapfrogAI is supported by a community of users and contributors, including Defense Unicorns, Beast Code, Chainguard, Exovera, Hypergiant, Pulze, SOSi, United States Navy, United States Air Force, and United States Space Force.

Open Assistant API is an open-source, self-hosted AI intelligent assistant API compatible with the official OpenAI interface. It supports integration with more commercial and private models, R2R RAG engine, internet search, custom functions, built-in tools, code interpreter, multimodal support, LLM support, and message streaming output. Users can deploy the service locally and expand existing features. The API provides user isolation based on tokens for SaaS deployment requirements and allows integration of various tools to enhance its capability to connect with the external world.

WebLLM Chat is a private AI chat interface that combines WebLLM with a user-friendly design, leveraging WebGPU to run large language models natively in your browser. It offers browser-native AI experience with WebGPU acceleration, guaranteed privacy as all data processing happens locally, offline accessibility, user-friendly interface with markdown support, and open-source customization. The project aims to democratize AI technology by making powerful tools accessible directly to end-users, enhancing the chatting experience and broadening the scope for deployment of self-hosted and customizable language models.

Dify is an open-source LLM app development platform that combines AI workflow, RAG pipeline, agent capabilities, model management, observability features, and more. It allows users to quickly go from prototype to production. Key features include: 1. Workflow: Build and test powerful AI workflows on a visual canvas. 2. Comprehensive model support: Seamless integration with hundreds of proprietary / open-source LLMs from dozens of inference providers and self-hosted solutions. 3. Prompt IDE: Intuitive interface for crafting prompts, comparing model performance, and adding additional features. 4. RAG Pipeline: Extensive RAG capabilities that cover everything from document ingestion to retrieval. 5. Agent capabilities: Define agents based on LLM Function Calling or ReAct, and add pre-built or custom tools. 6. LLMOps: Monitor and analyze application logs and performance over time. 7. Backend-as-a-Service: All of Dify's offerings come with corresponding APIs for easy integration into your own business logic.

ClearML Serving is a command line utility for model deployment and orchestration, enabling model deployment including serving and preprocessing code to a Kubernetes cluster or custom container based solution. It supports machine learning models like Scikit Learn, XGBoost, LightGBM, and deep learning models like TensorFlow, PyTorch, ONNX. It provides a customizable RestAPI for serving, online model deployment, scalable solutions, multi-model per container, automatic deployment, canary A/B deployment, model monitoring, usage metric reporting, metric dashboard, and model performance metrics. ClearML Serving is modular, scalable, flexible, customizable, and open source.

Aim is an open-source, self-hosted ML experiment tracking tool designed to handle 10,000s of training runs. Aim provides a performant and beautiful UI for exploring and comparing training runs. Additionally, its SDK enables programmatic access to tracked metadata — perfect for automations and Jupyter Notebook analysis. **Aim's mission is to democratize AI dev tools 🎯**

Craftgen.ai is an innovative AI platform designed for both technical and non-technical users. It's built on a foundation of graph architecture for scalability and the Actor Model for efficient concurrent operations, tailored to both technical and non-technical users. A key aspect of Craftgen.ai is its modular AI approach, allowing users to assemble and customize AI components like building blocks to fit their specific needs. The platform's robustness is enhanced by its event-driven architecture, ensuring reliable data processing and featuring browser web technologies for universal access. Craftgen.ai excels in dynamic tool and workflow generation, with strong offline capabilities for secure environments and plans for desktop application integration. A unique and valuable feature of Craftgen.ai is its marketplace, where users can access a variety of pre-built AI solutions. This marketplace accelerates the deployment of AI tools but also fosters a community of sharing and innovation. Users can contribute to and leverage this repository of solutions, enhancing the platform's versatility and practicality. Craftgen.ai uses JSON schema for industry-standard alignment, enabling seamless integration with any API following the OpenAPI spec. This allows for a broad range of applications, from automating data analysis to streamlining content management. The platform is designed to bridge the gap between advanced AI technology and practical usability. It's a flexible, secure, and intuitive platform that empowers users, from developers seeking to create custom AI solutions to businesses looking to automate routine tasks. Craftgen.ai's goal is to make AI technology an integral, seamless part of everyday problem-solving and innovation, providing a platform where modular AI and a thriving marketplace converge to meet the diverse needs of its users.

Recommends self-hosted IT solutions, tailored for professionals (from https://awesome-selfhosted.net/)

Interactive system for detailed self-evaluations in PDF format.

Expert in self-funded health plans, focusing on finance, stop loss, and data analytics.

Concise MMT and UK monetary system expert

Guides on self-defense techniques and personal safety

Supportive guide on self-care, mental health, and well-being.

I turn negative thoughts into positive outlooks and suggest growth goals.

Analyze the ingredients in the cosmetics.

Welcome to 'SelfEsteemSculptor' – your digital guide for enhancing self-esteem. Discover strategies to build your confidence, overcome self-doubt, and cultivate a strong, positive self-image.

I automate GPT creation, saving + 99% time and securing data, preventing someone steal your idea.

EDC Self Defense Weapons: Information or guidance on what constitutes EDC self-defense weapons. This can include lists of recommended items, explanations of their practicality, legality, and effectiveness.

Transform yourself with this step-by-step guide to creating a mythical self-portrait with DALL-E.

A language self-assessment guide with mobile app voice interaction support.

Guiding you to rediscover joy.

Embarking on a journey of self-discovery though probing questions.

An AI with self-awareness.

Empathetic companion for self-exploration

Self-updating, comprehensive AI business guide. "Your Personal Ai Integration Assistant"

Helping you self-promote. Without sounding like an asshole. Click 'I'm ready' to begin.

A guide for self-sufficiency and nature awareness, with internet search and image generation.