AI tools for test
Related Jobs:
Related Tools:
TestDriver
TestDriver is an AI-powered testing tool that helps developers automate their testing process. It can be integrated with GitHub and can test anything, right in the GitHub environment. TestDriver is easy to set up and use, and it can help developers save time and effort by offloading testing to AI. It uses Dashcam.io technology to provide end-to-end exploratory testing, allowing developers to see the screen, logs, and thought process as the AI completes its test.
Testim
Testim is an AI-powered UI and functional testing platform that helps accelerate test authoring, reduce test maintenance, and release higher-quality apps faster. It offers a range of features such as fast authoring speed, test stability, root cause analysis, and TestOps, making it an efficient and effective solution for product development teams.
Testsigma
Testsigma is a cloud-based test automation platform that enables teams to create, execute, and maintain automated tests for web, mobile, and API applications. It offers a range of features including natural language processing (NLP)-based scripting, record-and-playback capabilities, data-driven testing, and AI-driven test maintenance. Testsigma integrates with popular CI/CD tools and provides a marketplace for add-ons and extensions. It is designed to simplify and accelerate the test automation process, making it accessible to testers of all skill levels.
testRigor
testRigor is an AI-based test automation tool that allows users to create and execute test cases using plain English instructions. It leverages generative AI in software testing to automate test creation and maintenance, offering features such as no code/codeless testing, web, mobile, and desktop testing, Salesforce automation, and accessibility testing. With testRigor, users can achieve test coverage faster and with minimal maintenance, enabling organizations to reallocate QA engineers to build API tests and increase test coverage significantly. The tool is designed to simplify test automation, reduce QA headaches, and improve productivity by streamlining the testing process.
Testlio
Testlio is a trusted software testing partner that maximizes software testing impact by offering comprehensive solutions for quality challenges. They provide a range of services including manual and automated testing, tailored testing strategies for diverse industries, and a cutting-edge platform for seamless collaboration. Testlio's AI-enhanced solutions help reduce risk in high-stake releases and ensure smarter decision-making. With a focus on quality reliability and efficiency, Testlio is a proven partner for mission-critical quality assurance.
Teste.ai
Teste.ai is an AI-powered platform offering a comprehensive set of software testing tools based on artificial intelligence. It enables users to create test cases, test plans, and step-by-step guides easily from software requirements or documentation. The platform covers functional, usability, performance, security, and API testing, simplifying test data generation and offering specialized prompts for each type of test. Teste.ai aims to streamline the test specification process, optimize tester's time, and enhance testing effectiveness by providing personalized prompts and a variety of testing tools.
TestMarket
TestMarket is an AI-powered sales optimization platform for online marketplace sellers. It offers a range of services to help sellers increase their visibility, boost sales, and improve their overall performance on marketplaces such as Amazon, Etsy, and Walmart. TestMarket's services include product promotion, keyword analysis, Google Ads and SEO optimization, and advertising optimization.
TestCraft
TestCraft is an AI-powered assistant in software testing that leverages the capabilities of GPT-4 to simplify the testing process and enhance product quality. It generates automated tests for various automation frameworks and programming languages, helps in ideation by producing innovative test ideas, ensures project accessibility by identifying potential issues, and streamlines the testing process by transforming test ideas into automated tests. TestCraft aims to make software testing more efficient and effective.
Testmint.ai
Testmint.ai is an online mock test platform designed to help users prepare for competitive exams. It offers a wide range of practice tests and study materials to enhance exam readiness. The platform is user-friendly and provides a simulated exam environment to improve test-taking skills. Testmint.ai aims to assist students and professionals in achieving their academic and career goals by offering a comprehensive and effective exam preparation solution.
TestArmy
TestArmy is an AI-driven software testing platform that offers an army of testing agents to help users achieve software quality by balancing cost, speed, and quality. The platform leverages AI agents to generate Gherkin tests based on user specifications, automate test execution, and provide detailed logs and suggestions for test maintenance. TestArmy is designed for rapid scaling and adaptability to changes in the codebase, making it a valuable tool for both technical and non-technical users.
Testportal
Testportal is an online assessment platform that allows users to create their own tests, quizzes, and exams. It is used by businesses and educational institutions to assess the skills and knowledge of their employees and students. Testportal offers a variety of features, including AI-powered question generation, automatic grading, and comprehensive insights and analytics. It also integrates with Microsoft Teams and provides enterprise-grade security and data protection.
TestFit
TestFit is a real estate feasibility platform that uses AI to help developers, architects, contractors, and brokers evaluate deals and make better decisions. It provides real-time insights into design, cost, and constructability, and integrates with a variety of other software tools. TestFit can help users save time and money, and make more informed decisions about their real estate projects.
Testimonial
Testimonial is an AI-powered tool that helps businesses collect and embed customer testimonials on their website in minutes. It provides an easy-to-use interface that allows users to create beautiful testimonial widgets, customize them to match their brand, and embed them on any web page. Testimonial also offers a range of features to help businesses manage their testimonials, such as the ability to collect testimonials from multiple sources, moderate them before they are published, and track their performance.
TestFit
TestFit is a real estate feasibility platform that helps users maximize site potential and get the right deals done faster. It uses real-time AI for rapid iterations, allowing users to evaluate deals in hours instead of weeks. TestFit also integrates with other software, such as Revit and Enscape, to streamline the design and documentation process.
TestSprite Beta
TestSprite Beta is a free online tool that allows you to create and share animated sprites. With TestSprite Beta, you can easily create custom sprites for your games, websites, or other projects. TestSprite Beta is a powerful tool that offers a wide range of features, including the ability to create and edit sprites, add animations, and export your sprites in a variety of formats.
TestimonialEasy
TestimonialEasy is an AI-powered tool designed to help users easily generate fake testimonials for websites. With the power of GPT-4, users can quickly create convincing testimonials by simply entering a website URL. The tool is built with care and attention to detail by Kadirhan, providing a seamless experience for users looking to enhance their website's credibility.
Quizzly.ai
Quizzly.ai is an AI-powered quiz widget designed to elevate website success by redefining engagement, loyalty, and data for publishers. It utilizes advanced AI algorithms to create interactive quizzes that captivate users, increase engagement rates, and provide valuable insights into audience preferences and behaviors. The application offers customizable quiz sponsorships, incremental monetization opportunities, and seamless integration to enhance user experience and optimize revenue.
FaceSymAI
FaceSymAI is an online tool that utilizes advanced AI algorithms to analyze and determine the symmetry of your face. By uploading a photo, the AI examines your facial features, including the eyes, nose, mouth, and overall structure, to provide an accurate assessment of your facial symmetry. The analysis is based on mathematical and statistical methods, ensuring reliable and precise results. FaceSymAI is designed to be user-friendly and accessible, offering a free service to everyone. The uploaded photos are treated with utmost confidentiality and are not stored or used for any other purpose, ensuring your privacy is respected.
Carbonate
Carbonate is an AI-driven automated end-to-end testing tool that allows users to create auto-healing browser tests without any coding. It understands the behavior of applications and adapts tests accordingly, mimicking real user interactions. The tool features an intelligent recorder that translates user actions into runnable tests, interactive test playback for real-time debugging, and supports dynamic rendering and shadow DOM. Carbonate aims to simplify the testing process and improve efficiency by leveraging AI technology.
DeepUnit
DeepUnit is a software tool designed to facilitate automated unit testing for developers. By utilizing DeepUnit, developers can ensure the quality and reliability of their code by automatically running tests to identify bugs and errors. The tool streamlines the testing process, saving time and effort for developers while improving the overall performance of their software projects. DeepUnit offers a user-friendly interface and seamless integration with popular development environments like NPM and VS Code.
Game QA Strategist
Advises on QA tests based on recent game code changes, including git history. Learn more at regression.gg
Test Shaman
Test Shaman: Guiding software testing with Grug wisdom and humor, balancing fun with practical advice.
Raven's Progressive Matrices Test
Provides Raven's Progressive Matrices test with explanations and calculates your IQ score.
IQ Test Assistant
An AI conducting 30-question IQ tests, assessing and providing detailed feedback.
Test Case GPT
I will provide guidance on testing, verification, and validation for QA roles.
Cognitive Bias Detector
Test ideas for cognitive biases or challenge your own views. Member of the Hipster Energy Team. https://hipster.energy/team
Expert Testers
Chat with Software Testing Experts. Ping Jason if you won't want to be an expert or have feedback.
Security Testing Advisor
Ensures software security through comprehensive testing techniques.
Usability Testing Advisor
Enhances user experience through rigorous usability testing and feedback.
testzeus-hercules
Hercules is the world’s first open-source testing agent designed to handle the toughest testing tasks for modern web applications. It turns simple Gherkin steps into fully automated end-to-end tests, making testing simple, reliable, and efficient. Hercules adapts to various platforms like Salesforce and is suitable for CI/CD pipelines. It aims to democratize and disrupt test automation, making top-tier testing accessible to everyone. The tool is transparent, reliable, and community-driven, empowering teams to deliver better software. Hercules offers multiple ways to get started, including using PyPI package, Docker, or building and running from source code. It supports various AI models, provides detailed installation and usage instructions, and integrates with Nuclei for security testing and WCAG for accessibility testing. The tool is production-ready, open core, and open source, with plans for enhanced LLM support, advanced tooling, improved DOM distillation, community contributions, extensive documentation, and a bounty program.
TestSpark
TestSpark is a plugin for generating unit tests that integrates AI-based test generation tools. It supports LLM-based test generation using OpenAI, HuggingFace, and JetBrains internal AI Assistant platform, as well as local search-based test generation using EvoSuite. Users can configure test generation settings, interact with test cases, view coverage statistics, and integrate tests into projects. The plugin is designed for experimental use to augment existing test suites, not replace manual test writing.
easyAi
EasyAi is a lightweight, beginner-friendly Java artificial intelligence algorithm framework. It can be seamlessly integrated into Java projects with Maven, requiring no additional environment configuration or dependencies. The framework provides pre-packaged modules for image object detection and AI customer service, as well as various low-level algorithm tools for deep learning, machine learning, reinforcement learning, heuristic learning, and matrix operations. Developers can easily develop custom micro-models tailored to their business needs.
promptfoo
Promptfoo is a tool for testing and evaluating LLM output quality. With promptfoo, you can build reliable prompts, models, and RAGs with benchmarks specific to your use-case, speed up evaluations with caching, concurrency, and live reloading, score outputs automatically by defining metrics, use as a CLI, library, or in CI/CD, and use OpenAI, Anthropic, Azure, Google, HuggingFace, open-source models like Llama, or integrate custom API providers for any LLM API.
ps-fuzz
The Prompt Fuzzer is an open-source tool that helps you assess the security of your GenAI application's system prompt against various dynamic LLM-based attacks. It provides a security evaluation based on the outcome of these attack simulations, enabling you to strengthen your system prompt as needed. The Prompt Fuzzer dynamically tailors its tests to your application's unique configuration and domain. The Fuzzer also includes a Playground chat interface, giving you the chance to iteratively improve your system prompt, hardening it against a wide spectrum of generative AI attacks.
empirical
Empirical is a tool that allows you to test different LLMs, prompts, and other model configurations across all the scenarios that matter for your application. With Empirical, you can run your test datasets locally against off-the-shelf models, test your own custom models and RAG applications, view, compare, and analyze outputs on a web UI, score your outputs with scoring functions, and run tests on CI/CD.
askui
AskUI is a reliable, automated end-to-end automation tool that only depends on what is shown on your screen instead of the technology or platform you are running on.
aiverify
AI Verify is an AI governance testing framework and software toolkit that validates the performance of AI systems against a set of internationally recognised principles through standardised tests. AI Verify is consistent with international AI governance frameworks such as those from European Union, OECD and Singapore. It is a single integrated toolkit that operates within an enterprise environment. It can perform technical tests on common supervised learning classification and regression models for most tabular and image datasets. It however does not define AI ethical standards and does not guarantee that any AI system tested will be free from risks or biases or is completely safe.
ChainForge
ChainForge is a visual programming environment for battle-testing prompts to LLMs. It is geared towards early-stage, quick-and-dirty exploration of prompts, chat responses, and response quality that goes beyond ad-hoc chatting with individual LLMs. With ChainForge, you can: * Query multiple LLMs at once to test prompt ideas and variations quickly and effectively. * Compare response quality across prompt permutations, across models, and across model settings to choose the best prompt and model for your use case. * Setup evaluation metrics (scoring function) and immediately visualize results across prompts, prompt parameters, models, and model settings. * Hold multiple conversations at once across template parameters and chat models. Template not just prompts, but follow-up chat messages, and inspect and evaluate outputs at each turn of a chat conversation. ChainForge comes with a number of example evaluation flows to give you a sense of what's possible, including 188 example flows generated from benchmarks in OpenAI evals. This is an open beta of Chainforge. We support model providers OpenAI, HuggingFace, Anthropic, Google PaLM2, Azure OpenAI endpoints, and Dalai-hosted models Alpaca and Llama. You can change the exact model and individual model settings. Visualization nodes support numeric and boolean evaluation metrics. ChainForge is built on ReactFlow and Flask.
llm-workflow-engine
LLM Workflow Engine (LWE) is a powerful command-line interface (CLI) and workflow manager for large language models (LLMs) like ChatGPT and GPT4. It allows users to interact with LLMs directly from their terminal, making it easy to automate tasks and build complex workflows. LWE supports the official ChatGPT API, providing access to all supported models through your OpenAI account. Additionally, it features a simple plugin architecture that enables users to extend its functionality and integrate with other LLMs. LWE also offers a Python API for integrating LLM capabilities into Python scripts. Notable projects built using the original ChatGPT Wrapper, which LWE evolved from, include bookast, ChatGPT.el, ChatGPT Reddit Bot, Smarty GPT, ChatGPTify, and selection-to-chatgpt.
aircrack-ng
Aircrack-ng is a comprehensive suite of tools designed to evaluate the security of WiFi networks. It covers various aspects of WiFi security, including monitoring, attacking (replay attacks, deauthentication, fake access points), testing WiFi cards and driver capabilities, and cracking WEP and WPA PSK. The tools are command line-based, allowing for extensive scripting and have been utilized by many GUIs. Aircrack-ng primarily works on Linux but also supports Windows, macOS, FreeBSD, OpenBSD, NetBSD, Solaris, and eComStation 2.
sql-eval
This repository contains the code that Defog uses for the evaluation of generated SQL. It's based off the schema from the Spider, but with a new set of hand-selected questions and queries grouped by query category. The testing procedure involves generating a SQL query, running both the 'gold' query and the generated query on their respective database to obtain dataframes with the results, comparing the dataframes using an 'exact' and a 'subset' match, logging these alongside other metrics of interest, and aggregating the results for reporting. The repository provides comprehensive instructions for installing dependencies, starting a Postgres instance, importing data into Postgres, importing data into Snowflake, using private data, implementing a query generator, and running the test with different runners.
LLMFarm
LLMFarm is an iOS and MacOS app designed to work with large language models (LLM). It allows users to load different LLMs with specific parameters, test the performance of various LLMs on iOS and macOS, and identify the most suitable model for their projects. The tool is based on ggml and llama.cpp by Georgi Gerganov and incorporates sources from rwkv.cpp by saharNooby, Mia by byroneverson, and LlamaChat by alexrozanski. LLMFarm features support for MacOS (13+) and iOS (16+), various inferences and sampling methods, Metal compatibility (not supported on Intel Mac), model setting templates, LoRA adapters support, LoRA finetune support, LoRA export as model support, and more. It also offers a range of inferences including LLaMA, GPTNeoX, Replit, GPT2, Starcoder, RWKV, Falcon, MPT, Bloom, and others. Additionally, it supports multimodal models like LLaVA, Obsidian, and MobileVLM. Users can customize inference options through JSON files and access supported models for download.
StableToolBench
StableToolBench is a new benchmark developed to address the instability of Tool Learning benchmarks. It aims to balance stability and reality by introducing features such as a Virtual API System with caching and API simulators, a new set of solvable queries determined by LLMs, and a Stable Evaluation System using GPT-4. The Virtual API Server can be set up either by building from source or using a prebuilt Docker image. Users can test the server using provided scripts and evaluate models with Solvable Pass Rate and Solvable Win Rate metrics. The tool also includes model experiments results comparing different models' performance.
cover-agent
CodiumAI Cover Agent is a tool designed to help increase code coverage by automatically generating qualified tests to enhance existing test suites. It utilizes Generative AI to streamline development workflows and is part of a suite of utilities aimed at automating the creation of unit tests for software projects. The system includes components like Test Runner, Coverage Parser, Prompt Builder, and AI Caller to simplify and expedite the testing process, ensuring high-quality software development. Cover Agent can be run via a terminal and is planned to be integrated into popular CI platforms. The tool outputs debug files locally, such as generated_prompt.md, run.log, and test_results.html, providing detailed information on generated tests and their status. It supports multiple LLMs and allows users to specify the model to use for test generation.
aiverify
AI Verify is an AI governance testing framework and software toolkit that validates the performance of AI systems against internationally recognised principles through standardised tests. It offers a new API Connector feature to bypass size limitations, test various AI frameworks, and configure connection settings for batch requests. The toolkit operates within an enterprise environment, conducting technical tests on common supervised learning models for tabular and image datasets. It does not define AI ethical standards or guarantee complete safety from risks or biases.
mistral-inference
Mistral Inference repository contains minimal code to run 7B, 8x7B, and 8x22B models. It provides model download links, installation instructions, and usage guidelines for running models via CLI or Python. The repository also includes information on guardrailing, model platforms, deployment, and references. Users can interact with models through commands like mistral-demo, mistral-chat, and mistral-common. Mistral AI models support function calling and chat interactions for tasks like testing models, chatting with models, and using Codestral as a coding assistant. The repository offers detailed documentation and links to blogs for further information.
agent-evaluation
Agent Evaluation is a generative AI-powered framework for testing virtual agents. It implements an LLM agent (evaluator) to orchestrate conversations with your own agent (target) and evaluate responses. It supports popular AWS services, allows concurrent multi-turn conversations, defines hooks for additional tasks, and can be used in CI/CD pipelines for faster delivery and stable production environments.
aioquic
aioquic is a Python library for the QUIC network protocol, featuring a minimal TLS 1.3 implementation, a QUIC stack, and an HTTP/3 stack. It is designed to be embedded into Python client and server libraries supporting QUIC and HTTP/3, with IPv4 and IPv6 support, connection migration, NAT rebinding, logging TLS traffic secrets and QUIC events, server push, WebSocket bootstrapping, and datagram support. The library follows the 'bring your own I/O' pattern for QUIC and HTTP/3 APIs, making it testable and integrable with different concurrency models.
artkit
ARTKIT is a Python framework developed by BCG X for automating prompt-based testing and evaluation of Gen AI applications. It allows users to develop automated end-to-end testing and evaluation pipelines for Gen AI systems, supporting multi-turn conversations and various testing scenarios like Q&A accuracy, brand values, equitability, safety, and security. The framework provides a simple API, asynchronous processing, caching, model agnostic support, end-to-end pipelines, multi-turn conversations, robust data flows, and visualizations. ARTKIT is designed for customization by data scientists and engineers to enhance human-in-the-loop testing and evaluation, emphasizing the importance of tailored testing for each Gen AI use case.