playword

Automate browsers with AI to boost productivity and make testing more enjoyable!

Stars: 52

Visit

PlayWord is a tool designed to supercharge web test automation experience with AI. It provides core features such as enabling browser operations and validations using natural language inputs, as well as monitoring interface to record and dry-run test steps. PlayWord supports multiple AI services including Anthropic, Google, and OpenAI, allowing users to select the appropriate provider based on their requirements. The tool also offers features like assertion handling, frame handling, custom variables, test recordings, and an Observer module to track user interactions on web pages. With PlayWord, users can interact with web pages using natural language commands, reducing the need to worry about element locators and providing AI-powered adaptation to UI changes.

README:

PlayWord

Supercharge your web test automation experience with AI.

📦 Installation

Choose the package that best suits your needs.

@playword/core

The @playword/core package provides the core features of PlayWord and can be used as Node.js modules.

It includes the following modules:

PlayWord: Enables browser operations and validations using natural language inputs to interact with web pages.
Observer: Mounts a monitoring interface on the browser to record and dry-run captured test steps.

# Install with any package manager you prefer
npm install @playword/core --save-dev

@playword/cli

The @playword/cli package enables you to use the features of PlayWord directly through the command line.

For ease of use, I recommend running this package with npx.

# Run a PlayWord test
npx @playword/cli test --headed --verbose -b webkit

# Start the Observer
npx @playword/cli observe -b chromium -v

See documentation for usage examples and options.

📘 Getting Started

PlayWord supports multiple AI services, including Anthropic, Google, and OpenAI. You can select the appropriate provider based on your requirements.

OpenAI

There are two ways to provide the required API key to PlayWord:

1. Export the API key as an environment variable:

export OPENAI_API_KEY="sk-..."

2. Pass the API key as a parameter during initialization:

import { chromium } from 'playwright'

const browser = await chromium.launch()
const context = await browser.newContext()

const playword = new PlayWord(context, {
  aiOptions: {
    baseURL: 'https://...', // Custom API endpoint (If applicable)
    openAIApiKey: 'sk-...',
    model: 'gpt-4o' // If not specified, the default model is gpt-4o-mini.
  }
})

Google

1. Export the API key as an environment variable:

export GOOGLE_API_KEY="AI..."

2. Pass the API key as a parameter during initialization:

const playword = new PlayWord(context, {
  aiOptions: {
    googleApiKey: 'AI...',
    model: 'gemini-2.0-flash' // If not specified, the default model is gemini-2.0-flash-lite.
  }
})

Anthropic

Since Anthropic does not offer its own embeddings model, integrating Anthropic requires an additional API key for embeddings.

Currently, PlayWord supports the following providers for embeddings:

VoyageAI (officially recommended by Anthropic)
OpenAI
Google

1. Export API keys as environment variables:

export ANTHROPIC_API_KEY="sk-..."
export VOYAGEAI_API_KEY="pa-..."

2. Pass the API keys as parameters during initialization:

const playword = new PlayWord(context, {
  aiOptions: {
    anthropicApiKey: 'sk-...',
    voyageAIApiKey: 'pa-...',
    model: 'claude-3-7-sonnet-latest' // If not specified, the default model is claude-3-5-haiku-latest.
  }
})

📜 PlayWord Options

Name	Type	Default	Description
aiOptions	object	{}	Configuration options for the AI instance.
debug	boolean	false	Whether to enable debug mode.
delay	number	250	Delay between each step in milliseconds.
record	boolean \| string	false	Whether to record actions performed and where to save the recordings.

💬 Communicate with Browser

In its basic usage, you can use the say method to interact with the page.

No need to worry about locating elements or performing interactions—PlayWord handles all of that for you.

await playword.say('Navigate to https://www.google.com')

await playword.say('Type "Hello, World!" in the search bar')

await playword.say('Press enter')

✅ Assertion

PlayWord uses keywords to identify whether a step is an assertion. This approach ensures more stable results compared to relying solely on AI judgment.

Using PlayWord within Playwright Test

import { PlayWord } from '@playword/core'
import { expect, test } from '@playwright/test'

test('get started link', async ({ context }) => {
  const playword = new PlayWord(context, { debug: true, record: 'recordings/getStartLink.json' })

  await playword.say('go to https://playwright.dev/')
  await playword.say('click the link "Get started"')

  expect(await playword.say('Verify if the installation heading is visible')).toBe(true)
})

The input starting with any of the following case-insensitive keywords will be recognized as an assertion:

are
assert
assure
can
check
compare
confirm
could
did
do
does
ensure
expect
guarantee
has
have
is
match
satisfy
shall
should
test
then
was
were
validate
verify

🖼️ Frame Handling

To interact with elements inside frames, simply instruct PlayWord to switch to the desired frame.

await playword.say('Go to https://iframetester.com')

await playword.say('Type "https://www.saucedemo.com" in the URL field')

await playword.say('Click the render button')

await playword.say('Switch to the frame with the url "https://www.saucedemo.com"')

// Perform actions inside the frame
await playword.say('Type standard_user into the username field')

🔧 Custom Variables

Hardcoding sensitive information in your test cases is not a good practice. Instead, use custom variables with the syntax {VARIABLE_NAME} and define them in your environment settings.

# .env
USERNAME=standard_user
PASSWORD=secret_sauce

// Load environment variables
import 'dotenv/config'

// {USERNAME} and {PASSWORD} will be replaced with the values from the environment
await playword.say('Input {USERNAME} in the username field')
await playword.say('Input {PASSWORD} in the password field')

🔴 Recordings

PlayWord supports recording test executions and replaying them later for efficient and consistent testing.

// Save recordings to the default path (.playword/recordings.json)
const playword = new PlayWord(context, { record: true })

// Save recordings to a custom path (Must be `.json`)
const playword = new PlayWord(context, { record: 'spec/test-shopping-cart.json' })

If recordings are available, PlayWord prioritizes using them to execute tests, reducing the need to consume API tokens.

If a recorded action fails, PlayWord automatically retries it using AI.

✨ Using AI during Playback

To ensure PlayWord uses AI for specific steps during playback, start the input with [AI].

await playword.say('[AI] click the "Login" button')

await playword.say('[AI] verify the URL matches "https://www.saucedemo.com/inventory.html"')

🖥️ Observer

The Observer module tracks user interactions on web pages and swiftly generates accurate test steps using AI.

Upon activation, Playwright injects the Observer UI into every launched browser webpage. As you manually interact with the page, the AI interprets your actions, generates corresponding test steps, and records action details.

✨ Observer Features

The Observer provides several controls to manage and interact with your test recordings:

Accept: Add test steps to the recording. (Can also be invoked by pressing the a key)
Cancel: Skip test steps without adding them to the recording. (Can also be invoked by pressing the c key)
Preview: View the test steps recorded so far.
Clear: Delete recorded test steps.
Dry Run: Trial-run the recorded test steps. (Can press the esc key to stop the dry-run process)

And it captures various user interactions on the webpage as follows:

Click: Triggered when an element on the webpage is clicked.
Hover: Triggered when hovering over an element for more than three seconds
Input: Triggered after entering content into an input field and then clicking the input field again.
Navigate: Triggered when the page navigates to a new URL or is refreshed.
Select: Triggered after selecting an option from a dropdown menu.

For complex actions and assertions that the Observer cannot directly record, you can manually edit the step descriptions, enabling the AI to accurately capture your intentions.

📘 Getting Started with Observer

To start using the Observer, create a PlayWord instance in headed mode, pass it to the Observer, and initiate observation with Playwright.

import { chromium } from 'playwright'
import { Observer, PlayWord } from '@playword/core'

const browser = await chromium.launch({ headless: false /** Enable headed mode */ })
const context = await browser.newContext()

const playword = new PlayWord(context)
const observer = new Observer(playword, {
  delay: 500,
  recordPath: 'spec/test-login.json'
})

// Start the Observer
await observer.observe()

// Open a new page to observe
await context.newPage()

📜 Observer Options

Name	Type	Default	Description
delay	number	250	Delay between each step in milliseconds during the dry-run process.
recordPath	string	.playword/recordings.json	Where to save the recordings. (Must be `.json`)

🌟 Why use PlayWord?

Aspect	Traditional Testing	PlayWord
Dev Experience	Locating elements is very frustrating.	AI takes care of locating elements. Say goodbye to locators.
Dev Speed	Time is needed for writing both test cases and code.	Test cases serve both as documentation and executable tests.
Maintainance	High maintenance cost due to UI changes.	AI-powered adaption to UI changes.
Learning Curve	Requires knowledge of testing frameworks and tools.	Just use natural language to execute tests.

📜 Supported Actions in PlayWord and PlayWord Observer

Page Actions

Click on an element
Go to a specific URL
Hover over an element
Press a key or keys
Scroll in a specific direction (top, bottom, up, down)
Select an option from a select element
Sleep for a specific duration in milliseconds
Switch to a specific frame
Switch to other pages
Type text into an input field or textarea
Wait for text to appear on the page

Assertion

Check if an element contains specific text
Check if an element does not contain specific text
Check if an element content is equal to specific text
Check if an element content is not equal to specific text
Check if an element is visible
Check if an element is not visible
Check if the page contains specific text
Check if the page does not contain specific text
Check if the page title is equal to specific text
Check if the page URL matches specific RegExp patterns

Enjoy PlayWord and stay tuned for more features in future releases! 🚀🎉

For Tasks:

Click tags to check more tools for each tasks

interact with web pages record and dry-run test steps handle assertions manage frames track user interactions

For Jobs:

automation engineer quality assurance analyst software developer in test test automation architect web application tester

Alternative AI tools for playword

Similar Open Source Tools

playword

github

: 52

code2prompt

Code2Prompt is a powerful command-line tool that generates comprehensive prompts from codebases, designed to streamline interactions between developers and Large Language Models (LLMs) for code analysis, documentation, and improvement tasks. It bridges the gap between codebases and LLMs by converting projects into AI-friendly prompts, enabling users to leverage AI for various software development tasks. The tool offers features like holistic codebase representation, intelligent source tree generation, customizable prompt templates, smart token management, Gitignore integration, flexible file handling, clipboard-ready output, multiple output options, and enhanced code readability.

github

: 734

xFasterTransformer

xFasterTransformer is an optimized solution for Large Language Models (LLMs) on the X86 platform, providing high performance and scalability for inference on mainstream LLM models. It offers C++ and Python APIs for easy integration, along with example codes and benchmark scripts. Users can prepare models in a different format, convert them, and use the APIs for tasks like encoding input prompts, generating token ids, and serving inference requests. The tool supports various data types and models, and can run in single or multi-rank modes using MPI. A web demo based on Gradio is available for popular LLM models like ChatGLM and Llama2. Benchmark scripts help evaluate model inference performance quickly, and MLServer enables serving with REST and gRPC interfaces.

github

: 247

RainbowGPT

RainbowGPT is a versatile tool that offers a range of functionalities, including Stock Analysis for financial decision-making, MySQL Management for database navigation, and integration of AI technologies like GPT-4 and ChatGlm3. It provides a user-friendly interface suitable for all skill levels, ensuring seamless information flow and continuous expansion of emerging technologies. The tool enhances adaptability, creativity, and insight, making it a valuable asset for various projects and tasks.

github

: 86

langserve

LangServe helps developers deploy `LangChain` runnables and chains as a REST API. This library is integrated with FastAPI and uses pydantic for data validation. In addition, it provides a client that can be used to call into runnables deployed on a server. A JavaScript client is available in LangChain.js.

github

: 1.9k

mcp-graphql

mcp-graphql is a Model Context Protocol server that enables Large Language Models (LLMs) to interact with GraphQL APIs. It provides schema introspection and query execution capabilities, allowing models to dynamically discover and use GraphQL APIs. The server offers tools for retrieving the GraphQL schema and executing queries against the endpoint. Mutations are disabled by default for security reasons. Users can install mcp-graphql via Smithery or manually to Claude Desktop. It is recommended to carefully consider enabling mutations in production environments to prevent unauthorized data modifications.

github

: 62

wanda

Official PyTorch implementation of Wanda (Pruning by Weights and Activations), a simple and effective pruning approach for large language models. The pruning approach removes weights on a per-output basis, by the product of weight magnitudes and input activation norms. The repository provides support for various features such as LLaMA-2, ablation study on OBS weight update, zero-shot evaluation, and speedup evaluation. Users can replicate main results from the paper using provided bash commands. The tool aims to enhance the efficiency and performance of language models through structured and unstructured sparsity techniques.

github

: 560

deep-research

Deep Research is a lightning-fast tool that uses powerful AI models to generate comprehensive research reports in just a few minutes. It leverages advanced 'Thinking' and 'Task' models, combined with an internet connection, to provide fast and insightful analysis on various topics. The tool ensures privacy by processing and storing all data locally. It supports multi-platform deployment, offers support for various large language models, web search functionality, knowledge graph generation, research history preservation, local and server API support, PWA technology, multi-key payload support, multi-language support, and is built with modern technologies like Next.js and Shadcn UI. Deep Research is open-source under the MIT License.

github

: 4.0k

suno-api

Suno AI API is an open-source project that allows developers to integrate the music generation capabilities of Suno.ai into their own applications. The API provides a simple and convenient way to generate music, lyrics, and other audio content using Suno.ai's powerful AI models. With Suno AI API, developers can easily add music generation functionality to their apps, websites, and other projects.

github

: 1.7k

metta

Metta AI is an open-source research project focusing on the emergence of cooperation and alignment in multi-agent AI systems. It explores the impact of social dynamics like kinship and mate selection on learning and cooperative behaviors of AI agents. The project introduces a reward-sharing mechanism mimicking familial bonds and mate selection to observe the evolution of complex social behaviors among AI agents. Metta aims to contribute to the discussion on safe and beneficial AGI by creating an environment where AI agents can develop general intelligence through continuous learning and adaptation.

github

: 102

Fabric

Fabric is an open-source framework designed to augment humans using AI by organizing prompts by real-world tasks. It addresses the integration problem of AI by creating and organizing prompts for various tasks. Users can create, collect, and organize AI solutions in a single place for use in their favorite tools. Fabric also serves as a command-line interface for those focused on the terminal. It offers a wide range of features and capabilities, including support for multiple AI providers, internationalization, speech-to-text, AI reasoning, model management, web search, text-to-speech, desktop notifications, and more. The project aims to help humans flourish by leveraging AI technology to solve human problems and enhance creativity.

github

: 33.6k

graphiti

Graphiti is a framework for building and querying temporally-aware knowledge graphs, tailored for AI agents in dynamic environments. It continuously integrates user interactions, structured and unstructured data, and external information into a coherent, queryable graph. The framework supports incremental data updates, efficient retrieval, and precise historical queries without complete graph recomputation, making it suitable for developing interactive, context-aware AI applications.

github

: 18.5k

airbadge

Airbadge is a Stripe addon for Auth.js that provides an easy way to create a SaaS site without writing any authentication or payment code. It integrates Stripe Checkout into the signup flow, offers over 50 OAuth options for authentication, allows route and UI restriction based on subscription, enables self-service account management, handles all Stripe webhooks, supports trials and free plans, includes subscription and plan data in the session, and is open source with a BSL license. The project also provides components for conditional UI display based on subscription status and helper functions to restrict route access. Additionally, it offers a billing endpoint with various routes for billing operations. Setup involves installing @airbadge/sveltekit, setting up a database provider for Auth.js, adding environment variables, configuring authentication and billing options, and forwarding Stripe events to localhost.

github

: 84

EasyInstruct

EasyInstruct is a Python package proposed as an easy-to-use instruction processing framework for Large Language Models (LLMs) like GPT-4, LLaMA, ChatGLM in your research experiments. EasyInstruct modularizes instruction generation, selection, and prompting, while also considering their combination and interaction.

github

: 381

action_mcp

Action MCP is a powerful tool for managing and automating your cloud infrastructure. It provides a user-friendly interface to easily create, update, and delete resources on popular cloud platforms. With Action MCP, you can streamline your deployment process, reduce manual errors, and improve overall efficiency. The tool supports various cloud providers and offers a wide range of features to meet your infrastructure management needs. Whether you are a developer, system administrator, or DevOps engineer, Action MCP can help you simplify and optimize your cloud operations.

github

: 65

openai-kotlin

OpenAI Kotlin API client is a Kotlin client for OpenAI's API with multiplatform and coroutines capabilities. It allows users to interact with OpenAI's API using Kotlin programming language. The client supports various features such as models, chat, images, embeddings, files, fine-tuning, moderations, audio, assistants, threads, messages, and runs. It also provides guides on getting started, chat & function call, file source guide, and assistants. Sample apps are available for reference, and troubleshooting guides are provided for common issues. The project is open-source and licensed under the MIT license, allowing contributions from the community.

github

: 1.4k

For similar tasks

playword

github

: 52

AutoNode

AutoNode is a self-operating computer system designed to automate web interactions and data extraction processes. It leverages advanced technologies like OCR (Optical Character Recognition), YOLO (You Only Look Once) models for object detection, and a custom site-graph to navigate and interact with web pages programmatically. Users can define objectives, create site-graphs, and utilize AutoNode via API to automate tasks on websites. The tool also supports training custom YOLO models for object detection and OCR for text recognition on web pages. AutoNode can be used for tasks such as extracting product details, automating web interactions, and more.

github

: 116

WebDreamer

WebDreamer is a model-based planning tool for web agents that uses large language models (LLMs) as a world model of the internet to predict outcomes of actions on websites. It employs LLM-based simulation for speculative planning on the web, offering greater safety and flexibility compared to traditional tree search methods. The tool provides modules for world model prediction, simulation scoring, and controller actions, enabling users to interact with web pages and achieve specific goals through simulated actions.

github

: 82

For similar jobs

aiscript

AiScript is a lightweight scripting language that runs on JavaScript. It supports arrays, objects, and functions as first-class citizens, and is easy to write without the need for semicolons or commas. AiScript runs in a secure sandbox environment, preventing infinite loops from freezing the host. It also allows for easy provision of variables and functions from the host.

github

: 227

askui

AskUI is a reliable, automated end-to-end automation tool that only depends on what is shown on your screen instead of the technology or platform you are running on.

github

: 83

bots

The 'bots' repository is a collection of guides, tools, and example bots for programming bots to play video games. It provides resources on running bots live, installing the BotLab client, debugging bots, testing bots in simulated environments, and more. The repository also includes example bots for games like EVE Online, Tribal Wars 2, and Elvenar. Users can learn about developing bots for specific games, syntax of the Elm programming language, and tools for memory reading development. Additionally, there are guides on bot programming, contributing to BotLab, and exploring Elm syntax and core library.

github

: 179

ain

Ain is a terminal HTTP API client designed for scripting input and processing output via pipes. It allows flexible organization of APIs using files and folders, supports shell-scripts and executables for common tasks, handles url-encoding, and enables sharing the resulting curl, wget, or httpie command-line. Users can put things that change in environment variables or .env-files, and pipe the API output for further processing. Ain targets users who work with many APIs using a simple file format and uses curl, wget, or httpie to make the actual calls.

github

: 592

LaVague

LaVague is an open-source Large Action Model framework that uses advanced AI techniques to compile natural language instructions into browser automation code. It leverages Selenium or Playwright for browser actions. Users can interact with LaVague through an interactive Gradio interface to automate web interactions. The tool requires an OpenAI API key for default examples and offers a Playwright integration guide. Contributors can help by working on outlined tasks, submitting PRs, and engaging with the community on Discord. The project roadmap is available to track progress, but users should exercise caution when executing LLM-generated code using 'exec'.

github

: 5.8k

robocorp

Robocorp is a platform that allows users to create, deploy, and operate Python automations and AI actions. It provides an easy way to extend the capabilities of AI agents, assistants, and copilots with custom actions written in Python. Users can create and deploy tools, skills, loaders, and plugins that securely connect any AI Assistant platform to their data and applications. The Robocorp Action Server makes Python scripts compatible with ChatGPT and LangChain by automatically creating and exposing an API based on function declaration, type hints, and docstrings. It simplifies the process of developing and deploying AI actions, enabling users to interact with AI frameworks effortlessly.

github

: 501

Open-Interface

Open Interface is a self-driving software that automates computer tasks by sending user requests to a language model backend (e.g., GPT-4V) and simulating keyboard and mouse inputs to execute the steps. It course-corrects by sending current screenshots to the language models. The tool supports MacOS, Linux, and Windows, and requires setting up the OpenAI API key for access to GPT-4V. It can automate tasks like creating meal plans, setting up custom language model backends, and more. Open Interface is currently not efficient in accurate spatial reasoning, tracking itself in tabular contexts, and navigating complex GUI-rich applications. Future improvements aim to enhance the tool's capabilities with better models trained on video walkthroughs. The tool is cost-effective, with user requests priced between $0.05 - $0.20, and offers features like interrupting the app and primary display visibility in multi-monitor setups.

github

: 934

AI-Case-Sorter-CS7.1

AI-Case-Sorter-CS7.1 is a project focused on building a case sorter using machine vision and machine learning AI to sort cases by headstamp. The repository includes Arduino code and 3D models necessary for the project.

github

: 67