Automation reference · v1

Drive a rendered page with bounded actions

Add an actions array to POST /screenshot. Each action locates an element, performs one bounded operation, and returns a machine-readable result. The browser runs the sequence in order and fails fast unless an individual action opts into continueOnError.

Locate

Use CSS, XPath, text, ARIA role/name, labels, IDs, test IDs, or coordinates.

Act

Click, type, scroll, navigate, wait, work with tabs, upload, or assert.

Bound

At most 25 actions per request and a 24s/52s wall-clock action budget.

01 · First flow

Automate a login-style flow

Use responseType: "json" so the response includes actionResults and any final Markdown, analysis, or artifact metadata. Secrets in this example are placeholders; never commit real passwords or cookies.

ordered actions

{
  "url": "https://example.com/login",
  "responseType": "json",
  "actions": [
    {
      "action": "click",
      "target": { "role": { "role": "button", "name": "Sign in", "exact": true } }
    },
    { "action": "type", "target": { "label": "Email" }, "text": "person@example.com" },
    { "action": "type", "target": { "label": "Password" }, "text": "YOUR_PASSWORD" },
    { "action": "pressKey", "target": { "label": "Password" }, "key": "Enter" },
    { "action": "waitForUrl", "url": "/dashboard", "exact": false },
    { "action": "assert", "target": { "css": "h1" }, "expected": "visible" }
  ],
  "extractMarkdown": true
}

Action result shape

Each entry has index, action, status (passed, failed, or skipped), and durationMs. Failed entries may include code, error, and safe details.

02 · Locate

Choose the most stable locator

Every action uses either the legacy selector string or a new target locator object, never both. Semantic snapshot node IDs are not accepted as durable targets; copy a selector candidate from the snapshot instead.

Locator	Shape	When to use
css	{ "css": ".buy-button" }	Best when the target has a stable class or data attribute.
xpath	{ "xpath": "//button[normalize-space()='Buy']" }	Useful when the relationship between elements matters.
text	{ "text": { "value": "Buy now", "exact": true } }	Matches visible text; exact defaults to true.
role	{ "role": { "role": "button", "name": "Buy now" } }	Uses the accessible role and accessible name.
label	{ "label": "Email" }	Targets a form control by its associated label.
placeholder	{ "placeholder": "Search" }	Targets an input by placeholder text.
id / name	{ "id": "email" } or { "name": "query" }	Uses standard HTML identity fields.
testId	{ "testId": "save", "testIdAttribute": "data-testid" }	Uses a test ID; the attribute can be customized.
coordinates	{ "coordinates": { "x": 120, "y": 48, "space": "viewport" } }	Fallback for canvas or visual controls. Prefer semantic locators.

Recommended orderPrefer role/name or label, then a stable test ID, then a purposeful CSS selector. Use coordinates only when the control cannot be described semantically.

Exact matching

Text and role/name locators support exact. It defaults to true; set it to false when the target text contains additional dynamic text.

03 · Act

Action catalog and shared fields

Interaction

click, type, select, check, uncheck, hover, focus, pressKey

Movement

scroll, scrollIntoView

Navigation

navigate, back, forward, reload

Waiting

wait, waitForText, waitForUrl, waitForRequest, waitForResponse, waitForNetworkIdle

Pages

openTab, switchTab, closeTab

Verification

assert

Files

upload

target / selector	Use exactly one. target is the locator object; selector is the legacy CSS string. They cannot be combined.
frame	Targets a same-origin iframe. Cross-origin frames are metadata-only.
tabId	Selects the caller-visible tab for actions that operate on a particular tab.
url	URL for navigate or expected URL for waitForUrl.
text	Text for type or expected text for waitForText.
key	Puppeteer key name such as Enter, Tab, or Control+A.
amount / time	Scroll amount and wait/delay duration. Defaults are 500 and 1,000 ms.
expression	Browser-side expression for waitForFunction. Trusted JavaScript only.
expected	Assertion expression: exists, visible, checked, unchecked, disabled, enabled, text:VALUE, or equals:VALUE.
continueOnError	Default false. Records a failed result and continues when true.

wait, select, and assert

{
  "actions": [
    { "action": "select", "target": { "label": "Country" }, "value": "IN" },
    { "action": "waitForNetworkIdle", "time": 1000 },
    { "action": "assert", "target": { "css": ".results" }, "expected": "visible" },
    { "action": "scroll", "amount": 600 },
    { "action": "waitForText", "text": "Results", "exact": true }
  ]
}

04 · Context

Work with same-origin frames and multiple tabs

Frames

Set the action frame field to address a same-origin iframe. Cross-origin frames are reported in semantic metadata but cannot be entered for actions.

Tabs

openTab creates a tab, switchTab selects a caller-visible tab ID, and closeTab closes it. The final response includes IDs of tabs that remain open.

tab sequence

{
  "actions": [
    { "action": "openTab", "url": "https://example.com/help" },
    { "action": "switchTab", "tabId": "tab-2" },
    { "action": "waitForText", "text": "Help center" },
    { "action": "switchTab", "tabId": "main" },
    { "action": "closeTab", "tabId": "tab-2" }
  ]
}

05 · Files

Upload inline base64 files

The upload action accepts a bounded inline base64 file with a name and optional MIME type. The legacy s3Key source is withdrawn and returns a validation error; it is not a supported way to reference caller storage.

upload action

{
  "actions": [
    {
      "action": "upload",
      "target": { "label": "Choose file" },
      "files": [
        { "name": "resume.pdf", "mimeType": "application/pdf", "base64": "BASE64_FILE_CONTENT" }
      ]
    }
  ]
}

Keep uploads transient. Base64 file data is used to drive the browser input and is not exposed in results. Keep files small enough for the request-body limits and never place secrets in a public example.

06 · Operate

Understand failures, partial work, and time budgets

Actions are fail-fast by default. A failed action stops later actions unless that action sets continueOnError: true. A wall-clock action budget can also skip remaining steps.

Synchronous

24 seconds. A budget exhaustion returns HTTP 206, marks remaining actions as skipped, and reports metadata.budgetWarning.

Durable worker

52 seconds for async captures, monitor runs, batch children, and crawl children. Use async: true when navigation or actions are likely to exceed the synchronous window.

failed	The action ran and returned an error. Inspect code and error.
skipped	The action did not run because a previous failure or the budget stopped the sequence.
partialResult	Some frames, action results, or metadata were captured before a timeout.
waitForFunction	Trusted browser-side JavaScript. It has the same Business-plan, account-entitlement, and global-switch requirements as injectJS.

07 · Checklist

Make automation reliable

Inspect the page with a semantic snapshot or the playground before writing actions.
Prefer accessible role/name, label, and stable test-ID locators over generated class names.
Keep one intent per action and use explicit waits for the state you need.
Use continueOnError only for genuinely optional steps; do not hide a required failure.
Use async: true for long navigation, multiple whole-DOM passes, or long action chains.
Treat actionResults as data. A 200 response does not mean every action passed.
Never send real credentials in documentation, logs, or committed examples.

Continue with rich capture for DOM evidence or jobs & webhooks for durable workflows.