Fix key mapping in Anthropic CUA Client

[miguelg719]

why

Anthropic agents in CUA mode are unable to issue key presses (not to be confused with type actions)

what changed

The format for the anthropic tool computer_20250124 replies with:

{
  "action":"key",
  "text":"BackSpace"
}

wasn't properly mapped to our internal action abstraction: keypress, which accepts parameter keys. It was issued directly from the anthropic format. Updated AnthropicCUAClient.ts to account for this and map appropriately

test plan

• Tested on sample eval

---

Summary by cubic

Fixes key action mapping in Anthropic CUA so agents can send key presses (e.g., Backspace) correctly instead of failing on the "key" action.

• Bug Fixes
  • Map Anthropic "key" to internal "keypress" and pass keys from input.text.
  • Remove the old "key" path and Playwright key mapping to avoid mismatches.

^{Written for commit b9716b9. Summary will update automatically on new commits.}

[changeset-bot]

🦋 Changeset detected

Latest commit: b9716b9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

No issues found across 2 files

[greptile-apps]

Greptile Overview

Greptile Summary

Fixed Anthropic CUA agents to properly handle key press actions by mapping the key action type to the internal keypress action format.

• Removed old key action handler that used mapKeyToPlaywright utility and standalone processing
• Added key action support to the unified keypress handler, mapping input.text to the keys parameter
• Removed unused mapKeyToPlaywright import from AnthropicCUAClient

This change enables Anthropic agents in CUA mode to issue key presses (like Backspace, Enter, etc.) which were previously failing.

Confidence Score: 4/5

• This PR fixes a bug and is mostly safe to merge with one type casting issue to address
• The core fix correctly maps Anthropic's "key" action to the internal "keypress" action. However, there's a type casting issue on line 777 where input.text (a string) is cast to string[], which could cause the handler to iterate over characters instead of treating it as a single key.
• Pay close attention to packages/core/lib/v3/agent/AnthropicCUAClient.ts line 777 for the type casting fix

Important Files Changed

File Analysis

Filename	Score	Overview
.changeset/shy-sides-crash.md	5/5	Standard changeset file for tracking the patch release
packages/core/lib/v3/agent/AnthropicCUAClient.ts	4/5	Fixed key action mapping to properly convert Anthropic's "key" action to internal "keypress" action, but has a potential type casting issue

Sequence Diagram

sequenceDiagram
    participant Agent as Anthropic Agent
    participant Client as AnthropicCUAClient
    participant Handler as v3CuaAgentHandler
    participant Browser as Playwright Page

    Agent->>Client: Tool use: {action: "key", text: "BackSpace"}
    Client->>Client: convertToolUseToAction()
    Note over Client: Maps "key" action to "keypress"
    Note over Client: Extracts text parameter
    Client->>Client: Return {type: "keypress", keys: text}
    Client->>Handler: Execute keypress action
    Handler->>Handler: Normalize keys to array
    Handler->>Handler: Map key to Playwright format
    Handler->>Browser: page.keyPress(mapped)
    Browser-->>Handler: Key pressed
    Handler-->>Client: Action complete
    Client-->>Agent: Screenshot with result

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}