Skip to main content
Zeus Desktop provides comprehensive browser action recording, automatically capturing user operations in the browser as editable, replayable workflows.

Overview

Workflow recording is built on the Chrome DevTools Protocol (CDP), capturing DOM-level browser events with higher accuracy and stability compared to traditional coordinate-based recording approaches.

Core Capabilities

CapabilityDescription
Precise RecordingCDP-based DOM event capture with dual locator strategy: CSS selectors + semantic features
Visual EditingDrag-and-drop reordering, step-by-step editing, insert/delete action steps
ParameterizationMark fixed values as variables for dynamic injection
Parameterized ReplayEnter parameter values during test replay to verify workflow correctness
Video SyncAction events are time-aligned with recorded video; click a step to jump to the video timestamp

Quick Start

1. Select a Browser Profile

Before recording, select a browser profile. Each profile has its own independent cookies, cache, and login state.

2. Start Recording

  1. Click the Start Recording button on the recording page
  2. Select a browser profile
  3. Optionally enable video recording (captures screen visuals simultaneously)
  4. The browser launches automatically and enters recording mode

3. Perform Actions

Operate the browser normally. The system automatically captures the following events:
  • Page navigation (URL changes)
  • Mouse clicks
  • Text input and modification
  • Keyboard keystrokes
  • Page scrolling
  • Form submission
  • Drag and drop

4. Stop Recording

Click the Stop Recording button. The recorded action sequence is displayed in the editor for further editing and saving.

Supported Action Types

Browser Events

TypeDescriptionParameterizable
navigateNavigate to a specified URL✅ URL can be parameterized
clickClick a page element
inputType text into an input field✅ Input value can be parameterized
changeChange a form control value (dropdowns, etc.)✅ Value can be parameterized
scrollScroll the page or an element
keypressPress a keyboard key (Enter, Tab, etc.)
waitWait for a specified duration
submitSubmit a form
dragDrag an element to a target position

System-level Events (Extensions)

TypeDescription
system_clickSystem-level mouse click
system_keypressSystem-level keyboard input
app_switchApplication switch
applescriptAppleScript execution
shell_commandShell command execution

Element Locator Strategy

During recording, each action’s target element uses a multi-strategy locator approach to ensure high accuracy during replay:

Locator Priority

PriorityMethodConfidenceDescription
1testId1.00Exact match on data-testid attribute
2exact0.95Full semantic match (role + name)
3selector0.85CSS selector match
4fuzzy0.70Fuzzy match (text content, ARIA attributes)
5position0.60Visual position match
6nth0.50Index-based fallback

Semantic Features (ElementFeatures)

Rich semantic features are recorded for each action’s target element:
interface ElementFeatures {
  role?: string          // ARIA role
  name?: string          // Accessible name
  testId?: string        // data-testid
  ariaLabel?: string     // aria-label
  text?: string          // Text content
  placeholder?: string   // placeholder attribute
  tagName?: string       // HTML tag name
  type?: string          // input type
}
These features enable the replay engine to locate the correct target element through semantic information even when the page DOM structure has changed.

Editing Workflows

Workflow Editor

After saving a recording, you can modify the action sequence in the workflow editor:
  • Drag-and-drop reordering: Adjust the execution order of action steps
  • Edit actions: Modify selectors, values, URLs, and other properties
  • Insert steps: Add new action steps at any position
  • Delete steps: Remove unnecessary actions
  • Video sync: Click a step to automatically jump to the corresponding timestamp in the recorded video

Editing Action Details

Click any action step to open the editing dialog:
  • Basic info: Action type, description
  • Selectors: CSS selector, XPath, element features
  • Value: Input value, navigation URL
  • Parameterization settings: Mark as variable, set variable name and default value

Parameterization

Parameterization converts fixed values in a recording into dynamically injectable variables — a key capability for enabling AI Agent-driven execution.

What is Parameterization

Using a login flow as an example:
StepDuring RecordingAfter Parameterization
Enter usernameadmin{username}
Enter password123456{password}
Navigate to systemhttps://sys.example.com{systemUrl}
After parameterization, these values are no longer fixed — they are supplied externally at replay/invocation time.

Single Action Parameterization

  1. Click an action step in the workflow editor to open the editing dialog
  2. Toggle on the Parameterize switch
  3. Set the Variable Name (e.g., username)
  4. Optionally enter a Parameter Description (e.g., “Login username”)
  5. Optionally set a Default Value (e.g., admin)
  6. Save
Once parameterization is enabled, the action’s value field displays as a variable reference (e.g., {username}), and the actual value is injected at runtime.

Batch Parameterization

For workflows with multiple input actions, you can use the Batch Parameterize feature:
  1. If there are non-parameterized input/change/navigate actions, a Batch Parameterize prompt appears at the top of the workflow editor
  2. Click it, and the system automatically generates variable names based on element attributes (name, id, placeholder, etc.)
  3. Current values are automatically set as default values
  4. You can further adjust settings in the editing dialog

Variable Resolution

The replay engine uses {variableName} syntax for variable substitution:
Input value: "Hello, {username}! Your code is {verifyCode}"
Variables passed: { username: "John", verifyCode: "123456" }
Actual execution: "Hello, John! Your code is 123456"
Variable resolution supports:
  • Full replacement: {username}John
  • Inline replacement: https://{host}/apihttps://example.com/api
  • Unmatched variables are kept as-is: {unknown}{unknown}

Replay Testing

Basic Replay

For workflows without parameterized actions, simply select a browser profile and start replay.

Parameterized Replay

When a workflow contains parameterized actions, clicking replay opens a Parameter Input Dialog:
  1. All parameterized fields are automatically displayed
  2. Parameter name, description, and default value are shown
  3. Required parameters (no default value) are marked in red
  4. Enter test values and click Start Replay

Replay Engine

Replay is CDP-based and has the following characteristics:
FeatureDescription
Element WaitAutomatically waits for the target element to appear (up to 10 seconds)
Retry MechanismAutomatically retries failed actions (up to 3 times, 1-second interval)
Timeout ProtectionSkips a single action after 30-second timeout
Progress FeedbackShows real-time execution progress and step status
Visual StatusHighlights the current execution step in the editor (blue = in progress / green = success / red = failed)

Video Recording

Recording Options

OptionDescription
Screen/Window SourceChoose to record the entire screen or a specific window
Resolution720p / 1080p / 4K
Frame Rate30fps / 60fps
Microphone AudioRecord voice narration
System AudioRecord system sound (macOS requires BlackHole)

Video-Action Synchronization

Video and action events are aligned by timestamps:
  • Each action records a videoTimestamp (seconds relative to recording start)
  • Clicking an action step in the editor automatically jumps the video to the corresponding time
  • During replay, the video preview syncs with action execution

Storage Estimates

ResolutionFrame RateDurationFile Size
720p30fps10 minutes~80-120 MB
1080p30fps10 minutes~150-250 MB
1080p30fps1 minute~15-25 MB

Local Storage

Workflow data is stored in the local file system:
{userData}/workflows/
├── {workflow-id}.json          # Workflow definition (action sequence + metadata)
├── {workflow-id}/
│   ├── recordings/             # Recorded video files
│   │   └── recording.webm
│   ├── screenshots/            # Keyframe screenshots
│   │   ├── action-001.png
│   │   └── action-002.png
│   └── metadata.json           # Sync metadata

Workflow JSON Structure

{
  "id": "wf-abc123",
  "name": "Log in to system and query data",
  "actions": [
    {
      "id": "act-001",
      "type": "navigate",
      "url": "https://system.example.com",
      "timestamp": 1700000000000
    },
    {
      "id": "act-002",
      "type": "input",
      "selector": "#username",
      "value": "{username}",
      "isParameterized": true,
      "variableName": "username",
      "paramDescription": "Login username",
      "defaultValue": "admin",
      "elementFeatures": {
        "role": "textbox",
        "name": "Username",
        "placeholder": "Enter username"
      }
    }
  ],
  "profileId": "profile-001",
  "toolName": "login_and_query",
  "toolDescription": "Log in to the system and query specified data",
  "isPublished": true
}

Next Steps

After recording and parameterizing a workflow, you can publish it as an RPA tool so that AI Agents can invoke it remotely.