> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeus.agentspro.cn/llms.txt
> Use this file to discover all available pages before exploring further.

# RPA Workflow

> Publish recorded browser workflows as MCP tools for AI Agent remote invocation

# RPA Workflow

Zeus Desktop supports publishing recorded and parameterized browser workflows **as standard MCP (JSON-RPC 2.0) tools**, enabling AI Agents to drive browser-based business operations just like calling regular functions.

## Overview

### Architecture

```mermaid theme={null}
flowchart LR
    subgraph Cloud["Cloud / AI Backend"]
        Agent["AI Agent"]
        WS["WebSocket Server"]
    end

    subgraph Desktop["Client / Zeus Desktop"]
        WSC["WebSocket Client"]
        MCP["MCP Server\n(JSON-RPC 2.0)"]
        RE["CDP Replay Engine"]
        BR["Browser Reader"]
        HITL["HITL Manager"]
        Browser["Chrome Browser\n(Silent Mode)"]
    end

    Agent -->|"Call Tool"| WS
    WS <-->|"WebSocket\n(Client-initiated)"| WSC
    WSC --> MCP
    MCP -->|"workflow_*"| RE
    MCP -->|"browser_read"| BR
    MCP -->|"hitl_prompt"| HITL
    RE --> Browser
    BR --> Browser
```

### Design Principles

| Principle                   | Description                                                                        |
| --------------------------- | ---------------------------------------------------------------------------------- |
| **No Open Ports on Client** | All communication goes through client-initiated WebSocket connections              |
| **Standard Protocol**       | Strictly follows JSON-RPC 2.0 / MCP protocol specification                         |
| **Dynamic Registration**    | Published workflows are automatically registered as callable tools without restart |
| **Silent Execution**        | Browser can run silently in the background without disturbing the user             |
| **Strict Flow**             | Workflows execute steps sequentially; exploratory actions are not allowed          |

## Publishing a Workflow as a Tool

### Publishing Process

1. **Complete Recording**: Record browser actions and save as a workflow
2. **Parameterize**: Mark values that need dynamic injection as variables (see [Recording - Parameterization](/en/desktop/recording#parameterization))
3. **Publish Tool**: Click the **Publish Tool** button in the workflow editor
4. **Configure Metadata**:
   * **Tool Name** (`toolName`): Identifier used by the Agent when calling, e.g. `query_power_data`
   * **Tool Description** (`toolDescription`): Tells the Agent what this tool does
5. **Confirm Publishing**: The tool is immediately available after saving

### Publish Dialog

The publish dialog automatically extracts all parameterized actions from the workflow and generates a tool parameter list:

| Field          | Source                | Description                                     |
| -------------- | --------------------- | ----------------------------------------------- |
| Parameter Name | `variableName`        | Variable name used as tool input parameter name |
| Description    | `paramDescription`    | Description of parameter purpose                |
| Default Value  | `defaultValue`        | Optional default value                          |
| Required       | No default → required | Parameters without default values are required  |

### Generated Tool Definition

After publishing, the system automatically generates an MCP-compliant tool definition:

```json theme={null}
{
  "name": "workflow_query_power_data",
  "description": "Log in to the power system and query electricity usage data for a specified region",
  "inputSchema": {
    "type": "object",
    "properties": {
      "username": {
        "type": "string",
        "description": "Login username"
      },
      "password": {
        "type": "string",
        "description": "Login password"
      },
      "region": {
        "type": "string",
        "description": "Region code for query",
        "default": "110000"
      },
      "profileId": {
        "type": "string",
        "description": "Browser profile ID (optional override)"
      }
    },
    "required": ["username", "password"]
  }
}
```

## Agent Invocation Flow

### Tool Discovery

The Agent retrieves all available tools on the client via the MCP `tools/list` method:

```json theme={null}
// Agent → Desktop (JSON-RPC Request)
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/list"
}

// Desktop → Agent (JSON-RPC Response)
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "tools": [
      {
        "name": "desktop_exec",
        "description": "Execute shell commands on desktop"
      },
      {
        "name": "browser_read",
        "description": "Extract data from browser pages"
      },
      {
        "name": "hitl_prompt",
        "description": "Request user input during automation"
      },
      {
        "name": "workflow_query_power_data",
        "description": "Log in to the power system and query electricity usage data for a specified region",
        "inputSchema": { "..." }
      }
    ]
  }
}
```

### Tool Invocation

When the Agent decides to call a workflow tool, it sends a standard `tools/call` request:

```json theme={null}
// Agent → Desktop
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "workflow_query_power_data",
    "arguments": {
      "username": "operator_001",
      "password": "secure_pass",
      "region": "330100"
    }
  }
}
```

### Execution Result

```json theme={null}
// Desktop → Agent
{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\"success\": true, \"totalActions\": 8, \"completedActions\": 8, \"failedActions\": 0}"
      }
    ]
  }
}
```

## Built-in Tools

In addition to dynamically registered workflow tools, Desktop also provides the following built-in MCP tools:

### desktop\_exec

Execute shell commands on the client.

```json theme={null}
{
  "name": "desktop_exec",
  "arguments": {
    "command": "ls -la /tmp/reports/"
  }
}
```

### browser\_read

Extract data from browser pages with multiple extraction methods:

| Action       | Description                                 | Return Value              |
| ------------ | ------------------------------------------- | ------------------------- |
| `text`       | Extract text content from a page or element | Plain text string         |
| `table`      | Parse HTML tables into structured data      | 2D array (headers + rows) |
| `form`       | Extract current form field values           | Field name-value mapping  |
| `html`       | Get the HTML of an element or page          | HTML string               |
| `url`        | Get the current page URL                    | URL string                |
| `screenshot` | Take a page screenshot                      | Base64 PNG image          |

**Example call:**

```json theme={null}
{
  "name": "browser_read",
  "arguments": {
    "action": "table",
    "selector": "#data-table",
    "profileId": "profile-001"
  }
}
```

**Typical scenario:** The Agent first executes `workflow_query_power_data` to navigate to the data page, then uses `browser_read` to extract table data from the page for analysis.

### hitl\_prompt

Request manual user input during an automated workflow — useful for captchas, SMS verification codes, and other values that cannot be obtained automatically.

```json theme={null}
{
  "name": "hitl_prompt",
  "arguments": {
    "title": "SMS Verification Code",
    "message": "Please enter the 6-digit verification code sent to your phone",
    "inputType": "text",
    "timeout": 120
  }
}
```

**Execution flow:**

1. The system shows a notification to alert the user
2. An input dialog appears in the Desktop UI
3. After the user enters a value, it is returned to the Agent
4. If no input is provided before timeout, a timeout status is returned

**Interaction method priority:**

| Priority | Method         | Description                                                                    |
| -------- | -------------- | ------------------------------------------------------------------------------ |
| 1        | Renderer popup | When the Desktop main window is visible, show a dialog in the renderer process |
| 2        | Native dialog  | When the main window is not visible, use an Electron native dialog             |
| 3        | Timeout        | If no response within the configured time, return a timeout status             |

## Silent Browser Mode

When the Agent remotely calls a workflow tool, the browser launches in **silent mode**, without displaying a browser window on the user's screen:

### Silent Mode Features

| Feature                    | Description                                                     |
| -------------------------- | --------------------------------------------------------------- |
| **Window Not Visible**     | Browser window is positioned off-screen (-2400, -2400)          |
| **Fixed Viewport**         | Window size is fixed at 1280×720                                |
| **Minimal Resources**      | GPU, extensions, popups, and notifications are disabled         |
| **Isolated Configuration** | Uses the specified browser profile, isolating cookies and state |

### Silent Mode Launch Arguments

```
--window-position=-2400,-2400
--window-size=1280,720
--disable-gpu
--disable-extensions
--disable-popup-blocking
--disable-notifications
```

## Dynamic Tool Registration

### Registration Timing

Desktop syncs tool information to the backend at the following times:

1. **When WebSocket connection is established**: Sends a registration message containing all available tools
2. **When a workflow is published/unpublished**: Notifies the backend to update the tool list

### Registration Message

```json theme={null}
{
  "type": "register",
  "deviceId": "device-abc123",
  "capabilities": ["desktop_exec", "browser_control", "workflow_execution"],
  "available_tools": [
    "desktop_exec",
    "browser_read",
    "hitl_prompt",
    "workflow_query_power_data",
    "workflow_generate_report"
  ]
}
```

### Backend Tool Creation

When the AI Backend (Python) receives the tool list, it dynamically creates a LangChain `StructuredTool` for each workflow tool:

```python theme={null}
# Dynamically create Pydantic schema
DynamicModel = create_model(
    "QueryPowerDataInput",
    username=(str, Field(description="Login username")),
    password=(str, Field(description="Login password")),
    region=(str, Field(default="110000", description="Region code for query")),
)

# Create LangChain Tool
tool = StructuredTool.from_function(
    name="workflow_query_power_data",
    description="Log in to the power system and query electricity usage data for a specified region",
    args_schema=DynamicModel,
    func=lambda **kwargs: call_desktop_tool("workflow_query_power_data", kwargs),
)
```

## Typical Business Scenarios

### Scenario 1: Power Data Collection

```mermaid theme={null}
sequenceDiagram
    participant Agent as AI Agent
    participant Desktop as Zeus Desktop
    participant Browser as Chrome (Silent)
    participant System as Power System

    Agent->>Desktop: tools/call workflow_login_power_system
    Desktop->>Browser: Launch silent browser
    Browser->>System: Navigate to login page
    Browser->>System: Enter username/password
    Browser->>System: Submit login
    Desktop-->>Agent: Login successful

    Agent->>Desktop: tools/call browser_read (table)
    Desktop->>Browser: Extract data table
    Desktop-->>Agent: Return structured data

    Agent->>Agent: Analyze data, generate report
```

### Scenario 2: Operations Requiring Verification Codes

```mermaid theme={null}
sequenceDiagram
    participant Agent as AI Agent
    participant Desktop as Zeus Desktop
    participant User as End User
    participant Browser as Chrome

    Agent->>Desktop: tools/call workflow_submit_report
    Desktop->>Browser: Execute until captcha step
    Note over Desktop: Manual input detected

    Agent->>Desktop: tools/call hitl_prompt
    Desktop->>User: Show notification + input dialog
    User->>Desktop: Enter verification code "123456"
    Desktop-->>Agent: Return verification code

    Agent->>Desktop: Continue workflow execution
    Desktop->>Browser: Enter verification code, submit
    Desktop-->>Agent: Operation complete
```

### Scenario 3: Batch Data Processing

The Agent can call workflow tools in a loop to handle batch tasks:

1. Call `workflow_login` to log in to the system
2. Call `workflow_query_data` in a loop (with different parameters) to extract data across multiple pages
3. Call `browser_read` to retrieve page data
4. Aggregate and analyze, then call `workflow_generate_report` to produce a report

## Error Handling

### Error Codes

| Error Code | Description              |
| ---------- | ------------------------ |
| `-32001`   | Tool execution error     |
| `-32002`   | Workflow execution error |
| `-32003`   | Workflow not found       |
| `-32004`   | Browser not running      |

### Replay Engine Fault Tolerance

| Mechanism              | Parameters              | Description                                        |
| ---------------------- | ----------------------- | -------------------------------------------------- |
| **Element Wait**       | Max 10s                 | Wait for target element to appear in the DOM       |
| **Action Retry**       | 3 attempts, 1s interval | Automatically retry failed actions                 |
| **Timeout Protection** | 30s per action          | Skip to the next step if a single action times out |

### Execution Result

After workflow execution completes, a structured result is returned:

```typescript theme={null}
interface WorkflowExecutionResult {
  success: boolean
  totalActions: number
  completedActions: number
  failedActions: number
  results: ActionResult[]
}
```

## Compatibility

### Windows 7 Support

Since some clients run on Windows 7, note the following:

* Electron 22.x is the last version to support Windows 7
* The corresponding Chromium engine version must be used
* Some modern Web APIs may not be available

### Domestic OS Support

Compatibility with domestic operating systems (UOS / Kylin):

| Platform      | Architecture | Distribution Format |
| ------------- | ------------ | ------------------- |
| UOS           | x64 / arm64  | `.deb`              |
| Kylin         | x64 / arm64  | `.rpm` / `.deb`     |
| Generic Linux | x64          | `.zip` (portable)   |

## Related Documentation

* [Recording](/en/desktop/recording) - Record, edit, and parameterize workflows
* [Desktop Overview](/en/desktop/overview) - Desktop application overview
* [Workflow Recording](/en/desktop/workflow-recording) - Video + audio + event multi-layer recording technical design