Video Workspace

Overview

The Video Workspace enables AI-driven video creation through a structured pipeline. When a project has type: "video", the workspace UI replaces the default trajectory/files tabs with Storyboard and Timeline panels, while the left-side chat orchestrates the generation flow using the Agent’s ReAct pattern.

Architecture

┌─────────────────────┐     ┌──────────────────────────────────┐
│    Chat (left)       │     │     Video Workspace (right)      │
│                      │     │  ┌──────────┐ ┌──────────┐      │
│  CreativeBriefCard   │     │  │Storyboard│ │ Timeline │  TOC │
│  ScriptCard          │     │  │  Panel   │ │  Panel   │      │
│  StoryboardCard ─────┼────►│  │          │ │          │      │
│  VisualGenPathCard   │     │  └──────────┘ └──────────┘      │
│  VisualAssetsCard    │     └──────────────────────────────────┘
│  AudioAssetsCard     │
│  TimelineLinkCard ───┼────► Timeline panel
└─────────────────────┘

Pipeline Phases

Phase	Agent Skill	Output Card	Backend Tool
1	`creative-brief`	CreativeBriefCard	`video_output_creative_brief`
2	`script-writer` (optional)	ScriptCard	`video_output_script`
3	`storyboard-designer`	StoryboardCard	`video_plan_storyboard`
4	`visual-gen-path`	VisualGenPathCard	`video_output_visual_gen_path`
5	`visual-media-generator`	VisualAssetsCard	`video_generate_element_image` / `video_generate_start_frame` / `video_generate_shot`
6	`audio-media-generator`	AudioAssetsCard	`video_generate_narration` / `video_generate_music`
7	`video-assembler`	TimelineLinkCard	`video_assemble_timeline`

Utility: resource-analyzer — triggered when user uploads reference files.

Storyboard Data Model

videoStoryboard
├── storyboardElement (Key Elements)
│   └── elementAsset (image versions)
├── storyboardSegment (Segments)
│   └── storyboardShot (Shots)
│       └── shotAsset (start frame + video versions)
└── storyboardAudio (Narration & Music)
    └── audioAsset (audio versions)

All tables defined in apps/web/src/db/schema/video.ts.

Frontend Components

Component	Path	Description
`VideoWorkspace`	`components/agent/video-workspace/VideoWorkspace.tsx`	Main container with tab switching
`StoryboardPanel`	`components/agent/video-workspace/StoryboardPanel.tsx`	Three collapsible sections
`TimelinePanel`	`components/agent/video-workspace/TimelinePanel.tsx`	Video preview + 3-track timeline
`StoryboardTOC`	`components/agent/video-workspace/StoryboardTOC.tsx`	Right-side outline navigation
`ElementCard`	`components/agent/video-workspace/storyboard/ElementCard.tsx`	Key Element display
`ShotCard`	`components/agent/video-workspace/storyboard/ShotCard.tsx`	Shot display with status/progress
`AudioCard`	`components/agent/video-workspace/storyboard/AudioCard.tsx`	Audio item with player
`MediaCard`	`components/agent/video-workspace/storyboard/MediaCard.tsx`	Image/video asset display
`ActionButtons`	`components/agent/video-workspace/storyboard/ActionButtons.tsx`	Quote, edit, regenerate actions

Chat Cards (Pipeline Results)

Card	File	Phase
`CreativeBriefCard`	`card/CreativeBriefCard.tsx`	1
`ScriptCard`	`card/ScriptCard.tsx`	2
`StoryboardCard`	`card/StoryboardCard.tsx`	3
`VisualGenPathCard`	`card/VisualGenPathCard.tsx`	4
`VisualAssetsCard`	`card/VisualAssetsCard.tsx`	5
`AudioAssetsCard`	`card/AudioAssetsCard.tsx`	6
`TimelineLinkCard`	`card/TimelineLinkCard.tsx`	7

State Management

videoStore.ts (Zustand) manages:

Storyboard data (elements, segments, shots, audio)
Timeline state (active shot, playhead position, playback)
Generation job tracking
quoteToChat() dispatches storyboard-quote events to ChatInput

SSE Events

Event	Direction	Purpose
`video_storyboard_init`	Backend → Frontend	Initial storyboard push
`video_element_update`	Backend → Frontend	Element asset update
`video_shot_progress`	Backend → Frontend	Shot generation progress
`video_shot_complete`	Backend → Frontend	Shot generation complete
`video_audio_complete`	Backend → Frontend	Audio generation complete

Handled in stream-processor.ts → updates videoStore.

Backend

AI Backend Services

File	Description
`services/video_service.py`	Provider abstractions (Seedance, NanoBanana, ElevenLabs, Suno)
`api/video.py`	FastAPI routes for generation endpoints
`utils/tools/built_in/video.py`	11 LangChain tools for Agent
`repository/prompts/VIDEO.md`	Pipeline orchestration prompt
`repository/skills/video/`	8 SKILL.md files

Web API Routes

Route	Methods	Description
`/api/video/storyboard`	GET, POST	Get/create storyboard
`/api/video/storyboard/[id]/elements`	POST, PATCH, DELETE	Element CRUD
`/api/video/storyboard/[id]/segments`	POST, PATCH, DELETE	Segment/shot CRUD
`/api/video/storyboard/[id]/audio`	POST, PATCH, DELETE	Audio CRUD
`/api/video/generate`	POST	Proxy to AI backend generation

Reference Mechanism

Users can reference storyboard items in chat using the quote button on any Element, Shot, or Audio card. This inserts a chip like [[ELEMENT:name|id]] into the chat input, which the Agent interprets to target specific items for modification.

Get Started

Fundamentals

Core Capabilities

Infra

Channels

Video Workspace

Overview

Architecture

Pipeline Phases

Storyboard Data Model

Frontend Components

Chat Cards (Pipeline Results)

State Management

SSE Events

Backend

AI Backend Services

Web API Routes

Reference Mechanism

​Overview

​Architecture

​Pipeline Phases

​Storyboard Data Model

​Frontend Components

​Chat Cards (Pipeline Results)

​State Management

​SSE Events

​Backend

​AI Backend Services

​Web API Routes

​Reference Mechanism

Overview

Architecture

Pipeline Phases

Storyboard Data Model

Frontend Components

Chat Cards (Pipeline Results)

State Management

SSE Events

Backend

AI Backend Services

Web API Routes

Reference Mechanism