Skip to main content

​
Overview

The Video Workspace enables AI-driven video creation through a structured pipeline. When a project has type: "video", the workspace UI replaces the default trajectory/files tabs with Storyboard and Timeline panels, while the left-side chat orchestrates the generation flow using the Agent’s ReAct pattern.

​
Architecture

┌─────────────────────┐     ┌──────────────────────────────────┐
│    Chat (left)       │     │     Video Workspace (right)      │
│                      │     │  ┌──────────┐ ┌──────────┐      │
│  CreativeBriefCard   │     │  │Storyboard│ │ Timeline │  TOC │
│  ScriptCard          │     │  │  Panel   │ │  Panel   │      │
│  StoryboardCard ─────┼────►│  │          │ │          │      │
│  VisualGenPathCard   │     │  └──────────┘ └──────────┘      │
│  VisualAssetsCard    │     └──────────────────────────────────┘
│  AudioAssetsCard     │
│  TimelineLinkCard ───┼────► Timeline panel
└─────────────────────┘

​
Pipeline Phases

PhaseAgent SkillOutput CardBackend Tool
1creative-briefCreativeBriefCardvideo_output_creative_brief
2script-writer (optional)ScriptCardvideo_output_script
3storyboard-designerStoryboardCardvideo_plan_storyboard
4visual-gen-pathVisualGenPathCardvideo_output_visual_gen_path
5visual-media-generatorVisualAssetsCardvideo_generate_element_image / video_generate_start_frame / video_generate_shot
6audio-media-generatorAudioAssetsCardvideo_generate_narration / video_generate_music
7video-assemblerTimelineLinkCardvideo_assemble_timeline
Utility: resource-analyzer — triggered when user uploads reference files.

​
Storyboard Data Model

videoStoryboard
├── storyboardElement (Key Elements)
│   └── elementAsset (image versions)
├── storyboardSegment (Segments)
│   └── storyboardShot (Shots)
│       └── shotAsset (start frame + video versions)
└── storyboardAudio (Narration & Music)
    └── audioAsset (audio versions)
All tables defined in apps/web/src/db/schema/video.ts.

​
Frontend Components

ComponentPathDescription
VideoWorkspacecomponents/agent/video-workspace/VideoWorkspace.tsxMain container with tab switching
StoryboardPanelcomponents/agent/video-workspace/StoryboardPanel.tsxThree collapsible sections
TimelinePanelcomponents/agent/video-workspace/TimelinePanel.tsxVideo preview + 3-track timeline
StoryboardTOCcomponents/agent/video-workspace/StoryboardTOC.tsxRight-side outline navigation
ElementCardcomponents/agent/video-workspace/storyboard/ElementCard.tsxKey Element display
ShotCardcomponents/agent/video-workspace/storyboard/ShotCard.tsxShot display with status/progress
AudioCardcomponents/agent/video-workspace/storyboard/AudioCard.tsxAudio item with player
MediaCardcomponents/agent/video-workspace/storyboard/MediaCard.tsxImage/video asset display
ActionButtonscomponents/agent/video-workspace/storyboard/ActionButtons.tsxQuote, edit, regenerate actions

​
Chat Cards (Pipeline Results)

CardFilePhase
CreativeBriefCardcard/CreativeBriefCard.tsx1
ScriptCardcard/ScriptCard.tsx2
StoryboardCardcard/StoryboardCard.tsx3
VisualGenPathCardcard/VisualGenPathCard.tsx4
VisualAssetsCardcard/VisualAssetsCard.tsx5
AudioAssetsCardcard/AudioAssetsCard.tsx6
TimelineLinkCardcard/TimelineLinkCard.tsx7

​
State Management

videoStore.ts (Zustand) manages:
  • Storyboard data (elements, segments, shots, audio)
  • Timeline state (active shot, playhead position, playback)
  • Generation job tracking
  • quoteToChat() dispatches storyboard-quote events to ChatInput

​
SSE Events

EventDirectionPurpose
video_storyboard_initBackend → FrontendInitial storyboard push
video_element_updateBackend → FrontendElement asset update
video_shot_progressBackend → FrontendShot generation progress
video_shot_completeBackend → FrontendShot generation complete
video_audio_completeBackend → FrontendAudio generation complete
Handled in stream-processor.ts → updates videoStore.

​
Backend

​
AI Backend Services

FileDescription
services/video_service.pyProvider abstractions (Seedance, NanoBanana, ElevenLabs, Suno)
api/video.pyFastAPI routes for generation endpoints
utils/tools/built_in/video.py11 LangChain tools for Agent
repository/prompts/VIDEO.mdPipeline orchestration prompt
repository/skills/video/8 SKILL.md files

​
Web API Routes

RouteMethodsDescription
/api/video/storyboardGET, POSTGet/create storyboard
/api/video/storyboard/[id]/elementsPOST, PATCH, DELETEElement CRUD
/api/video/storyboard/[id]/segmentsPOST, PATCH, DELETESegment/shot CRUD
/api/video/storyboard/[id]/audioPOST, PATCH, DELETEAudio CRUD
/api/video/generatePOSTProxy to AI backend generation

​
Reference Mechanism

Users can reference storyboard items in chat using the quote button on any Element, Shot, or Audio card. This inserts a chip like [[ELEMENT:name|id]] into the chat input, which the Agent interprets to target specific items for modification.