Edit Videos and Design Images with Claude code or Codex
When activeView=canvas is reported by the hook or editorctl get-state:
There are EXACTLY 3 canvas options. No others exist.
canvas-run-paperjs)canvas-run-matterjs)generate-image + canvas-add-image)REMOVED — do NOT offer these under any name:
When the user asks for something visual in canvas mode, present ONLY these three options using these EXACT labels. Copy them verbatim. Do not invent a fourth option or rephrase option 1 as anything design-related.
API Bridge: POST http://localhost:51847 — JSON body {"command":"<name>","args":{...}}
import urllib.request, json
def call(command, **args):
data = json.dumps({"command": command, "args": args}).encode()
req = urllib.request.Request(
"http://localhost:51847", data=data,
headers={"Content-Type": "application/json"}, method="POST"
)
return json.loads(urllib.request.urlopen(req).read())["result"]
call("ping") # → {"status":"ok","version":"1.0.0","port":51847}
call("help") # → full command list
| Command | Args | Returns |
|---|---|---|
get_project |
— | Full project JSON |
get_settings |
— | Model + provider config |
| Command | Args | Returns |
|---|---|---|
list_assets |
— | [{id, name, path, type, duration, semantic}] |
get_asset |
assetId |
Single asset with semantic + transcript |
import_files |
paths: string[] |
Imported asset records |
transcribe_asset |
assetId, language? |
{segments} via Whisper |
embed_assets |
all?: bool |
{embedded, total} via text-embedding-3-small |
search_media |
query, limit? |
Cosine similarity search (falls back to keyword) |
search_spoken |
query, limit? |
Substring search within transcribed segments |
| Command | Args | Returns |
|---|---|---|
list_sequences |
— | All sequences |
create_sequence |
name |
New sequence |
activate_sequence |
sequenceId |
Activated sequence |
get_tracks |
— | [{id, name, kind, clipCount}] for active sequence |
add_track |
kind: video|audio|caption |
Updated sequence |
| Command | Args | Returns |
|---|---|---|
list_clips |
sequenceId? |
All clips sorted by startTime, with trackKind |
add_clip |
assetId, trackId, startTime, duration?, inPoint? |
{clipId, clip} |
remove_clip |
clipId |
{success} |
move_clip |
clipId, startTime |
{success} |
trim_clip |
clipId, inPoint?, duration?, startTime? |
Updated clip |
split_clip |
clipId, time |
Updated sequence |
duplicate_clip |
clipId, offsetSeconds? |
{clipId, clip} |
update_clip_label |
clipId, label |
{success} |
| Command | Args | Returns |
|---|---|---|
list_effects |
— | Available effects + parameter docs |
add_effect |
clipId, effectType, parameters |
Updated project |
remove_effect |
clipId, effectId |
Updated project |
set_speed |
clipId, speed |
Updated project (0.1–10× playback) |
set_volume |
clipId, volume |
Updated project (0–4, 1=normal) |
set_transition |
clipId, side: in|out, type, duration? |
{success} |
| Command | Args |
|---|---|
undo |
— |
redo |
— |
| Type | Parameters |
|---|---|
fade_in |
duration (sec) |
fade_out |
duration (sec) |
color_grade |
brightness (-1→1), contrast (0.5→2), saturation (0→3) |
blur |
radius (px, default 5) |
sharpen |
amount (0→3, default 1) |
crossfade · dip_to_black · wipe · slide
inPoint = position in the source file where clip playback startsstartTime = position on the timeline where the clip is placedduration = how long the clip plays (can be shorter than source)project:updated → UI reflects changes livelist_clips(sequenceId) and check that each clip starts exactly when the previous clip ends if the edit is meant to be continuous.create_sequence or activate_sequence is enough on its own. Always verify the active sequence afterward with list_sequences or get_project.startTime from the cut plan.MANDATORY — NO EXCEPTIONS: If the user's message contains any URL or domain, you MUST run the fetch commands below and extract brand tokens before writing any canvas/design code. Do not use assumed or memorized brand colors. Fetch first, design second — every time. If the fetch fails, ask the user to paste the hex values; never fall back to guessing.
curl -sL "<url>" | grep -Eo '(#[0-9a-fA-F]{3,8}|font-family:[^;"}]+|font-size:[^;"}]+|border-radius:[^;"}]+)' | sort -u | head -60
Extract: background colors, surface colors, primary/accent colors, text colors, font families, font sizes/weights, border radii, spacing, and box-shadows. Also find the logo — it's the ground truth for brand colors and visual tone. Hard-code those exact values — never guess when a real URL is available.
# Find logo
curl -sL "<url>" | grep -Eo '(src|href)="[^"]*logo[^"]*"' | head -10
curl -sL "<url>" | grep 'og:image'
Use canvas_run_matterjs to place a live interactive physics scene on the canvas.
| Tool | Args |
|---|---|
canvas_run_matterjs |
id? (frame id to update), name?, script (JS string), width?, height? |
canvas_add_frame |
name, width, height, mode: 'matterjs' |
canvas_matterjs_scene |
style (balls/stacks/pendulum/bridge/ragdoll), gravity?, count?, colors? |
canvas_frames |
— → {frames: [{id, name, mode, width, height}]} |
Pattern A — Self-contained (rich UIs, custom 2D drawing, rAF loop)
const { Engine, Bodies, Composite, Mouse, MouseConstraint, Events } = Matter;
const W = 390, H = 844;
const engine = Engine.create({ gravity: { y: 1.5 } });
const canvas = document.querySelector('canvas'); // ← always use this; never create a new one
canvas.width = W; canvas.height = H;
const ctx = canvas.getContext('2d');
// ... add bodies, set up mouse ...
function loop() {
Engine.update(engine, 1000/60);
ctx.fillStyle = '#121212'; ctx.fillRect(0, 0, W, H);
// draw with ctx
requestAnimationFrame(loop);
}
loop();
Pattern B — Bodies only (template manages engine/render/runner)
// engine, render, width, height, and all Matter vars are already declared — do NOT redeclare them
var ground = Bodies.rectangle(width/2, height+25, width, 50, { isStatic: true, render: { fillStyle: '#334155' } });
var ball = Bodies.circle(width/2, 50, 30, { restitution: 0.8, render: { fillStyle: '#5b82f7' } });
Composite.add(engine.world, [ground, ball]);
engine.gravity.y = 1;
// Runner.run and Render.run are called automatically after this script — do not call them here
Runner.run needs two args: Runner.run(Runner.create(), engine) — single-arg Runner.run(engine) silently does nothing in Matter.js 0.20.0element: document.body in Render.create — spawns an invisible second canvas; use canvas: document.querySelector('canvas') insteadconst engine or let engine in Pattern B — the template already has var engine; redeclaring with const/let causes a SyntaxError that silently blanks the canvasRender.run / Runner.run in Pattern B — the template does this after your script runs; calling them twice causes double-speed or broken renderingEngine, Bodies, Composite, etc. are already destructured; const { Engine } = Matter will SyntaxErrorengine · render · width · height · Engine · Render · Runner · Bodies · Composite · World · Body · Events · Constraint · Mouse · MouseConstraint
editorctl canvas-add-frame "Name" 390 844 matterjs
editorctl canvas-run-matterjs <frameId> "$(cat script.js)"
editorctl canvas-frames
editorctl canvas-done
Use Remotion to generate animated video assets (title cards, lower thirds, slideshows, motion graphics) and import them directly into the timeline.
| Tool | Args | Returns |
|---|---|---|
video_editor_list_remotion_compositions |
— | List of composition IDs |
video_editor_render_remotion |
compositionId, outputFilename?, props?, durationInFrames?, fps? |
{outputPath, assetId} |
npm run remotion:studio # live preview at localhost:3000
npx remotion render remotion/src/index.ts <ID> out.mp4 --props '{"key":"value"}'
| ID | Props |
|---|---|
TitleCard |
title (str), subtitle? (str), backgroundColor, textColor, accentColor — 150 frames @ 30fps |
Slideshow |
images (abs path[]), frameDuration (frames), transitionDuration (frames) — 300 frames @ 30fps |
remotion/src/compositions/MyComp.tsx — export myCompSchema (zod) + MyComp componentremotion/src/Root.tsx with <Composition id="MyComp" ... />remotion-renders/ and are auto-imported when using the MCP tooldurationInFrames). Default fps = 30. 30 frames = 1 second.useCurrentFrame() + spring() + interpolate() for animation.assets = call("list_assets")
tracks = call("get_tracks")
vid = next(a for a in assets if a["type"] == "video")
vt = next(t for t in tracks if t["kind"] == "video")
# Place first 10s of video at t=0
r = call("add_clip", assetId=vid["id"], trackId=vt["id"],
startTime=0, duration=10, inPoint=0)
cid = r["clipId"]
# Fade in over 1s, color grade, transition out
call("add_effect", clipId=cid, effectType="fade_in", parameters={"duration": 1.0})
call("add_effect", clipId=cid, effectType="color_grade",
parameters={"brightness": 0.05, "contrast": 1.1, "saturation": 1.2})
call("set_transition", clipId=cid, side="out", type="dip_to_black", duration=0.8)