Orchsy: a multi-agent content pipeline for event vendors.
A multi-agent content pipeline for event vendors. Florists, planners and photographers upload a day of raw footage; Orchsy reads it with Claude Vision, decides whether it should become a reel, a carousel or a photo, assembles the edit, writes the caption in the vendor's own voice, and puts it in a queue for a human to approve.
The Problem
Event vendors sit on an enormous archive and post almost none of it. A single wedding produces hundreds of clips; the florist who shot them is loading a truck at 6am and has no time to cut a reel that evening. The default fix is a social media manager at $500–$1,000 a month, which most vendors in the client's founding cohort of 47 companies could not justify.
The brief was to replace that retainer with a system that produces posts good enough to publish — not drafts good enough to start from. Two constraints shaped everything. First, an authenticity rule: never generate fake imagery, only enhance footage the vendor actually shot. Second, the output has to sound like the specific vendor, not like a content tool — which rules out a single prompt applied to everyone.
Architecture
Rendering video is the part that breaks a serverless deployment. ffmpeg on a 20-second reel comfortably outruns a function timeout, so the work is split across two runtimes that share a Postgres job queue: the Next.js app writes a job row and returns immediately, and a long-running Node worker claims it.
Claiming uses SELECT … FOR UPDATE SKIP LOCKED inside a Postgres function, so adding worker instances adds throughput without any coordination layer — two workers polling the same table never take the same row. Jobs that die mid-flight are recovered by age-gated reapers rather than a heartbeat protocol: a row still marked running past its timeout goes back in the pool, and the same pattern rescues clips that stall halfway through ingestion.
The generation pipeline
The pipeline is the product. Everything else is plumbing around it.
The central abstraction is a format-agnostic edit decision list — a zod discriminated union describing the post as structure rather than pixels: which clip, which in-point, how long, what narrative role, what transition. One schema, three renderers. A photo takes the best frame, a carousel orders slides along a narrative arc, a reel hands each segment to ffmpeg. Adding a fourth output format means adding a renderer, not touching the intelligence.
Ordering was the subtlest decision. Pure chronology produces boring edits; pure narrative shuffling produces edits that feel wrong, because a viewer can tell when evening light precedes morning light. Narrative role wins at the top level, and the capture timestamp is the tie-break within a group of equally-weighted clips — verified on real footage by feeding the pipeline a deliberately shuffled morning-to-evening sequence and checking the output came back on the timeline.
Refine — editing by sentence
Vendors do not want a timeline editor. They want to say what is wrong. The approval queue takes plain English, interprets it into typed edit commands, applies them to the current EDL, and writes the result as a new version pointing at its parent — so undo and redo walk a chain of versions instead of mutating a row.
The honest detail in that diagram is the last line. The edit path scales segment durations without re-checking them against the footage available, and the renderer passes -ss in_point -t duration straight to ffmpeg — so a segment asking for more seconds than its clip holds simply delivers what exists. The refined EDL totalled 23.7s; the reel rendered at 20.4s. Clamping durations on the edit path is the next fix.
Inside the product
Approvals happen on a phone between events. Crew Capture is a no-account link a photographer can be handed on site.
Problems worth writing down
A misleading OAuth error cost weeks
Instagram token exchange failed with 'Error validating verification code — redirect_uri must be identical.' Every hour spent on redirect URIs was wasted: Meta returns that error when the client_secret does not validate, and the app secret had been rotated. The lesson generalises — when an error names a parameter, confirm the other credentials in the same request before trusting it.
Dropbox sync advanced its cursor past failures
Files whose analysis threw were logged and skipped, but the sync cursor moved on anyway, so those files were permanently invisible to every later sync. The fix was an idempotent ledger keyed on (vendor, dropbox_file_id), a real failed status instead of leaving rows pending, and per-file outcome counts surfaced in the UI so a partial sync is explicit rather than silent.
Clips with no EXIF vanished
The content tab joined events to clips, and clips only got an event through clustering, and clustering returned early when a timestamp was missing. Three reasonable decisions composing into a hole where uploads disappeared. Now ingestion falls back to upload time, an Unsorted bucket catches anything still unclustered, and a reaper re-queues clips stuck mid-processing.
iPhone HEVC broke frame extraction
10-bit HEVC from newer iPhones produced unusable keyframes until ffmpeg was told to convert the pixel format explicitly. Worth knowing before building anything that ingests phone video: the format your users actually shoot is not the format your test fixtures use.
A green deploy that served a four-week-old build
A production deploy failed type-checking, and Vercel kept the previous good build aliased — so the site looked healthy while quietly serving a month-old bundle. The root cause was a workspace dependency that could not resolve inside the app's build root. Two lessons: verify deploy status rather than inferring it from a successful push, and reproduce the deployment's build context locally before trusting a local build.
Engineering decisions
- Job queue over serverless background work — video rendering outlives any function timeout
- FOR UPDATE SKIP LOCKED for claiming, so horizontal scale needs no coordination layer
- One format-agnostic EDL schema shared by three renderers and validated on both sides of the queue
- Versioned EDLs with parent pointers, making undo and redo a walk rather than a mutation
- Age-gated reapers instead of heartbeats — simpler, and correct under a worker crash
- Natural language as the edit interface; typed commands as the execution layer
- Human approval on every publish, by design, not as a temporary safeguard
- OAuth tokens encrypted with AES-256-GCM at the application layer after pgsodium was deprecated
- Row Level Security on every vendor-facing table; writes through the service role only
Where it landed
~70s Enqueue → finished post3 Output formats, one schema3 Ingestion paths100% Posts reviewed by a human
The platform runs in production: the app on Vercel, the worker on DigitalOcean App Platform, both against a live Supabase project. Every number and screenshot on this page came from an end-to-end run through that deployment — real footage uploaded, analysed, rendered and refined by the deployed worker while writing this up.