How We Reduced Video Artifact Production Time for InterDead and Built a Reproducible Pipeline

What began as an attempt to speed up the creation of short in-world videos eventually turned into a controlled, engineering-driven system that defines the visual identity of InterDead.

Context and Objective

In InterDead, short videos function as artifacts — diegetic traces that appear both inside the application prototype, InterDeadProto, and on the project website.

To support steady content production, we needed a mechanism that would assemble these videos quickly, predictably, and with consistent stylistic rules, without re-editing each iteration by hand.

The structural formula behind all artifacts became simple:

audio + background image + emoji overlays aligned to word-level timing

Why Manual Editing Failed

Early attempts in Canva and Adobe Premiere Pro showed that manual assembly created a bottleneck. Each new video required full re-synchronization, stylistic consistency shifted from asset to asset, and even small revisions turned into separate micro-projects.

When we ran manual and pipeline-based production in parallel, the difference became measurable: manual editing averaged 2.5–3 hours, whereas the pipeline stabilized at 35–50 minutes. In practice, this meant a 65–75% time reduction with significantly higher consistency.

The Roughness We Chose to Keep

Pipeline-generated videos inevitably looked more technical than manually edited ones. Surprisingly, this worked in our favor.

The artifact aesthetic survives best when it avoids polished, studio-like motion design. Although FFmpeg would allow procedural animations, and sprite-based overlays could add motion, visual restraint better supports the internal logic of InterDead.

The roughness became a feature, not a flaw.

Pipeline Architecture

All development happens inside InterDeadReferenceLibrary/tools/, and the pipeline remains strictly linear and deterministic.

The process starts with `whisperx_timing_builder`, which takes audio input, runs WhisperX alignment, and produces an aligned.json file with precise word timings. Synchronizing by words — rather than lines — ensures objective, measurable cues.

The next stage is the LLM prompt step, which uses aligned.json, editorial blocks, templates, and the full transcript to generate a structured cues[] array for the overlays. If the ASR misses a fragment, the system marks it explicitly rather than inventing timing.

Full automation could assemble the final configuration automatically, but semantic control is more important than eliminating the last bit of manual oversight.

Finally, `emoji_overlay_video_builder` takes the completed configuration and assembles the video through FFmpeg. The tool uses a fixed emoji pack, a defined safe area, stable offsets, and deterministic overlays. No timeline editor is involved — only rule-based composition.

Supporting Assets

Music is generated via Suno. Commercial use requires an active subscription. Background images come from GPT-based generation and gradually form a recognizable palette and grain profile. Emoji rendering relies on the 72×72 Twemoji PNG set, which offers deterministic cross-platform consistency.

Outcome

The pipeline is not “AI for the sake of AI.” It is a disciplined production method that shortens repetitive work, enforces visual constraints, and keeps full engineering control over generative components.

The system continues to evolve, but its core principles — repeatability, predictability, and stylistic discipline — remain stable.

The full technical specification is documented here:

Reproducible video artifact pipeline