A single handheld iPhone shot of me walking down a side yard, turned into a walk through a lit metal service tunnel. The video goes through the pipeline one stage at a time, from the raw plate to the final composite.

A note on this test

This is an incredibly rough proof of concept, and there are a million things wrong with it. A real VFX artist would be able to do a nice composite and add good floor reflections and shadows. I barely scratched the surface of experimenting with different methods. But it's very easy to camera track this type of shot, model a 3D environment and render it photoreal with AI tools. Then a compositor can do traditional compositing and touch it up with AI.

Without AI, the tracking, modeling, textures, lighting and rendering of the 3D environment would be a lot of work. I did it on my laptop, controlled by Claude Code integrated with node-based ComfyUI, which connects to a service called fal that hosts all the latest models. You could do the same thing, and I think you could connect with Runway too.

Workflow

01 – Capture. Shot on an iPhone 16 Pro Max with the Blackmagic Camera app (1x lens, 23.976 fps). A simple handheld walk toward camera.

02 – Camera Track. Camera solved in Blender: 139 tracks, 0.38 px solve error, matched to a 25 mm lens.

03 – Blender Model. I modeled the tunnel in Blender and applied the solved camera, so the clay render of the environment moves exactly with the plate.

04 – Look Dev. An AI image bake-off in ComfyUI using clay frames as input, comparing Nano Banana Pro, FLUX.2 Pro and Seedream 5 Pro. Seedream 5 Pro won.

05 – Hero Still. Seedream 5 Pro re-lit the clay frame against the chosen look reference to make the photoreal hero frame.

06 – Backplate. LTX-2.3 render-to-real turned the moving clay render into a photoreal plate locked to the camera track. Kling and Wan were also tested for this step.

07 – Fusion Comp. I pulled a foreground matte of me with MatAnyone 2, then comped it in DaVinci Resolve Fusion with a foreground grade to match the tunnel light, grain and light wrap.

08 – AI Reflection Pass. Kling O3 and FLUX video edits generated floor reflections and contact shadows, which were blended back into the comp.

Tools

Blender · ComfyUI · fal · DaVinci Resolve (Fusion) · Claude Code
Models: Seedream 5 Pro, Nano Banana Pro, FLUX.2 Pro, LTX-2.3, Kling O3, MatAnyone 2