Context
By combining street photography in Barcelona with advanced AI motion control and custom sound design, the goal was to introduce a subtle, surrealist layer to static historic architecture without losing the structural integrity of the subject. The approach trys not to us AI for cheap unintentional effects but rather gain control and intention.
The project began with a series of architectural studies of the Santa Maria del Mar church and surrounding El Born & Gothic quarters in Barcelona. To ground the digital manipulation in reality, I recorded ambient city sounds, processed them in Ableton Live and added a harmonic layer to it.
Video
Sound on
Workflow
Unlike standard text-to-video generation which often results in unpredictable "hallucinations," this workflow utilizes Trajectory Control within ComfyUI. It gives Control over:
Path Animation: Using a spline-based editor, I manually drew the "wind" paths across the architectural facades. This allows for precise control over which direction pixels move and at what velocity [01:26].
Pinning & Static Constraints: To maintain the "subtlety" mentioned in the concept, specific architectural anchors (like the church towers) were "pinned" in the workflow [01:18]. This ensures the camera and core structure remain stationary while only the atmospheric elements—like light reflections and shadows—are influenced by the trajectory paths.
Temporal Interpolation: Using start/stop percentages and easing functions (Ease-In/Out) [02:11], the motion was synchronized to the BPM of the Ableton soundscape, creating a cohesive link between the visual "wind" and the audio environment.
