Symphony x AI

In collaboration with the Hong Kong Baptist University Symphony Orchestra, I created the visual counterpoint for a musical collage of works by Arvo Pärt, Collage über B-A-C-H, envisioned by Music Director and Conductor Johnny Poon. I created a custom software program that uses AI and machine learning to translate both the orchestra’s sound, as well as the conductor’s physical movement into streams of data that are processed in realtime to guide the real time graphics. By translating the sonic and the physical into the visual, I wanted to emphasize “artificial intelligence with a human touch,” where artists guide the creative process. For the first movement, the visuals transform to echo the contemplative, collage-like nature of the music. They transition into fluid, softer forms that ebb and flow with the dreamlike quality of the Bach-inspired composition. For the third movement, I played with the raw energy of the music, letting the AI-driven visuals pulse and flurry with the kinetic intensity of the orchestra. The textures and colors are dynamic, responding instantaneously to the peaks and troughs of the sonic landscape. 

My real-time, AI-facilitated audiovisual accompaniment encapsulates the concept of 'digital synesthesia'. In essence, synesthesia is a perceptual phenomenon that blends sensory pathways, and my software does just that: it translates auditory (orchestra sound) and kinesthetic (conductor movement) inputs into a large-scale visual tapestry. My goal was to foster a multi-sensory experience that transcends traditional boundaries, and emphasizes the profound influence of technology on our interpretation and perception of artistic expression.

Year Produced: 2023

Materials: Custom LED Stage Design, Projection

Technique: Custom gesture/motion tracking software, audio analysis and visualization, projection and LEledD mapping.

Credits: Curation - Hong Kong Generation Next Arts

Keywords: Max, Touch Designer, MadMapper

Previous
Previous

East West Encounters

Next
Next

Climate Momentum