
PERCEIVING SOUND, EMBODYING IMAGE
VISUALIZING SOUND
SONIFYING LIGHT
MASTER THESIS
BERLIN, 2025
ABOUT THE PROJECT
RESEARCH
RESEARCH
This thesis investigates how visuals in live audiovisual performances can emerge as an extension of sound rather than function as an independent or dominant layer. Starting from the premise that contemporary stage productions risk becoming technical spectacles, where visual intensity overrides the listening experience, this research proposes a shift toward a perceptually grounded approach.
Drawing from theories of multimodality, cross-modal perception, and post-Schaefferian sound studies, the thesis introduces the concept of extended listening—a mode of perception where sound is not only heard, but embodied, interpreted, and translated into visual form. Through an analysis of sonic material, including its intrinsic properties and associative dimensions, sound becomes a generative source for visual creation.
This thesis investigates how visuals in live audiovisual performances can emerge as an extension of sound rather than function as an independent or dominant layer. Starting from the premise that contemporary stage productions risk becoming technical spectacles, where visual intensity overrides the listening experience, this research proposes a shift toward a perceptually grounded approach.
Drawing from theories of multimodality, cross-modal perception, and post-Schaefferian sound studies, the thesis introduces the concept of extended listening—a mode of perception where sound is not only heard, but embodied, interpreted, and translated into visual form. Through an analysis of sonic material, including its intrinsic properties and associative dimensions, sound becomes a generative source for visual creation.




METHODOLOGY
METHODOLOGY
The research develops a methodology structured in three phases: analysis, mapping, and composition. This framework enables the translation of sound into visuals through both technical systems and metaphorical correspondences, culminating in real-time performances shaped by ephemerality and feedback loops between performer, system, and audience.
Through theoretical investigation and practice-based exploration, this work argues for an audiovisual practice where sound and image co-evolve, fostering immersive, emotionally resonant experiences that preserve the openness and subjectivity of listening.
The research develops a methodology structured in three phases: analysis, mapping, and composition. This framework enables the translation of sound into visuals through both technical systems and metaphorical correspondences, culminating in real-time performances shaped by ephemerality and feedback loops between performer, system, and audience.
Through theoretical investigation and practice-based exploration, this work argues for an audiovisual practice where sound and image co-evolve, fostering immersive, emotionally resonant experiences that preserve the openness and subjectivity of listening.







PERCEIVING SOUND, EMBODYING IMAGE
VISUALIZING SOUND SONIFYING LIGHT
MASTER THESIS
BERLIN, 2025
ABOUT THE PROJECT


PERCEIVING SOUND,
EMBODYING IMAGE
VISUALIZING
SOUND
SONIFYING
LIGHT
MASTER THESIS
BERLIN, 2025
ABOUT THE PROJECT
BECOMING INTERFACE
BECOMING INTERFACE
Becoming Interface is the practical extension of this research — a project that translates its theoretical framework into a wearable system where the body itself becomes the interface.
Building on the concept of Extended Listening introduced in the thesis, it maps three physiological processes onto a live audiovisual composition, each signal corresponding to a distinct mode of perceiving sound and space. The heartbeat — intuitive, internal, pre-conscious — provides the rhythmic foundation from which all events emerge. Breath — affective, expressive, embodied — modulates emotional tone and atmospheric texture, stretching or compressing the density and light of the piece. Eye movement — intentional, analytical, directional — navigates and disrupts the visual field, translating the quality of attention into spatial behavior: stillness anchors, restlessness fragments.
The body stops being a spectator and becomes the score. Prototype #1 is the first realized output: a 3'20" audiovisual piece tracing the arc from calm presence to sensory overload across four affective states, in which the rhythms of the body shape both what is heard and what is seen.
Becoming Interface is the practical extension of this research — a project that translates its theoretical framework into a wearable system where the body itself becomes the interface.
Building on the concept of Extended Listening introduced in the thesis, it maps three physiological processes onto a live audiovisual composition, each signal corresponding to a distinct mode of perceiving sound and space. The heartbeat — intuitive, internal, pre-conscious — provides the rhythmic foundation from which all events emerge. Breath — affective, expressive, embodied — modulates emotional tone and atmospheric texture, stretching or compressing the density and light of the piece. Eye movement — intentional, analytical, directional — navigates and disrupts the visual field, translating the quality of attention into spatial behavior: stillness anchors, restlessness fragments.
The body stops being a spectator and becomes the score. Prototype #1 is the first realized output: a 3'20" audiovisual piece tracing the arc from calm presence to sensory overload across four affective states, in which the rhythms of the body shape both what is heard and what is seen.
TECHNICAL APPROACH
TECHNICAL APPROACH
The system is built around three wearable sensor modules, each transmitting data wirelessly via OSC to TouchDesigner. Heartbeat is captured through a MAX30102 optical sensor worn on the wrist, connected to a Raspberry Pi Zero that reads the pulse waveform and extracts BPM in real time. Breath is detected by a resistive stretch sensor mounted on a chest elastic band — as the chest expands and contracts, resistance changes are read through an ADS1115 analog-to-digital converter and forwarded to the Pi. Eye movement is tracked using two QTR-1A infrared reflectance sensors embedded in eyeglass frames: positioned to face the eyes, they detect shifts in IR reflection as the pupil moves, with the analog signal converted by a second ADS1115 module and processed via a Raspberry Pi Zero or ESP32.
All three data streams feed into a TouchDesigner patch where signal values are mapped to audiovisual parameters, and sound is generated through Vital VST synthesizers controlled via Python scripting inside the environment.
The system is built around three wearable sensor modules, each transmitting data wirelessly via OSC to TouchDesigner. Heartbeat is captured through a MAX30102 optical sensor worn on the wrist, connected to a Raspberry Pi Zero that reads the pulse waveform and extracts BPM in real time. Breath is detected by a resistive stretch sensor mounted on a chest elastic band — as the chest expands and contracts, resistance changes are read through an ADS1115 analog-to-digital converter and forwarded to the Pi. Eye movement is tracked using two QTR-1A infrared reflectance sensors embedded in eyeglass frames: positioned to face the eyes, they detect shifts in IR reflection as the pupil moves, with the analog signal converted by a second ADS1115 module and processed via a Raspberry Pi Zero or ESP32.
All three data streams feed into a TouchDesigner patch where signal values are mapped to audiovisual parameters, and sound is generated through Vital VST synthesizers controlled via Python scripting inside the environment.
PROTOTYPE 01
PROTOTYPE 01
This prototype traces a perceptual journey through four affective states—relax, focus, tension, and o overstimulation—by translating subtle emotional shifts into audiovisual form.
It invites the viewer into a first-person exploration of a world shaped by inner intensity, where visual glitches and sonic distortions act as markers of psychological transition.
At its core lies a simulation of the body: the heartbeat defines rhythm and gives temporal structure to the environment; breath modulates sonic texture and light, adding emotional nuance and atmospheric stability; eye movement becomes the spatial engine—restlessness disrupts the image, triggering glitches and fragmentations, while stillness anchors attention.
As sound builds, so does the density of feedback and interference. The environment, once soft and diffused, becomes sharp, reactive, and spatially unstable—mirroring the body’s passage from calm observation to sensory overload. Here, perception is not passive: it moves, hesitates, locks on, flinches. The visual field tightens as focus emerges, fractures under stress, and eventually dissolves under overstimulation.
This 3.30-minute piece investigates how we emotionally read space and how the body—through its perceptual modes—writes
itself into the audiovisual language.
This prototype traces a perceptual journey through four affective states—relax, focus, tension, and o overstimulation—by translating subtle emotional shifts into audiovisual form.
It invites the viewer into a first-person exploration of a world shaped by inner intensity, where visual glitches and sonic distortions act as markers of psychological transition.
At its core lies a simulation of the body: the heartbeat defines rhythm and gives temporal structure to the environment; breath modulates sonic texture and light, adding emotional nuance and atmospheric stability; eye movement becomes the spatial engine—restlessness disrupts the image, triggering glitches and fragmentations, while stillness anchors attention.
As sound builds, so does the density of feedback and interference. The environment, once soft and diffused, becomes sharp, reactive, and spatially unstable—mirroring the body’s passage from calm observation to sensory overload. Here, perception is not passive: it moves, hesitates, locks on, flinches. The visual field tightens as focus emerges, fractures under stress, and eventually dissolves under overstimulation.
This 3.30-minute piece investigates how we emotionally read space and how the body—through its perceptual modes—writes
itself into the audiovisual language.
AX ©2026
AX ©2026
©2026