Combining geolocation and head-tracking to create immersive soundwalks on mobile devices
This project explores the feasibility of locative and spatial audio experiences on mobile devices. By using head-trackers and spatial audio techniques in locative soundwalks, virtual sound sources could be placed at a location and give the listener a realistic sense of directionality. This research tries to identify the challenges inherent in the implementation of this technology.
A prototype of mobile application is implemented in order to evaluate this technology in terms of technical performance and artistic potential.
A number of applications use geo-positional tracking to create locative soundwalks, in which the movements of the listener in space trigger sound sources.
Likewise, techniques to simulate spatial audio over headphones have been recently developed using head-trackers to capture the user’s head movements, especially for Virtual Reality.
However, the possibility to combine these two technologies has only been scarcely explored at the moment. Spatial audio could be used in locative soundwalks to make the experience more realistic and create deeper immersion.
An Android application was developed to play soundwalks in 3D with spatialization techniques.
It communicates with a Nordic «Thingy:52», a multi-sensor kit which was used as a head-tracker.
A design tool for web browsers would let users create their own soundwalks by placing sounds on a map. It wasn't implemented because of time constraints.
A soundwalk was composed to test the application in real conditions. It was created with various radio recordings relating the consequences of the COVID-19 crisis in Ireland.
The findings of this study show that the implementation of spatial and locative audio is possible nowadays on mobile devices, and has been found to be effective in immersing the listener in a virtual audio scene.
The application had good performances in the test runs. The computational load remained easily manageable by the device even when playing multiple sounds. The battery usage was also relatively low when using the application. The application was overall very reactive and the spatial impression created was convincing.
The head-tracker also gave satisfying results. It followed the head movements in a realistic manner and gave a good sense of being part of the virtual audio scene.
The geospatial tracking of the listener was however less reliable. Due to the limitations of GNSS signals, the location data provided tended to be imprecise in certain locations.