Crossroads AR app for iPhone and iPad

Painter Teresa Parod and Composer and Software Developer Bill Parod collaborate on multi-media public art. Their work combines mural paintings with situated spatial music in Augmented Reality (AR) mobile apps. Crossroads AR is a new project to bring this approach into a gallery setting, by combining Teresas oil painting and Bills spatial music in a mobile app and multichannel exhibition space. Its debut was at the IRCAM Forum - Special Edition ART/VR Spatialization, Paris, March, 2023.

About the paintings
We come to crossroads all the time in our lives. When two roads intersect, they present three choices in where to go or which direction to take. That impact might be immediate or delayed, minor or profound. You make a decision which direction to go. Either direction could have been a very nice life, but we have pivotal choices. Why do we make the choices? Do we have regrets for not making a different choice? Do we think of Robert Frosts’ “The Road Not Taken”? Or is there really only one path for us? - Teresa Parod
 
About the music
The Crossroads music is formed from over 500 improvised 5-string violin phrases, grouped into roughly 100 musical expressions by 25 different musical characters. Some of these characters play simple phrases in simple harmonies. Others express more nuanced feelings in more complex harmonies and tonalities, ranging from dark to jubilant. All these ensemble groups travel in flocks of various sizes, motion, and expanse - affecting and being affected by others they encounter. The little dramas and natural transitioning of these behaviors is the music’s primary compositional effort. A living musical experience of natural though unexpected evolution - like life on a coral reef, or in a forest, or in an urban plaza - is the aesthetic of this music. That seemed like a good place to start, to explore the concept and connect with the feelings of Teresa’s Crossroads paintings. - Bill Parod

About the app
The main view presents an AR camera that recognizes Teresa’s Crossroads paintings and responds by launching a set of ensembles formed for that painting. The ensemble’s characters are visually represented with ribbons of the imagery with birds taken from Teresa’s mural, Midwestern Birds. All are responsive to each other and with the ensembles already present. In this way, the viewer expands the size and expressive scope of the music. If the original paintings are not available, the app is also responsive to their digital versions. They and more information can be found at earful.be/crossroads.

There are also two VR views. VR1 is a mockup of the IRCAM exhibit space where Crossroads AR was first shown in March 2023. VR2 is a preview of a new work in development for the AppleTV platform, which also uses the ensembles. Each uses variants of the ensembles used in the AR view with slightly different sizes and behaviors. In all cases, the music evolves naturally on its own. One can simply put on headphones, put down the phone, and listen. The longer you listen, the more the characters and their ensembles reveal themselves in different ways

However - for conductors, players, tinkerers, or the fidgety - there are some gestural ways to affect the music:

Shake Shake angled right to stimulate simpler consonant harmonies
Shake angled left to stimulate more complex harmonies

Face-Down Turn the device face-down rest the ensembles
Face-Up Bring the device back face up to wake the ensembles

Zoom VR1 supports one-finger zoom and pan.
Toolbar VR2 supports touch toggling the toolbar off and on. This may be useful when doing video capture.

We hope you enjoy it.