Project VVRMA (Virtual CCRMA): Adventures in Computer Music Land!

CCRMA, Stanford University (kunwoo@ccrma.stanford.edu, ge@ccrma.stanford.edu)

---

Project VVRMA is an audiovisual interactive field trip to CCRMA (Center for Computer Research in Music and Acoustics, Stanford University) - reimagined in fully immersive VR. Funded by Stanford Accelerator for Learning of the Graduate School of Education, this project is a "VR Exploratorium for Music and Technology".

Project VVRMA embodies two main design ethos: 1) teach how computer music works and 2) express why it is beautiful. The target audience is the general public, especially people with no knowledge or experience with computer music. VVRMA is a large-scale VR design project that culminates new explorations of design approaches, philosophies, and frameworks. VVRMA is an act of tool-building, having a functional aim of education and aesthetic aim of expression.

VVRMA presents a virtual environment inside and outside of the Knoll, a historic mention that used to house Stanford's president and later the Music Department before becoming home to CCRMA in the 1980s. Upon navigating the building, users explore "zone of interests" (ZOIs), including computer music laboratories, studios, maker spaces, classrooms, and concert halls. However, this is where virtual reality ceases to resemble physical reality, for physical laws do not limit VVRMA. Each ZOI becomes its own fantastical, microcosmic playground.

VVRMA explores VR as an audio-driven pedagoical tool, an expressive medium, and a place for social interactions, centered around the field of computer music. The underlying audio-driven mechanics and dynamics incorporate real-time generative audio, developed in Chunity (ChucK + Unity).

[A Home in VVRMA]

We created a 3D model of CCRMA using Blender. We attempted to use AI 3D scanning tools at first, but the resulting vertices were misaligned and insufficient for accurate modifications. Thus, every part was hand-crafted. Having control over the vertices was crucial as we try to manipulate the environment (opening windows and doors, sacling rooms, adding materials and texture, and more). The building separates the Zone of Interests effectively, connects to the physical space of CCRMA, and creates an overall structure for the space of the "Exploratorium". As it holds a lot of similarities to the actual space, many different layers of context are transferred to the virtual medium, which the designers can take advantage of by introducing recontextualizing elements to the building.

[Zone of Interest #1 - From Sound to Brain]

This Zone of Interest is mainly inspired by the neuromusic lab at CCRMA. The users mainly learn about how sound pressure translates to mechanical energy and electric signals to the brain. The ZOI takes a form of a dark ride in amusement parks, where the user travels via boat to the ear canal, timpani, ossicles, cochlea, basilar membrane, and the brain. In the end, the user opens a door on a brain that transports the user back to the CCRMA neuromusic lab.

As pedagogy and expression are the two design ethos of VVRMA, ZOI 1 strives to educate clearly on the roles of each ear component, while also providing room for expression. For example, a narration explains how basilar membrane transfers electric signals to the brain based on the frequency content of the input audio. However, when enough signals are transmitted, the brain opens and flowers bloom out of it. This expresses how signals are more than just electric signals. They carry information, memory, meaning, emotion, music, and humanness. Sound-to-brain is important, yet brain-to-sound is humanistic.

The underlying mechanics and dynamics are audio-driven - the users can play soundtracks or make their own sounds using the headset's microphone, and observe how they interact with different elements in the ear, such as the ear canal, timpani, and the basilar membrane. The input sounds drive visual algorithms automatically.

[Zone of Interest #2 - The World of Artful Design]

This Zone of Interest is mainly inspired by Artful Design, written by Professor Ge Wang. The users mainly learn about audiovisual interaction designs, artful design, and humanistic engineering, while they experience playful musical environments like MIDI.CITI.VR. The ZOI begins from Professor Ge Wang's office, where the office itself becomes a vessel that launches into the sky. There are different layers in the sky, each representing a feature in Artful Design.

[Future Works]

Continuing our journey with the current prototype, we plan to develop more ZOIs and implement them inside VVRMA. The impending ZOI is on the rise of a Stanford VR Orchestra (SvOrK), a full-fledged research laboratory for new interfaces for musical expression in VR as well as an exploration of the social and cultural aspects of musical performance in VR. This will be our main focus for 2023-2024 at the VR Design Lab at CCRMA, and will investigate these questions from the roles of performers as well as audience / concert-goers. SvOrK will culminate in a new Zone of Interest for VVRMA but also will set into motion a new research and artistic initiative at CCRMA.

Morever, we plan to continue developing new ZOIs. For example, a ZOI on digital signal processing guides users to the realms of physical modeling with virtual wind instruments or guitar strings at the size of the Golden Gate Bridge and interactive learning of Fourier Transforms. In the Max Lab, the users may program computer music with Lego blocks. In professor Chris Chafe's office, the user might enter a portal, where poeple from different parts of the globe gather to play music together and learn about networked music performances. In the VR Lab, the user puts on a VR headset to enter "virtual, virtual reality, or VVR". These are infinite possibilites in exploring the field of computer music as as CCRMA touches on a diverse scale of works.

We aspire to release VVRMA as a free, publicly available VR app. In the app, we plan to create an exit survey (also in VR) to measure how VVRMA visitors meet the learning outcomes. The survey is a qualitative evaluation to learn the interplay between education and expression, the effectivenmss of VR's unique characteristics in learning, a sense of being and humanistic reflections, and the kinds of knowledge on computer music they acquired. From these, the author hopes to fomulate a set of design principles for designing an educational environment in VR.

[Websites]

CCRMA: ccrma.stanford.edu

CCRMA VR Design Lab: xr.stanford.edu

ChucK: chuck.stanford.edu

Chunity: chuck.stanford.edu/chunity

[Acknolwedgement]

We would like to thank the Stanford Accelerator for Learning at the Stanford Graduate School of Education for the virtual field trip grant, guiding us through its various stages, and allowing us to make this project possible. Beyond the prototype of VVRMA, the journey continues to further research in exploring the virtual medium with the lens of computer music.