Functions and definition of music techniques in video games
--
Introduction
The video game industry is one of the largest media in the world. It succeeded even Hollywood and the global music industry combined as in 2016 global revenues from games topped $101 billion (Esa, 2016). As the games industry becomes larger, even more attention is being paid to the rigorous study and examination of games.
In this post, the attention is regarded to music in video games. According to Luciano Berio’s definition, music is anything that is pitched, and has some regular division of time.
Linear Music
In most 2d games the music is played as a linear composed piece that is directly tied to the level or the game state. If the playthrough is still in the level and the music has reached his end, the music will restart from the beginning. Then when a new level is reached, the music will quickly change or fade out and it is replaced with the new one. This kind of music is often associated with classic games such as Super Mario Bros. (S.Miyamoto, Nintendo, 1985), Castlevania (H.Akamatsu, Konami 1986), Mega Man (Capcom, 1987) or Super Metroid (Nintendo, 1994).
Functions of Music
This linear methodology of composing music helps to set the broad tone of the different aspects of the game. However, video game music could serve different functions other than setting the mood of the levels. Film music theorist Annabel Cohen (1998) identifies eight functions of film music, each of which could as well be relevant to games:
- Providing continuity
- Inducting mood
- Communicating meaning
- Directing attention
- Providing memory cues (example: leitmotifs, which are some short musical segments linked to a precise character or a place)
- Increasing arousal and absorption
- Contributing to the aesthetics of the film
The first five support the narration in specific ways, whereas the other three involve enhancing the overall experience of the film, (A. Prechtl, Open University, 2016).
Downsides of Linear Music
Berndt and Hartmann (2008) argue that even though the above functions of film music are equally applicable to games, most of them have not been adequately utilized so far in games. This is likely due to the fact that in current implementation of game music, fine grained control over the structure and dynamics of the music is usually not feasible. The music’s ability to support the narrative is therefore limited to relatively long-term, high-level changes which may be adequate for conveying the broad tone and setting but not more intricate narrative details, (A. Prechtl, Open University, 2016).
Interactive Music
As the music becomes interactive, the player’s actions have influence on the music. As K. Collins (2008) said “interactive audio refers to those sound events that react to the player’s direct input”.
“…the nondiegetic music that plays in the background of scenes as underscore becomes diegetic when players decide to have their character play an instrument or sing along with the music”, Karen Collins (Collins, 2008, p. 125).
In some games the location-based trigger-points (Collins, 2008, p. 161) are used as cues to change the music. These triggers are a result of the player’s actions through movement, this change of music is also considered as interactive. An example of a location-based trigger-point is in the beginning of the sci-fi first-person shooter Unreal, (Epic Games, 1998), where the player has crash-landed on a strange planet. When the player character exits the space ship, a piece of immersive music is triggered, as he discovers that his spaceship has crash-landed on a beautiful rocky planet. The trigger for the music is therefore based on the player character’s location in the game space, as he leaves his spaceship.
Another example of interactive music is when it changes as the player is close to dangers, such as enemies. The change of the music is not based on the location in the game space, but according to the distance between the player and the enemy. For example, in Darksiders, (Vigil Games, THQ, 2010), high-intensive composed music is triggered when the main character enters a combat. Then when it ends, it is taken away with a stinger, or simply faded out. A stinger is a short clip of music that can be used to introduce, end or link various sections of a musical piece, (Media Music Now, 2017).
In such cases, the music communicates the high arousal one might expect of combat sequences, and directs the player’s attention to the enemies, but it does so in a generic, mostly uninformative way. That is, during these combat sequences the music simply remains at the same level of intensity without reflecting the actual intensity implied by the specific course of the combat.
In reality, the emotions implied by a game’s combat sequence would likely change throughout its course, depending on multiple factors such as the health status of the main character, and how strong and how many the enemies are. The lack of resolution in the control over the music, however, prevents the music from conveying these, which compromises its ability to fully support the narrative, (A. Prechtl, Open University, 2016).
Indeed, Berndt and Hartmann (2008) argue that game music often only provides a “superficial dramatization” of action sequences. The same can usually be said for other types of game sequences as well, especially when the music operates purely in the background.
Adaptive Music
To extend linear and interactive composed music in games there are two main techniques. The first technique is the adaptive music which addresses the linear use of music. Adaptive music is music that reacts to a game’s state, (K. Collins, 2011). Adaptive music directly connects musical features to game variables. These features can include adding or removing instrumental layers, changing the tempo, adding or removing processing, changing the pitch content, etc. These changes in adaptive music are directly linked to gameplay variables. The adaptivity of music can be understood as a dimension. Low levels of adaptivity may only adapt to a small set of in-game variables, while higher levels of adaptivity may adapt to tens or hundreds of in-game variables (Philippe Pasquier, CoG, 2019).
Generative Music
The other technique is generative music which addresses the creation of musical content itself. Most music is composed by an individual or team of human composers. As argued by P. Pasquier and A. Eigenfeldt (2016) computational creativity is a field that explores the automation of creative tasks, and Musical metacreation (MuMe) is a subfield of computational creativity that addresses automating the creation of music.
As Cale Plut (2019) included in his research a music can be considered generative within a video game if the music is produced by a systemic automation that is partially or completely independent of the gameplay. This independence can have a large range of possibilities. A generative linear system may be almost completely independent of the gameplay — a piece of music can be requested, and is then linearly played through regardless of the gameplay. A highly adaptive generative music system may use a large array of game variables to inform the generation of musical content.
In figure 1, the musical dimensions are displayed with their contextual relationships to each other. Many dimensions are not mutually exclusive, and a single system may address multiple aspects of many dimensions.
These dimensions can be grouped into three dimensional types:
- Musical Dimensions: These dimensions describe how a system structures music.
- Gameplay dimensions: These dimensions describe both how the generated musical content is used in the game, and how the musical generation is informed by the gameplay.
- Architecture dimensions: These dimensions describe inner structure, data, and algorithms used by the generative music system.
One of the examples in the indie scene that used generative music is Fract OSC (Phosfiend Systems,2014), a musical exploration game where the player solves puzzles by moving and interacting with abstracted physics objects. As seen in Figure 2, each of them consists of abstract fragments of actual synthesizer components such as buttons, knobs, sliders; that function as sonic and spatial markers, both allowing the player to create sounds through the game’s built-in subtractive synthesis engine, and to navigate the virtual world through warp points.
As shown in Figure 3, the soundtrack changes and evolve based on how the player understood the game mechanics, starting from static bass drones and subtle synth pads to vibrant arpeggios or lead sequenced melodies that mirror the structures and lighting schemed that are gradually generated as the player dive inside the game world.
While there are no clearly defined “goals” in Fract, the primary purpose of the game is to gradually reconstruct and revive, as shown in Figure 4, a defunct world through an exploration of its physical and sonic possibilities. Sound is the primary building block of gameplay, and the fundamental mediator between the game and the player, (Michael D’Errico, Journal 2016).
The non-linear and open-ended design of Fract:OSC points out many of Bogost’s “procedural” elements of gameplay, as well as a broader aesthetic of emergence that has influenced the “do-it-yourself” attitude of indie game design and development. This aesthetic embraces limited resources to do seemingly unlimited things, such as create & display vast 3d environments using finite computer processing power, or construct complex systems based on a single rule or game mechanic, (Michael D’Errico, Journal 2016).
Conclusions
Modern games tend to push the bar always over the top and creating an emotional music piece is not as easy as it seems to be. Even though linear music composition has different downsides when it needs to follow the player actions, it is broadly used in tons of indie games. Using the interactive and generative music composition is a powerful tool which delivers immersion and can follow exactly every step the character makes along the game. Even though the generative composition is immersive and it always creates different musical pieces, it could tend to be unstable and unpredictable so it is important to choose upon the needs and the budget of the project.
References
- M. Scirea, Affective music generation and its effect on player experience, Ph.D. thesis, IT University of Copenhagen (2017)
- Intelligent Systems, Nintendo, Super Metroid, 1994
- H. Akamatsu, Konami, Castlevania, 1986
- A. Kitamura, Capcom, Mega man, 1987
- Vigil Games, THQ, Darksiders, 2010
- Ryan Clark, Brace Yourself Games, Crypt of Necrodancer, 2015
- K. Collins: From Pac-Man to Pop Music Interactive Audio in Games and New Media, Ashgate Publishing Ltd, Farnham, Farnham (2011)
- Phosfiend Systems, Fract OSC, 2014
- Cale Plut, Generative music in video games: State of the art, challenges, and prospects https://www-sciencedirect-com.ezproxy.leedsbeckett.ac.uk/science/article/pii/S1875952119300795?via%3Dihub#b0015
- Cale Plut, Philippe Pasquier: Music Matters: An empirical study on the effects of adaptive music on experienced and perceived player affect https://www.researchgate.net/publication/336097325_Music_Matters_An_empirical_study_on_the_effects_of_adaptive_music_on_experienced_and_perceived_player_affect
- K. Collins: An introduction to procedural music in video games, 2009
- Michael D’Errico: Worlds of Sound: Indie Games, Proceduralism, and the Aesthetics of Emergence, University of LA, Journal 2016
- Anders B. R. Pedersen: Performative Music in video games, Roskilde University, 2015
- J. Cullimore, H. Hamilton, D. Gerhard: Directed transitional composition for gaming and adaptive music using q -learning, pp. 332–338
