Screenshot of Biophilia

If you’re into adaptive/interactive music, you should definitely check out the Biophilia app by Björk. This ‘full-length app-album’ is an interesting application in which studio-produced music, music apps and visual art meet.

Today is the official release of the alpha version with all the music tracks that can be bought from the free main (mother) app. The beta was published earlier this year.
[Björk website]
[iTunes link]

The example movie contains an explanation of the music system used in the Xbox-game Shortburst. It’s pretty self explanatory and shows the flexible system in real-time.

The description of the video:
“cell-based music”, or “horizontal resequencing” in a browser-based, simple, nln-player, with the music for the Xbox-game Shortburst.
The web-version of the nln-player was built with the Schillmania Soundmanager 2 library, php and javascript. The idea was to shift the focus from organising the musical material with, often complex, data-structures, to a very simple model in which the limitations for the composer were greater, but the administrative work and the needed understanding of (meta-)data was much less.
This same framework was used for the implementation of the interactive music in XNA5 for an Xbox game, Shortburst.

More information on www.greencouch.nl and www.nln-player.com

By the way, it was a great venue, featuring a truly wonderful performance installation by Matthias Oostrik. See the two pictures below I made before the summit started:

Festival of Games in Ottone

Festival of Games in Ottone

[Download the Slides as PDF]
More information and weblinks at FoG.AudioGames.net

See a slideshow below the break.

Game Audio Lab: an educational framework for research and design of realtime, nonlinear sound and music design. Photo © Sander Huiberts

The Music Design Group of the Utrecht School of the Arts under Jan IJzermans in the Netherlands has developed a lab for academic purposes which enables research and rapid prototyping of nonlinear audio. The system offers designers the ability to construct composite variables based on variables that are available in the original game system. In this paper, the architecture of the hardware and software as well as the motivation for this setup are discussed.

Nonlinear and adaptive systems for sound and music in games are gaining popularity due to their potential to enhance the game experience. The framework enables academics to map composite variables and adapt sound and music design in real-time during active game play. Composite variables were found to be more useful than mapping simple game parameters directly to the sound design. To give an example: mapping the state of an adaptive environment to the health level of the avatar often is too obvious for the player.

The paper also discusses the need for open game systems, which help designers to study sound design in realtime settings. Most game platforms are closed and protected by intellectual property. Interestingly, many academics mentioned this during the conference.

Unfortunately, I can’t publish the article here, so if you are interested it’s probably best to visit the AES website. The reference to the article is:

Huiberts, S. Van Tol, R and Went, K. (2009) GAME AUDIO LAB – AN ARCHITECTURAL FRAMEWORK FOR NONLINEAR AUDIO IN GAMES. In: Kelly, M. Proceedings AES 35th International Conference Audio for Games, 2009. Royal Academy of Engineering, London, UK.

Kees presenting at the AES. Photo: Sander Huiberts

Some video demos of applications (Half-Life 2 Source engine, OSC, Bridge software, FMOD and GStoolkit in Max/MSP) are listed below:

Credits for the game lab and these applications go to Arjen Schut, Mark Doeze, Maurice Alberts, Ramon Kerstens and Rogier Habraken. They truly did a wonderful job in this project.

Full demonstration Game Audio Lab Half Life 2: actual game play (select watch in high quality):

Simple demonstration: gunsound impact changes:

Simple demonstration: atmosphere dynamically changes with health status:

[For more information and the slides of this presentation, head over to adaptivemusicsystems.hku.nl]

To make things clear: we’re not talking about composing a song from the beginning to the end (linear music); the composer makes a large amount of musical ‘cells’ and the system selects new cells based on the rules of the composer (nonlinear music). Such an approach can be highly suitable for games, that mostly have a nonlinear character, as the music is able to correspond with the narrative or the presupposed experience of the player. And at least, we’re preventing the repetitive background track that drives players crazy.

The adeptive toolkit is still being developed, and now the developers have published a demonstration video of the toolkit with music composed by Than van Nispen. This is an interesting annotated video which shows the value of this system. What you see is an overview screen with music cells and a two-dimensional dipolar state diagram, which triggers the music. Notice the real-time “Notepad”-information which shows what is going on (real-time!).

The research focused on a basic nonlinear (adaptive) music structure, with a composer (non programmer) friendly approach to use any audio with a basic data design for nonlinear music, to enable prototyping and communication between game developers and composers.

[Watch the annotated nonlinear game music in Adeptive Toolkit v0.9 video here]

[Visit the Adaptive music systems Research group here]

[1] Vincent Akkermans, Bertus van Dalen, Siebe Domeijer, Than van Nispen tot Pannerden and Pınar Temiz.

Update: the adeptive toolkit has been published here. Read info from the devs below the cut:

The Adeptive Toolkit is an interface to a nonlinear music structure.

At the Utrecht School of Music and Technology, Bernard Herrmann’s flexible film composition methods were researched: a method of ‘cell composition’ – a cell being a small musical element or motif. Cells can be succeeded by different other cells, the choice depending on the demand of the (game/interactive) system (for example: more tension).

Cells are connected by transitions. The connections give the sense of a network, hence the term ‘cellnetwork’. Composers can chop their music into flexible cell-pieces, connect these with conditional transitions and value the cells with some variable, for example: “tension – 5″.

A flexible and responsive continuity in music can be created. A composer will then typically find himself making more and different transitions and ‘musical paths’ when he finds out how to serve the demands of the game better.

A video game engine communicates a request to the music structure, for example: avatar health 1. In this case, the music will transition to the ‘almost dead’ music IF the composer indeed provided a convenient transition network through his material.

Generally the ‘cellnetwork method’, reflected in the Toolkit, has been found an interesting way to deal with nonlinear music in contexts.The use of an open-source Max/MSP elaboration of the Cellnetwork Method could provide quick access to editing and expanding the system, as Max/MSP is used by many music technologists, sound designers and composers.

If you like to evaluate this toolkit, I’d suggest to contact the developers. They are interested in collaborating with other parties in order to expand this toolkit.