A question concerning "dynamic music" in videogames

[al3xand3r]

Well, I was reading through the Zelda TP thread, and many voiced their disappointment for the OST being MIDI and not fully orchestrated. Then some people argued that MIDI lends itself better to "dynamic music" than orchestrations.

To begin, I take it that "dynamic music" in the videogame world refers to music that changes according to what is occurring onscreen. For example, in Zelda TP, there is the boss theme, which turns into a heroic theme when you start causing damage, and switches back to the evil boss theme when the boss begins counterattacking.

Am I correct? Also, I'd like someone knowledgeable on the subject to verify the statement that MIDI lends itself better to "dynamic music" than orchestrations, and explain WHY that is so. I'd also like to know if there are other games with "dynamic music" (in the way I defined it) in other games, since Zelda TP is the first game I noticed that on. Perhaps I have played other games that include it, but it must not have been done right, because I never noticed that effect until now.

[lazygecko]

I can't really see how sequenced music is a better choice. Some clever crossfading usually provides more than satisfactory results.

[Corporal Eschebone]

Wow, and I thought they stopped using MIDI in video games in the 90s.

[lazygecko]

Sequenced files have lasted well into the latter days of the PS2.

[JJT]

Shadow of the Colossus used this approach as I recall, and that music wasn't sequenced.

[Baleshadow]

Shadow of the Colossus used this approach as I recall, and that music wasn't sequenced.

That was the first thing that came to my mind as well.

[OverCoat]

Not necessarily MIDI, though you could have situation-based music in all kinds of forms [raw audio, music modules, etc].

I'll talk about the two sides here and give two shining examples of each. Let's say, Deus Ex and Oblivion. I think both games had fantastic music, though I tend prefer Siren/Michiel over Soule SRY. Anyway, both games also have, in a way, player-controlled music. When you decide to attack someone, the music goes to "battle mode, or if you start talking to someone, a more low-key part of the song starts up, to give focus to the conversation. There is town music which is separate from the Exploration music in Oblivion, which puts you more into the "let's go shopping or steal shit from someone's house" mood. Unfortunately though, the music isn't as varied as in Deus Ex, but I think all it takes is a little extra work that Soule wasn't paid to do

The difference between these games is that Deus Ex uses music mods [.umx], and Oblivion just uses .mp3 files. I believe, however, that it has nothing to do with the format of the music, rather just the skill of the composer and how deep he wants to go. You can use either a Geo Metro or an Escalade to get to the same party.

As for efficiency, I believe that Deus Ex takes the cake. So there's your answer from me, I'd have to go with sequenced music. The music files are small, and the way the songs are sequenced is basically just code for the programmers. This eliminates the need for crossfading, and the composer can actually track an inbetween pattern if he or she wishes to do so. The DISADVANTAGE, however, is that sequenced music can sound fakey or, well, sequenced. Deus Ex still has amazing music, but they weren't really making realistic orchestral scores.

[The Mac Attack]

I don't think it's necessarily true either.

In fact, I believe (someone correct me if I'm wrong) that LucasArt's iMuse system helped them to do these kinds of things.

Take for example in the Curse of Monkey Island, on Blood Island. Whenever you returned to the map screen, it would always have a smooth transition from whatever music was playing to the general Blood Island theme. I think their trick there was mostly to use cymbal swells* to cover up any awkwardness.

Another point in the game which does something similar is the insult swordfighting. The music would change depending on whether you won or lost. The way I think they did it there was to somehow align the change to a beat (or maybe a bar) in the main battle theme, and fade between them at that point.

EDIT: Another example is moving inbetween the four different places on the rollercoaster at the end.

*A roll on a suspended cymbal with yarn mallets, as DrumUltimA has informed me.

[Steben]

Shadow of the Colossus used this approach as I recall, and that music wasn't sequenced.

But it didn't work well.

*dramatic music*

"I'm climbin... climbin' up the Colossus leg... sweet I reached his lower back!"

*music takes five seconds to fade out and fade into the heroic theme*

"Well that wasn't a very good musical segue OH CRAP I DROPPED BACK ONTO ITS LEG" D:

*music takes another five seconds to fade out and and fade back into the dramatic theme*

The advantage of sequenced music is that you can do more musically than a simple fade out/in when the music needs to change up. It's all about those few notes of transition that make the difference, in my opinion.

[lazygecko]

The advantage of sequenced music is that you can do more musically than a simple fade out/in when the music needs to change up. It's all about those few notes of transition that make the difference, in my opinion.

There's no reason it can't be in recorded music. Hell, they do it all the time on TV shows.

[watkinzez]

I don't think it's necessarily true either.

In fact, I believe (someone correct me if I'm wrong) that LucasArt's iMuse system helped them to do these kinds of things.

Take for example in the Curse of Monkey Island, on Blood Island. Whenever you returned to the map screen, it would always have a smooth transition from whatever music was playing to the general Blood Island theme. I think their trick there was mostly to use cymbal swells* to cover up any awkwardness.

Another point in the game which does something similar is the insult swordfighting. The music would change depending on whether you won or lost. The way I think they did it there was to somehow align the change to a beat (or maybe a bar) in the main battle theme, and fade between them at that point.

EDIT: Another example is moving inbetween the four different places on the rollercoaster at the end.

*A roll on a suspended cymbal with yarn mallets, as DrumUltimA has informed me.

qfe. iMuse I think hasn't been bested in gaming today, and it still managed to sound damn good.

I like to think of dynamic music as a realtime cutscene as opposed to the pre-rendered kind. The interactive medium will always benefit from using MIDI in terms of being seamless, and not at all disconnected with the gameplay. You can try and emulate the dynamics a la Final Fantasy VIII (or at least the intro cutscene in the demo, that's all I played), but it's all down to how smart the designer is in transitioning them.

Then again, it certainly wouldn't hurt if Kondo and crew updated their samples for once.

[Corporal Eschebone]

Out of curiosity, is Burnout 2's music sequenced? Because the music and the segues are both top notch.

[watkinzez]

Out of curiosity, is Burnout 2's music sequenced? Because the music and the segues are both top notch.

That stuff was played live, if I remember the credits. Just cleverly disguising the transition with the boost SFX.

[Tricklozen]

Well, I was reading through the Zelda TP thread, and many voiced their disappointment for the OST being MIDI and not fully orchestrated. Then some people argued that MIDI lends itself better to "dynamic music" than orchestrations.

To begin, I take it that "dynamic music" in the videogame world refers to music that changes according to what is occurring onscreen. For example, in Zelda TP, there is the boss theme, which turns into a heroic theme when you start causing damage, and switches back to the evil boss theme when the boss begins counterattacking.

Am I correct? Also, I'd like someone knowledgeable on the subject to verify the statement that MIDI lends itself better to "dynamic music" than orchestrations, and explain WHY that is so. I'd also like to know if there are other games with "dynamic music" (in the way I defined it) in other games, since Zelda TP is the first game I noticed that on. Perhaps I have played other games that include it, but it must not have been done right, because I never noticed that effect until now.

MIDI does lend itself more to interactive music than streaming audio, in the sense that one could control individual sounds, but rarely, if ever has it been used for other purposes than to fade between alternate sections of the same song, like in Super Mario 64; upon entering water, a string section fades in. In that sense, streaming audio could be used to the same effect. How would this be done, then? Would the same song fade in but with a string section mixed in? That would clutter up the sound immensly, so the best solution would be to keep the string section as a separate stream, mixing it in real time on top of the base song, but that would require more system resources and hardware support to pull off, because two streams would now be read from the media and mixed in real time (consoles in particular have little memory).

This leads me to the answer why Zelda TP did not use streaming audio; hardware limitations. Although there would probably be more than enough storage for streaming audio on the media, the GC may have no way of mixing two or more streams in real time in a timely manner while also loading other game data. Sound effects are always available in memory, which is why all the GC has to do is load the sounds once and use the musical performance to generate an audio stream through its hardware mixer.

A readily available example of interactive music using DirectMusic:

http://www.lynnemusic.com/directmusic.html

The engine uses the meta-data about the sounds and segments to align playback or adjust to key changes.

[OverCoat]

It might just be lack of interest on the dev team's or the composer's end. Rather than having a totally ambitious music engine, they'd probably want to work on voice acting or something. Music isn't a huge priority for game developers. Especially in America. In terms of what makes a game a game, music is actually less important than, say, having good ingame textures. I know hardware can handle multiple tracks of raw audio at once, but the engines people use to develop games probably don't accommodate this stuff.

[ifirit]

Streaming audio and sequencers don't have to be directly in competition. Certain game makers use a combination of the two to impliment "dynamic music" in computer and console games. The best case-in-point that I can use is the original soundtrack for Halo. The composers, "Total Audio" (Martin O'Donnell & Michael Salvatori), used both live sections of music with sequenced synths and sounds. From reading the linear notes from the OST, you'll notice that many of the tracks are suites, made into a structured composition from the various parts and pieces used to construct the dynamic music program.

As such, the composers didn't really create the soundtrack in the game as a set piece of music, with the exception to FMV music and other programmatic sections. This was created by having numerous different sections of music composed of individual streaming audio "tracks" and the sequenced pieces. So many that a set piece of music had to be arranged post-production for the original soundtrack, which in itself is oxy-moronic. But, again, this deals more with the issue of composer ability versus sound design decisions.

When creating a dynamic music program, a game developer really has to look at the degrees of freedom in a game in order to place the system into effective use. If the game is fast-paced and requires a fast number of shifts in player activity within a short amount of time, sequenced music becomes a better mode of adapting the music programming, as the individual sections of the music can be much more easily controlled by the engine with less invasiveness. But in games that can be long and steady in pace, because they require greater levels of concentration and attention, a streaming audio approach would be better suited for the game, like the Hitman series which impliments a streaming audio design.

Still, no game falls under a single catagory of gameplay anymore and would be difficult to impliment a single design for the entire game, which many developers tend to do anyway, since it keeps engine sizes small. However, this can often range from having an okay music system with minor bumps and hiccups to just being awful sound design. (We all know at least one game like that.)

Anyway, as games become more interactive, sound design systems need to be able to keep up and using a hybrid of the streaming audio and sequenced music designs will likely produce better results. Music becomes more adaptive to the gamers interactions and more creative music designs can arise, especially for the music game genre.

Still another factor that should be considered is the genre of the music itself. For the most part, the majority of the music under discussion here is orchestral and orchestral-hybrids. For this type of genre, greater care has to be taken into the music programming because of the familiarity of the material. Yet, there are other genres that can get away with using purely sequenced music sound designs because of the uncommonality of their usage in games. Obiviously, electronica and its subgenres can use entirely sequenced sound designs, because of the nature of the genre itself. Games like Syphon Filter and the Splinter Cell series have a more synthetic edge than other games and it works to great effect because of the sequencing. Ambient music is another genre in which dynamic music programs can be purely sequenced music because much of the atmosphere isn't really music, but sound recreation. The original Silent Hill uses completely sequenced music to develop the ambience in a dynamic music system to create an atmosphere of increasing psychological terror.

As such, the way game developers write music is going to change for games that use a dynamic sound design system. Tempo shifts and time signature changes will be handled as individual sections of music. Key changes can be made more organic in the context of an active environment. The implimentation of dynamic music system can be beneficial, but only when proper design choices are made to do so.

Ultimately, it comes down to the designer's decision as to which system works better for a game, as this is a matter of personal opinion. But, there are some decisions that are more appropriate than others. Personally, I'd like to see more innovation in sound design than the tradition orchestral set-ups in games.

[al3xand3r]

From all my time in OCR, this thread has resulted in the most research for me, especially after all that you guys have said and critiqued.

I didn't know there was so much behind something I mistook to be easily implementable.

I'm not a musician by any means, so I have to read and reread all these posts to grasp the concept.

My head hurts.

[Dhsu]

The best way to learn would be to actually try it for yourself. You can download the DirectMusic Producer that Tricklozen mentioned from here. Going through the tutorial would explain a lot of things.

[Mouser X]

Someone asked about games that have "dynamic music" in them. There's quite a few for the N64 that do. Banjo-Kazooie, Banjo-Tooie, Ocarina of Time, Paper Mario (I could be wrong on that one), Yoshi's Story (Is that the right name?), and a few others. Check USF Central, and check out some of the USF sets there. Often, teh "dynamic" games don't have tags yet, so check the preliminary section.

For the most part, it's as Tricklozen said. They add to the original song additional instruments. For example, in Banjo-Kazooie, there's many dynamic tracks. In the game, the song changes, depending where on the map/playfield you are. However, in the USF set, all the instruments from the entire area are played simultaneously. This is why dynamic USF sets are often untagged. To get "proper" songs, you need to track down each individual instrument, and turn it on, or off, depending on the song you're trying to get. It's a lot of work to track that stuff down, and shut it off properly.

So, yes, sequenced based music is helpful for dynamic tracks, since it's usually easier to implement it that way. Though, with enough software/CPU power, you could do Tricklozen's method, and combine new segments into the song.

Anyway, I just wanted to point out some games that use dynamic tracks, as well as a little background info on how they work (depending on the game). Mouser X out.

[Villainelle]

More examples: Metal Gear Solid and Splinter Cell series. Many suspense/stealth games use dynamic cues to great effect.

[Kureejii Lea]

Sonic Heroes (don't look at me like that, it's got some neat music) has a track with about twelve variations to that change around as you go through different sections of the level. Guess it's the same idea there...?

[CE]

Guys, Wind Waker had this too.

[OverCoat]

Yeah. And so did Ocarina of Time

[Scars Unseen]

The first game I remember noticing this on was Skies of Arcadia, during boss fights. Mostly I remember thinking "wow, that's neat," until I got into a fight where my party was constantly going from full strength to the brink of death. At that point the music kept going back and forth and I started thinking "man, that's annoying."

So I'd say that I'm all for subtle manipulations of the music caused by what's going on in game, but make help immersion rather than feel like a gimmick.