Wwise

poses as a severe limitation to the use ofiMuse as well.

There are many design decisions we can learn from, though. Even if digital audio has almost entirely substituted MIDI in games, maybe the idea of using a symbolic representation to in-troduce real-time operations in composition sequences should not be entirely discarded. After all, the Hook and Marker commands used by iMuse came out of Michael Land’s demand as a composer for more control over the dynamic behaviors of his creations inside the game. The Command Queues also point to Farnell’s proposition on procedural audio, since they are essen-tially a data structure that stores a procedure to be executed under an audio-oriented instruction set, making them a more flexible and powerful solution than modernad hoc implementations of sample-based real-time soundtrack effects.

As a reference work for our research, iMuse provides an example of an effective middleware architecture. It is a specially useful design to handle musical sequences, which comes naturally from using a MIDI-based format. But that is also a limitation, since making more abstractly structured compositions becomes more difficult. So is the dependency on synthesizers, which further restricts aesthetic possibilities. Even though we have seen that sample-based designs are too rigid, that does not mean that samples are to be abandoned altogether. It would be rather interesting to see how to conciliate iMuse’s ideas with the rich possibilities of sample audio we have in the present day.

3.4 RELATED TECHNOLOGIES 51 System Description

Wwise is a project-oriented tool: for every game the user wants to create a soundtrack for, a new Wwiseproject must be started and worked on. Inside a project, there are three great hierarchies:

the Actor-Mixer Hierarchy, the Interactive Music Hierarchy, and the Master-Mixer Hierarchy. In each hierarchy, we create and manipulate logical trees whose nodes represent different abstractions in the game soundtrack, and the general flow of data starts at the roots, descends all the way to the leaves carrying control directives, then goes back to the roots bringing the audio itself. TheActor-Mixer Hierarchy allows us to create and manipulate Sound Ob-jects, Motion Effects ObOb-jects, Containers, and Game Syncs. Sound Objects work like symbolic links to audio samples, so essentially they are the leaves in the hierarchy tree, being the source of the base sounds used to compose the soundtrack. Motion Effects Objects are mostly the same, except they provide signals to be used in motion feedback of game controllers instead of audio signal. Containers simply group many object nodes as a single sound, allowing localization of game titles across different languages. Game Syncs are the most interesting: they are the inner nodes of the tree, and define the behavior of their children. We will describe them in more detail soon. TheInteractive Music Hierarchyworks much like the Actor-Mixer Hierarchy. The main difference is that the leaf nodes are Music Track objects, which can be grouped as children of Music Segment nodes to make a “layered” composition. Aside from that, the Game Sync nodes are still there, being the most relevant structure for real-time effects.

Figure 3.13:Screen capture of a piece of theWwise interface. It shows the Hierarchy explorer, as well as a part of the property editor of the selected node, a Sound Object [Aud16b].

TheMaster-Mixer Hierarchy, on the other hand, is very different from the other two. It controls the flow of audio and motion signals that come from the other hierarchies, forwarding them up the tree until they reach the root – which is the audio master bus of the middleware. The roots from each of the other hierarchies are linked to leaf nodes in the Master-Mixer Hierarchy,

allowing the sound designer to specify how each sound is supposed to be played. Figure 3.14 illustrates an example of how all three hierarchies could be connected.

Figure 3.14: Example of object hierarchy in aWwise project [Aud15b].

However, these hierarchies represent only the structure of the soundtrack. To actually execute them to obtain playback, the user needs to specify a series of behaviors, possibly with real-time constraints. For that, there are three main features Wwise supports:Events,Game Objects, and the already mentioned Game Syncs.Events are basically named hooks triggered by the programmers through the Wwise SDK from within the game code. On the sound designer’s side, the user can specify two kinds of Events: Action Events or Dialogue Events. Action Events contain a list of commands much like the command queues fromiMuse. Each command can play, stop or change a property in one or many sound objects across the project hierarchy. Figure 3.15 illustrates an Action Event being executed. Dialogue Events can only play sound objects, but on the other hand they provide a conditional branching mechanism. They map a series of state names to specific sound objects. When the Event is triggered by code, the actual value of the state in that context is checked to know which conditional path to follow in the Dialogue Event list. As the name suggests, this feature was originally intended to support dialogues with multiple voice samples from which to choose from, like when a character responds differently to the player according to a choice he or she made. Figure 3.16 illustrates how Dialogue Events could be used to implement a play-by-play voice in a sports game.

Game Objectsare an abstraction within theWwise digital content interface that the sound

3.4 RELATED TECHNOLOGIES 53

Figure 3.15: Example of an Action Event inWwise. [Aud15b].

designers use to reference possible virtual actors from inside the game. They could be the charac-ters themselves, objects that produce sounds, environment noises, or even User Interface elements with feedback sounds. Game Objects come with properties like volume, pitch, position, ori-entation, and Game Sync information. They must be registered, managed and unregistered by the game code. Many key API functions, likeAK::SoundEngine::PostEvent(), require or optionally accept a Game Object as a parameter. This makes all the consequent effects of that function call be executed from the context of that Game Object. For instance, it makes any sounds played use binaural effects to simulate the 3D spatialization of the corresponding actor in-game.

Lastly, Game Syncs are intermediate nodes in the Actor-Mixer and Interactive Music Hi-erarchy trees. When “played”, each of one them follows a specific, real-time behavior, using its child nodes as building blocks. When evoked with a Game Object as a parameter, they use the Game Sync information it carries as input to the behavior that will be used in that case. The available Game Sync nodes are the following.

• States. They can be assigned a number of property presets (volume, etc.), each with a specific name. The State can only have one activated preset at a time. By activating

Figure 3.16:Example of a Dialogue Event inWwise. [Aud15b].

one from code, all of its property values override the current values in the corresponding properties of child nodes.

• Switches. They are assigned a set of possible state values, each mapping to a different child node. When played, Switches check their current state and forward execution only to the matching child node.

• Real-Time Parameter Controls (RTPCs). As their name implies, they provide a real-time mechanism to control property values in nodes from the hierarchy trees. They do so by being assigned a transformation curve, which is used to map input values obtained

3.4 RELATED TECHNOLOGIES 55 from API calls into the actual property values. This could make a RTPC for the player’s character health possibly correspond to a logarithmic increase in, say, the volume property of the affected nodes.

• Triggers. They are similar to Events in the sense that they must be posted from game code and require a Game Object to define their playback scope. The difference is that they belong in the Hierarchy tree, and the only action they do is play their immediate children, which can cascade into further behaviors and effects down the Actor-Mixer or Interactive Music Hierarchies.

Once all the desired objects are put into the project hierarchies, they can be exported as Sound Banksto different target platforms and languages. This Sound Banks are then loaded into the game through the API of theWwise SDK. The programmers can reference the structures imported either through their name, using character strings, or through their auto-assigned identification numbers.

Critical Analysis

Wwise is a solid alternative for game audio middleware at present, and this is backed by the many respected titles in the industry that have relied on it. Its design approach is very different from usual DAWs musicians are used to. Notably, the notion of time is not clear at first when learning through the first steps of the tool. Even if the user can create a sound object and play with it as much as he or she likes, it is not evident how that is tied to the gameplay. At the same time, this tree-based architecture is powerful and versatile. As long as the user masters the interactions between the sound objects and the game, there are endless possibilities to the combinations and behaviors that can be achieved. Nonetheless, this middleware is ultimately sample-based. It does provide support for custom plug-ins, which allows for lower level DSP-generated audio, but its standard feature set still restricts the sound design of the game to samples and combinations thereof. Besides, it is a commercial product. Its license permits use free of charge for non-commercial projects and a limited use for commercial projects. Anything else beyond that requires payment.

In terms of what we can learn from this technology, the “building blocks” approach we saw in Wwise and its success in the industry might suggest that more abstract interfaces, in con-trast to the traditional timeline based DAWs and composing tools, are also welcome by sound professionals. The types of Game Syncs and the Event system provide a good reference for what AudioKineticconsiders important features for a game audio middleware after ten years of expe-rience in this market. It is important that our own middleware considers the benefits of providing similar tools or, at least, the means for the user to achieve equivalent effects.