(a) Some developers ofSound Wanderer.
De Lucca (left), game and sound designer, and Wilson Kazuo Mizutani (right), game designer and programmer.
(b) Some conference participants playing Sound Wanderer. The game was designed to be played usingPlayStation-compatible gamepads.
Figure 6.5: Pictures taken by De Lucca during the 12th International Symposium on Computer Music Multidisciplinary Research held in São Paulo, June 2016.
game application, but that it can be used together with industry-tested game engines (Godot in our case).
1. Independence from Programmers
Since the Soundtrack Creation Kit is but a set ofPure Data Abstractions and Externals, and the sound designer is free to do anything he or she is capable of in the visual programming language of Pure Data, this means that they have control over all the aspects of the game soundtrack.
The Markov chain implementations in the puzzle rooms ofSound Wanderer are a good example of how much liberty and control the sound designer has when producing the soundtrack. The only part they cannot control are when Commands (along with their parameters) are sent and where the Soundtrack Events are positioned inside the virtual sound space of the Audio Engine, but this is practically inevitable. Only the game application knows about the game state (which carries the timings and the game entity whereabouts inside the virtual world simulation). Even if the Soundtrack Creation Kit had direct access to the game state, it would still require that the host application provided it in some way in the first place, so the dependency would still be there on way or the other.
2. Real-Time Controlled Music Sequencing
The Sequencer feature of the Soundtrack Creation Kit implements support for this System Re-quirement by offering a relatively simple mechanism for sequencing soundtrack behavior. We demonstrated its capabilities in the background music of the first part of the “Damnation Room”
in Sound Wanderer, when the electronic music responds to whether the player is moving by adding or removing notes from one of the melody sequences. Additionally, the Sequencer is not the only way the Soundtrack Creation Kit can achieve real-time controlled sequencing. The Markov chains in the puzzle rooms have their own sequencing mechanism, which manipulates the melody in real-time note by note. This means that even if the Sequencer is too simple for
6.2 ADVANCED FEATURES 119 advanced sequencing implementations, it can still be done by programming it directly in Pure Data.
3. Real-Time Controlled Synthesis
This System Requirement is satisfied by the Synthesizer feature of the Soundtrack Creation Kit, which allows at least as many timbres as the Minimoog synthesizer, and has been successfully demonstrated in the “Damnation Room” of Sound Wanderer too. The second example from Section5.4.5 also shows how it is possible to change a timbre at its wave shape level in a real-time interaction with the game application. Besides, the possibility still remains of synthesizing timbres usingPure Data directly, like in the first example from Section 5.4.5.
4. Music Transitions
There is no particular feature in the Soundtrack Creation Kit that directly meets this System Requirement, but it is nevertheless supported and Sound Wanderer clearly demonstrates that.
During the second part of the “Damnation Room”, the game soundtrack transitions between five different looped drum samples. All transitions there are properly synchronized, displaying no perceptible artifacts when the middleware changes from one sample to the next. The Sample feature from the Soundtrack Creation Kit helps in the case of sample-based transitions, but in the more general case transitions are feasible through a direct implementation thanks the procedural audio approach of the middleware.
5. Variable Mix
The Output Bus Abstraction in the Soundtrack Creation Kit, which is the main audio output of each Soundtrack Event, comes with a volume control which allows for real-time variable mix between Soundtrack Event instances. For a variable mix within the same Patch, a direct Pure Data implementation is required, which is demonstrated by the latest version of our Mari0 extension prototype, with its soundtrack being composed of a variable mix between the drums, the bass and the melody. Thus, this System Requirement is also met.
6. Variable Frequency Filter
The[vorpal_filter]from the Synthesizer feature in the Soundtrack Creation Kit supports this System Requirement by emulating the filter of the Minimoog synthesizer. This is demon-strated by the hi-hat in the electronic music of the first part of the “Damnation Room” inSound Wanderer, since it essentially plays a noise signal band-pass filtered at a frequency modulated by a sinusoidal wave. As with other Requirements, it could also have been done directly inPure Data.
7. 3D Audio Spatialization
By enabling the 3D audio spatialization feature of the OpenAL programming library in the Audio Engine API – more specifically in vorpal::SoundtrackEvent::setAudioSource
– theVORPALmiddleware successfully meets this System Requirement. This is clearly shown by the second example of Section5.4.5, but is also present throughout most of theSound Wanderer soundtrack, specially with the talking obstacles in the second part of the “Damnation Room”.
However, it is thanks to this that all the Soundtrack Events can only output mono sounds, as discussed in Section 4.2.3.
8. Playback Randomization
We have demonstrated support for this System Requirement both at the Markov chains from the puzzle rooms and the talking obstacles in the “Damnation Room” of Sound Wanderer. In the Markov chains, the game manages to randomize music themes on the granularity level of its score, choosing the next note to be played stochastically. In the “Damnation Room”, a recorded speech is chosen from a pool of samples to be played when the player comes near the obstacles.
There is also a randomization in the effect applied to the sample chosen.
9. Polyphony Control
Polyphony control could be implemented at two different levels inside theVORPALmiddleware.
The first is to allow Patches developed with the Soundtrack Creation Kit to control polyphony inside themselves, and this is indeed supported – it is only a matter of using multiple Synthesizers and Samples and regulating how many can be playing at the same time. The second level of polyphony control would be between Soundtrack Events, and this is not supported, because it has a slight conflict with the next System Requirement. While we do consider that this Requirement is met in the current implementation, we intend to improve its support to include polyphony control between Soundtrack Events in future versions of the middleware.
10. Resource Usage Optimization
We support this feature by using C++ smart pointers that self-manage their own allocation process through reference-counted garbage collection. This essentially means that as long as the game code holds a reference to a Soundtrack Event instance, it will be kept in memory, playing its sounds. Unused Events are interrupted and removed from memory, thus optimizing this resource usage. However, this also means that a Soundtrack Event that is referenced in game code cannot be prematurely removed by, say, a polyphony control mechanism. To do that, the Audio Engine would either have to invalidate an Event instance already referenced by game code, or allow unreferenced instances to persist in memory until they expire or their polyphony threshold demands so.