The second part of the thesis gives a good practical look into my usual workflow and process for creating sound effects for a mobile game. The secondary goal is to give the reader an insight into the actual creation and design process of the audio content for a mobile game.
The bleeps, bloops and blips
Frogger', an arcade game developed and published by Konami in 1981, introduced a dynamic approach to video game music. This type of approach to game music was further enhanced by 'Dig Dug', an arcade game released by Namco in 1982 where the music would respond to the player's movements.
Rise of home video gaming and chiptunes
A good example of this complex audio content is one of the most popular video games of all time, 'Pacman'. Another audio milestone in games was reached in 1983 when the Digital Instrument Interface (MIDI) was introduced to the market.
Game audio comes of age
Actraiser,” a video game developed by Quintet, pushed this new audio technology to the limit with the game's music score composed by Yuzo Koshiro. One of the very first games with a fully orchestrated live score was 'The Lost World: Jurassic Park', developed by Dreamworks Interactive (Square Enix Music 2007).
Squeezing sound into your pocket
One thing that really advanced the mobile gaming industry was the launch of the App Store in 2008. The App Store quickly became the go-to source for buying and downloading the latest mobile games.
The Ear and the Perception of Sound
The inner ear is mainly made up of the cochlea and vestibular system, which aids in balance. These mechanical vibrations are conducted to the ear canal, which amplifies them before hitting the eardrum, causing it to vibrate at the same frequency. Then the mechanical vibrations continue their journey through the hammer, anvil and stirrup, finally hitting the oval window.
The oval window guides the vibrations into the cochlea, where they interact with the frequency-sensitive hair cells that are connected to the nerve fibers. When the vibrations hit the hair cells, they bend and send a nerve impulse to the brain.
Sound Waves
When there is no disturbance in the air pressure, the particles are at rest. When there is a disturbance in the air pressure, the particles start to move up and down. Also, when the pressure decreases below the rest point, the particles move down and this is called a raft action.
Phase
The Speed of Sound
Frequency
Amplitude
Decibels
Acoustics
Reverberation
Sound effects are one of the three main elements that give a video game its own feel and emotion. They are an integral part of any game and their job is to fully immerse the player in the virtual world of the game. When combined with the other two main elements (music and dialogue), the entire gaming experience reaches a whole other level, one that is more believable, engaging and satisfying at the same time (Viers 2008, 2).
In the film industry, sound effects are usually categorized into five different sections. This method of categorization is also easily applicable to video games, as they become more cinematic in style and have similar elements in common with feature films (Viers 2008, 4).
Hard Effects
Any sound, except music or speech, artificially reproduced to create an effect in a dramatic presentation, such as the sound of a storm or a creaking door (Dictionary.com).
Foley Sound Effects
Background Effects
Production Elements
Sound Design Effects
It's at this stage where you can see the project for the first time and start to paint the overall audio palette and workflow in your mind. One of the best ways to do this is to try to get as much material as you can from the development team and go through it thoroughly. Game design documents, early concept art, rough animations, and even playable builds can really help you dive into a project and start the audio design process.
It's also important to point out that any asset the team creates and designs in the pre-production and production stages will still go through a lot of iterations and changes.
Planning
Scheduling
The audio post-production process in games usually revolves around adjusting and testing the audio content in the game. When the project hits the audio "post-production" phase, almost all the focus is on testing the audio content in the game, while the necessary adjustments are made either to the audio files or directly to the code. I wanted enough time to properly test the audio content in the game, so I reserved a week for it.
Over the course of this week, I've been testing all the audio assets in the game over and over again, noting down the changes that needed to be made either in the performance or in the audio files themselves. Sound designers and composers create audio content for a game in the form of sound effects, ambiences and musical scores.
Preparations
In the production phase, every member of the development team is working closely together to bring the game to life. Level designers focus on level construction and game designers continue to develop the overall game design.
Recording
I wanted the "bird entering the water" sound effect to sound like a mix of a bullet and a spinning object hitting the water at the same time. I started experimenting with objects I had in the recording room and found the right bubbles fairly quickly. These sounds would be played as the various blocks fell into the water and bounced back to the surface.
When everything was ready, I started dropping single oranges into the water tank (so far so good). Most of the stuff I recorded ended up in the game in one form or another.
Editing
I couldn't control the water splash well enough in this small environment, so I decided to overlay some of the recorded sounds with sounds from commercial libraries. I spent three whole days recording content for various in-game events, while also not forgetting to experiment. After three days of recording, I gathered a lot of great audio material to use during the process of creating the final audio assets.
I like to keep things very organized, so I use the folders feature and group similar files under the same folder. This will make the design process easier so I don't have to spend extra time adding gain to the audio files.
Noise reduction
A few distinct keywords in the file name will help you see or understand what sounds the file contains. Peak normalization is an automated process that changes the level of each sample in a digital audio signal by the same amount so that the loudest sample reaches a certain level (Hometracked 2008). As you can see from the picture, I fixed some low frequency kicks and openings in the middle and bottom of the file.
There were also a lot of high frequency clicks in the middle and end of the file that I didn't want, so I removed them as well.
Designing and mixing
I placed a compressor plugin on each of the layers and compressed them slightly. The current drone layer length was about 2 minutes and I wanted the atmosphere to be about 1 minute, so I had to cut 1 minute out of it. Finally, when the melody was added to the sphere, the mood and atmosphere of the water system was complete.
The first thing I did for most of the ambient layers was to add reverb to them. I started layering some of the layers on either side, leaving the duduk layer in the middle.
Mastering
The above examples were just a brief glimpse into the entire process of designing and mixing all audio content. All audio files were subjected to some general procedures, such as high-pass low-pass filtering and gentle down-compression to tame occasional loud noises.
Batch processing / converting
File naming and formats
Delivery and Implementation of sound effects
Once all the files were properly implemented, I began to listen critically to the balance of the audio assets. At this point I opened the sound management file where I could easily adjust both individual and group channel volumes. I would change the volumes of the audio items, save the Sound Manager file, commit the file to the server, and replay the game so I could hear the volume changes in action.
New games are developed at breakneck speed and are often rushed to market. The downside to this is that components of the game are overlooked and often that is the sound of the game.
Team size
In this chapter, I will briefly introduce some of the basic challenges that are often present when working with mobile game audio.
File size
One of the small challenges with mp3 is the fact that it doesn't loop seamlessly. When an uncompressed file is converted to mp3, the compression algorithm adds samples to the audio file, making seamless loops impossible.
Hardware limitations
The most challenging part was writing a short but informative chapter on the history and development of game audio. In the end, I think I managed to write a good short chapter on the history of game audio, its major turning points and technological advances. At one point I wanted to include more examples in this chapter to explain more of the audio content to the reader.
The last major part of the thesis was to present some fundamental challenges that are commonly present in the field of mobile game audio and then try to suggest solutions to those challenges. I'm also glad that I now have a deeper knowledge of the history of game audio and the scientific side of sound.
