Introduction to Interactive Audio | Basic Concepts
Updated: Aug 7
The concept of audio linearity is the foundation to understand the main differences between audio for media such as television and cinema and interactive audio. The first concepts to examine are the basic differences between linear and non-linear audio.
LINEAR AUDIO
Linear audio is fixed to a strict and never-changing timeline in which all audio assets are executed at the same exact position each time the media is played. Linear audio is not dependant of any changes in real time and therefore, its nature is not interactive nor dynamic. Lineal audio features a static narrative structure.
In a linear world, Darth Vader is always Luke's father as the dialogue remains the same since 1980!
NON-LINEAR AUDIO
One of the fundamental differences of interactive audio is its dynamic and unpredictable nature. It is classified as “non-linear audio,” and we can define it as:
Audio that responds in real time to various changes such as the game's atmosphere, parameters defined by programming, and/or player's direct and indirect actions. Non-linear audio features a dynamic narrative structure.
As you can see, decision making can open an infinite amount of variants in the story and the ending.
Non-linear audio has the following characteristics:
Adaptive: Reaction to conditions or states within the game's internal system or the environment.
Reactive: Reaction to the player's indirect actions—those performed without the need for physical interaction—such as the music changing when near an enemy.
Interactive: A reaction to a direct action by the player, typically in response to the use of a physical surface.
Dynamic: Real-time changes to elements of the audio mix, also known as real-time sound engineering. A good example of this is side-chain relationships between buses for dynamic control of voice volume.
In a non-linear world, Darth Vader can be whatever results from the variables in the game.
The ultimate goal of interactive audio is ensuring a sense of immersion for the player or user.
Randomness is a key element of interactive sound design that allows us to avoid ear fatigue, as it helps eliminate the perception of repetition in the audio material during extended periods of gameplay.
Two predominant approaches to implementing audio in a video game or interactive project can be identified:
Through the native audio engines of game engines such as Unity, Unreal Engine, or Godot, to name a few.
Through audio-specialised middleware like Wwise or FMOD.
AUDIO GAME ENGINES
An audio engine is a native component of a game engine (Unity, Unreal, Godot, etc.) that processes and organizes all the audio for an interactive project. Its main functions are:
Playback - Music, SFX, crossfades, loops, one-shots (sounds that play once per event).
Basic logic - Is Playing, Is Not Playing.
Basic spatialisation.
Digital audio coding and decoding.
Basic processing - Delay, Compresión, Reverb.
MIDDLEWARE
A middleware is a third-party software that provides a specialised interface for implementing interactive audio. It acts as a "bridge" between the sound artist and the game engine.
It allows the user to work specifically on all audio-related aspects without interfering with the work of other development teams/areas working on the video game or application.
We use middleware because it handles:
Playback of reactive or dynamic musical system behaviors, playback of interactive sound effects (SFX), and management of dynamic transitions.
Applies complex randomisation behaviours.
Manages sound banks dynamically.
Offers advanced spatialisation functions.
Coding, decoding, conversion/compression, optimisation and debugging.
Manages the data coming from the game engine via soundbanks and code.
Real Time Digital Signal Procesing (delay, reverb, compresión) which allows for real-time dynamic mixing.
FILE NAMING
Before starting any project, it is important to note that a clear, well-organized naming structure is key to identifying a specific file among the thousands of audio files often involved in a project. Proper audio file naming is crucial for their subsequent retrieval and implementation.
Naming conventions vary by studio and the specific needs of each project, and they can be as granular or detailed as required. Prefixes are used to organize audio files according to their category.
Audio Category | Prefix |
SFX (Sound effects) | sx |
Music (Track, stem, transition or stinger) | mx |
Dialogue | dx |
Voice Over | vo |
Dialogue:
Any voice belonging to a character within the space where the action takes place and which is conversational in nature.
Voiceover:
Any voice that is narrated and external to any character within the scene.
Una de las estructuras más utilizadas para nombrar archivos es el estilo snake_case, el cual utiliza un guión bajo para separar campos.
Aunque las estructuras de nombrado pueden cambiar de acuerdo a las necesidades y preferencias de cada proyecto o estudio, estructuras y ejemplos de nombre como los que a continuacion se presentan pueden ser encontradas muy comumente en la industria.
Para efectos de sonido:
Prefijo_subcategoría_acción/capa_[sub-acción/sub-capa]_variación
Ejemplo: sx_ambiente_bosque_venado_01
Para Música:
Prefijo_nivel_estado_instrumento_variación_bpm/métrica_duración en compases.
Ejemplo: mx_nivel1_exploracion_bajo_parteA_110bpm4-4_L4
Para Diálogos:
Prefijo_personaje_contexto/ubicación_identificador_variación
Ejemplo: dx_mago_batalla_md010_01
Para Voiceovers:
Prefijo_personaje_acto/nivel_identificador_variación
Ejemplo: vo_enemigo_jefe_castillo014_03
This is it for now, but do not miss on the next chapter where we begin to explore the fundamentals of middleware like Wwise!
Sound on!
Comments