Using Audio
Introduction
In this manual, we'll learn how to add music tracks that are synchronized to our animation.
We'll add a music track to a Sequence.
If you're new to sequences, check out Working with Sequences or the Sequence API Reference.
Adding audio tracks to Sequences
We can attach audio tracks to Sequences using sequence.attachAudio. Theatre.js will then play the audio track every time the Sequence is played with the timings in sync.
console.log('Loading audio...')sheet.sequence.attachAudio({ source: 'http://localhost:3000/audio.mp3' }).then(() => { console.log('Audio loaded!')})
In the above example, Theatre.js will:
-
fetch()the audio file -
Create a Web Audio API context.
If the browser is blocking the audio context, Theatre.js will wait for a user gesture (e.g., a click/touch/key down) to initiate it. It's best to prompt the user to initiate audio playback. For example, we can show a button. Once the user clicks on that button (or anywhere else), Theatre.js will initiate the audio context.
-
Decode the audio.
-
Resolve the returned Promise. After this, when you call Sequence.play, the audio track will play simultaneously and in sync.
Create a custom audio graph
If you would like to have more control over audio loading or audio setup, you can provide your own audio graph, like so:
// create an AudioContext using the Audio APIconst audioContext = new AudioContext()// create an AudioBuffer from your audio file or generate one on the flyconst audioBuffer: AudioBuffer = someAudioBuffer// the audio output.const destinationNode = audioContext.destinationsheet.sequence .attachAudio({ source: audioBuffer, audioContext, destinationNode, }) // this promise resolves immediately as everything is already provided .then(() => { sequence.play() })
Or you re-use the Sequence's built-in audio graph, which is exposed through the result of the attachAudio(/*...*/) promise:
sheet.sequence .attachAudio({ source: '/music.mp3', }) .then((audioGraph) => { // this is the audioContext that the sequence created. const audioContext = audioGraph.audioContext // this is the main gainNode that the sequence will feed its audio into const sequenceGain = audioGraph.gainNode // disconnect it from audioGraph.destinationNode so we can feed it into our // own audioGraph. // at this point, audio would be inaudible sequenceGain.disconnect() // create our own GainNode const loweredGain = audioContext.createGain() // lower gain (volume) to 10% loweredGain.gain.setValueAtTime(0.1, audioContext.currentTime) // connect the sequence's gain to our lowered gain sequenceGain.connect(loweredGain) // and connect the lower gain to the audioContext's destination loweredGain.connect(audioContext.destination) // now sequence's audio will be audible at 10% volume })
Advanced Audio+Animation Examples
- THREE.js + music synchronization CodeSandbox - Orb shader
- THREE.js + music synchronization CodeSandbox - Flower animation
Learn more
Want to learn more? Take a look at some more in-depth topics from our manual:
Projects
This guide covers creating projects, managing their states, saving and loading their states, and more.
Sheets
This guide covers Sheets in Theatre.js
Sheet Objects
This guide covers Sheet Objects in Theatre.js.
Prop types
Learn how to customize the props you want to animate with Theatre.js. When creating a Sheet Object, we define the props that we want to animate on it. Props can have different types which can be imported from "@theatre/core".
Working with Sequences
In this guide, we'll explore the tools that Theatre.js offers for creating animations.
Studio
Learn the different parts of the Studio.
Authoring extensions
The Theatre.js Studio API enables you to define extensions that extend the Studio's UI and/or extend the functionality of Studio.
Keyboard & Mouse Controls
A catalog of controls in Theatre.js Studio.
Was this article helpful to you?
Last edited on November 15, 2022.
Edit this page