Development guide: Loading and playing audio

In Javascript, there are two schools of thought when using a third-party library like Superpowered with external audio assets.

You can either load the audio externally outside of Superpowered, then connect an audio source into Superpowered with the Web Audio API to utilize Superpowered's brilliant and efficient DSP abilities. However, with this approach you will not have access to the amazing features of the Superpowered AdvancedAudioPlayer, such as zero player latency. We do not recommend this approach.

The better way is that you let Superpowered handle the audio loading in its entirety which opens up Superpowered's advanced scheduling, time-stretching and transport controls when used in combination with Superpowered's AdvancedAudioPlayer.

This guide demonstrates how to fetch, decode and play audio using Superpowered tools in Javascript.

We're setting up a simple load button which will trigger the fetching of an external audio file, and three sliders controlling the player volume, playback rate and playback pitch.

<div class="container">
    <img class="logo" src="/static/images/superpowered-white.svg" />
    <div class="controls">
        <h1 class="demoTitle">Loading and playing audio</h1>
        <button id="loadAssetButton" onclick="loadTrack()">
          Load and play local track
        </button>
        <h3 id="trackLoadStatus"></h3>
        <div id="bootedControls">
          <span>Player volume</span>
          <input
            type="range"
            min="0"
            max="1"
            step="0.01"
            defaultValue="0.5"
            oninput="onParamChange('localPlayerVolume', Number(this.value))"
          />
          <span>Playback rate</span>
          <input
            type="range"
            min="0.003"
            max="2"
            step="0.001"
            defaultValue="1"
            oninput="onParamChange('localPlayerRate', Number(this.value))"
          />
        </div>
        <span>Playback pitch</span>
          <input
            type="range"
            min="-1200"
            max="1200"
            step="1"
            defaultValue="0"
            oninput="onParamChange('localPlayerPitch', Number(this.value))"
          />
        </div>
    </div>
</div>

Just like the previous guide, we're calling onParamChange in our main application when the sliders are moved.

...
onParamChange(id, value) {
    this.processorNode.sendMessageToAudioScope({
      type: "parameterChange",
      payload: {
        id,
        value
      }
    });
}
...

Within our AudioWorkletProcessor, we need to create an instance of a Superpowered Advanced Audio Player. We set the player to loop when it gets to the the end of a file and we also set a gain value which we will update via our UI later.

...

// Runs after the constructor.
onReady() {
  this.player = new this.Superpowered.AdvancedAudioPlayer(
    this.samplerate,
    2,
    2,
    0,
    0.501,
    2,
    false
  );
  this.player.loopOnEOF = true;
  this.playerGain = 1;

  // Notify the main scope that we're prepared.
  this.sendMessageToMainScope({ event: "ready" });
}

...

Our audio processing callback is simple and will generate the output from the Advanced Audio Player, passing the volume in as an argument to it's processStereo function.

processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
  // Ensure the samplerate is in sync on every audio processing callback.
  this.player.outputSamplerate = this.samplerate;

  // Render into the output buffer.
  if (!this.player.processStereo(outputBuffer.pointer, false, buffersize, this.playerGain)) {
    // If no player output, set output to 0s.
    this.Superpowered.memorySet(outputBuffer.pointer, 0, buffersize * 8); // 8 bytes for each frame (1 channel is 4 bytes, two channels)
  }
}

Loading the audio and handling events

We need to handle the loading of the audio when the button is clicked. The button calls the loadTrack function in the main application.

Superpowered helps you to download and convert audio into the AudioInMemory format required for Superpowered. Under the hood it uses a Superpowered Decoder, which offers the following consistent codec support across all devices and web browsers:

• Stereo or mono pcm WAV and AIFF (16-bit int, 24-bit int, 32-bit int or 32-bit IEEE float).
• MP3: MPEG-1 Layer III (sample rates: 32000 Hz, 44100 Hz, 48000 Hz). MPEG-2 Layer III is not supported (mp3 with sample rates below 32000 Hz).
• AAC or HE-AAC in M4A container (iTunes) or ADTS container (.aac).

It will automatically create a dedicated Worker to download and decode the remote data, so it does not put any work on the thread from which it was called. This is a clear performance advantage over the standard WebAudio API decodeAudioData pattern. The downloadAndDecode helper can be called from the main scope or from inside a Superpowered AudioWorklet.

For an example of downloading and decoding an audio asset inside a Worker, please see Loading audio from within a Worker

...

loadTrack() {
  this.trackLoadingStatusRef.innerHTML = "Downloading and decoding track...";
  this.webaudioManager.audioContext.resume();
  const loadedCallback = this.processorNode.sendMessageToAudioScope.bind(
    this.processorNode
  );
  this.Superpowered.downloadAndDecode(
    "/static/audio/lycka.mp3",
    loadedCallback
  );
}

...

The callback we've provided will send the loaded event over to the AudioWorkletProcessor via the message port. We're also using this message port to handle parameter changes from our sliders above. Let's take a look at how these various types are handled:

...

onMessageFromMainScope(message) {
  if (message.type === "parameterChange") {
    if (message.payload.id === "localPlayerVolume") this.playerGain = message.payload.value;
    else if (message.payload.id === "localPlayerRate") this.player.playbackRate = message.payload.value;
    else if (message.payload.id === "localPlayerPitch") this.player.pitchShiftCents = message.payload.value;
  }
  if (message.SuperpoweredLoaded) {
    this.player.pause();
    this.sampleLoaded = true;
    this.player.openMemory(
      this.Superpowered.arrayBufferToWASM(message.SuperpoweredLoaded.buffer),
      false,
      false
    );
    this.player.seek(0);
    this.player.play();
    this.sendMessageToMainScope({ event: "assetLoaded" });
  }
}

...

Then we need to handle the assetLoaded event within the main application thread, so we can show the UI controls for the player.

...

onMessageProcessorAudioScope(message) {
  if (message.event === "ready") {
    // The processor node is now loaded
  }
  if (message.event === "assetLoaded") {
    this.loadAssetButtonRef.style.display = "none";
    this.playerVolumeRef.disabled = false;
    this.playerSpeedRef.disabled = false;
    this.playerPitchRef.disabled = false;
    this.trackLoadingStatusRef.style.display = "none";
  }
}

...

We've created the following sandbox for you to experiment with.