How to integrate Superpowered

Javascript introduction

For the purposes of this guide, we'll be using vanilla ES6 Javascript to demonstrate how to get started with the Superpowered SDK. Using Superpowered in Javascript requires the use of WebAssembly to achieve the blazing speed that Superpowered is capable of. Setting up Javascript to handle realtime low latency audio with WebAssembly might be a little outside of what you're used to as it requires thinking about using multiple threads and WebAssembly Linear Memory. Don't worry, we've made it easy to understand.

Please note that setting Superpowered up correctly allows more of the code you write to be transferred between JS and C++ with as few syntax changes as possible.

---

What is WebAssembly?

WebAssembly is a rapidly evolving technology with great momentum. It allows for high-performance data processing in the browser. In short: what needed to be done in a native app, can now be possible in the browser. We've been able to take the Superpowered C++ SDK and port it to WebAssembly, bring all the great unique features you know and love to the web.

Within the last few years WebAssembly has become very widely supported, to a point where it is now a viable technology to use in production. If you'd like to see the wide range of browsers supported, see here.

---

What about the Web Audio API?

Superpowered uses the Web Audio API to set up audio I/O in the browser. But the Web Audio API has inconsistencies and slightly different DSP implementations across browsers/devices which mean you can't guarantee your great product will perform and sound identically across devices. One reason why you need Superpowered!

Superpowered runs within the Web Audio API's AudioWorklets, that are scheduled within a standard WebAudio AudioContext and can be used in standard WebAudio API nodes. This means you can leverage API such as getUserMedia and the MediaSourceNode to get live microphone signals into the Superpowered ecosystem. Once within the superpowered ecosystem you can perform all of your processing chain in one AudioWorkletProcessor, or chain multiple AudioWorklets together in series or parallel.

---

Loading and serving the library

The Superpowered library is not yet available on NPM, only GitHub.

superpoweredSDK/web-audio-javascript-webassembly-SDK-interactive-audiohttps://github.com/superpoweredSDK/web-audio-javascript-webassembly-SDK-interactive-audio

The Superpowered Javascript SDK

Please download the superpowered library to your computer first.

git clone https://github.com/superpoweredSDK/web-audio-javascript-webassembly-SDK-interactive-audio superpowered

You'll need to both copy the Superpowered library into you Javascript application source code so it can be referenced and bundled by your main application, but you should also make the library available to be served statically and available to your frontend application at runtime. This is because the AudioWorklet processors that we will be writing later we need to be able to import the Superpowered WebAssembly and associated helper files, outside of the scope of your bundled JS application.

The library is just a single file (Superpowered.js).

---

Loading the library

Although Superpowered can be loaded into an existing audio context, we recommend your application's main AudioContext is created by the method we provide in the SuperpoweredWebAudio helper class. This extended AudioContext offers useful tools to make sure you can get audio in and out of Superpowered efficiently and with ease across browsers. Behind the scenes, you'll still have a standard AudioContext which you can use elsewhere in your application if required.

To load Superpowered into your Javascript application, the following steps should place in order:

1. Initialize the Superpowered WebAssembly with passing your license details. Note the async nature of this method.
2. Create an application wide WebAudioManager with the SuperpoweredGlue wrapper, passing in both the required samplerate and the instantiated WebAssembly.

In ES6 Javascript, that looks like this:

import './Superpowered.js';

const minimumSampleRate = 44100;

// option 1: if you don't use a transpiler or package manager that may change the source code of SuperpoweredGlue
const superpowered = await SuperpoweredGlue.Instantiate('ExampleLicenseKey-WillExpire-OnNextUpdate');
// option 2: if you use a transpiler or package manager that may change the source code of SuperpoweredGlue, such as Babel or Webpack
const superpowered = await SuperpoweredGlue.Instantiate('ExampleLicenseKey-WillExpire-OnNextUpdate', '/files_with_unchanged_source/Superpowered.js');

const webaudioManager = new SuperpoweredWebAudio(minimumSampleRate, superpowered);

After that, you'll have access to the helper methods under webaudioManager, and the standard AudioContext it sits on will be under webaudioManager.audioContext, with all of the standard WebAudio APIs available such as webaudioManager.audioContext.createGain() or audioEngine.webaudioManager.audioContext.currentTime for example.

---

Let's make some noise

We're going to keep things simple, so we'll be setting up a Superpowered Generator to create a sine tone, then pass that to the speakers. Make sure you turn down your speakers, unless you have a particular thing for a mathematically perfect A below middle C.

Setting up the required parts

AudioWorkletProcessor script

First, we need to create (and serve up) an AudioWorkletProcessor which will house a Superpowered Generator instance.

AudioWorkletProcessors operate on a separate, real-time audio thread, this is what makes them suitable to feed the audio stack with an uninterrupted and steady flow of data. But because of this, they are isolated and operate outside of the main Javascript thread. They should be fetched by the browser at runtime and cannot be bundled like traditional Javascript files.

// toneProcessor.js
// Import the SuperpoweredWebAudio helper to extend the SuperpoweredWebAudio.AudioWorkletProcessor class.
import '../Superpowered.js';

class ToneProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {

    // Runs after the constructor.
    onReady() {
        // Create an instance of a SP generator class.
        this.generator = new this.Superpowered.Generator(
            this.samplerate, // The initial sample rate in Hz. this.samplerate is inherited from the extended SuperpoweredWebAudio.AudioWorkletProcessor class.
            this.Superpowered.Generator.Sine // The initial shape.
        );

        this.generator.frequency = 440; // Generate a simple test tone at A4.

        this.genOutputBuffer = new this.Superpowered.Float32Buffer(2048); // An empty array to store the mono output of the generator.

        // Pass an event object over to the main scope to tell it everything is ready.
        // This is the callback defined as the final argument in webaudioManager.createAudioNodeAsync in the main thread.
        // Here we use it to tell the main thread that audio I/O is ready and processing audio.
        this.sendMessageToMainScope({ event: 'ready' });
    }

    // onDestruct is called when the parent destruct() method is called.
    // You should clear up all Superpowered object instances here.
    onDestruct() {
        this.generator.destruct();
        this.genOutputBuffer.free();
    }

    processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
        // Generates the next chunk of the sine wave.
        this.generator.generate(
            this.genOutputBuffer.pointer, // Output, pointer to floating point numbers. 32-bit MONO output.
            buffersize // Number of frames.
        );

        // Next we need to convert the mono output of the generator into the interleaved stereo format that the AudioContext expects.
        this.Superpowered.Interleave(
            this.genOutputBuffer.pointer, // Left mono input.
            this.genOutputBuffer.pointer, // Right mono input.
            outputBuffer.pointer, // Stereo output - this is routed to the AudioWorkletProcessor output.
            buffersize // Number of frames.
        );
    }
}

// The following code registers the processor script for the browser, note the label and reference.
if (typeof AudioWorkletProcessor !== 'undefined') registerProcessor('ToneProcessor', ToneProcessor);
export default ToneProcessor;

The AudioWorkletProcessor file(s) will need to be served up to the client side application at runtime, much like the Superpowered WebAssembly library above. For this example we'll have the toneProcessor.js file at http://localhost/static/processors/toneProcessor.js.

Note that the AudioWorkletProcessor is not bundled with your source code and it pulls its own copy of SuperpoweredWebAudio from the statically served Superpowered library files. It's important to understand where files are being served from here.

Firefox ES6 compatibility

With the release of Superpowered v2.6.1, we've updated our library to make the most of the increased AudioWorklet support across all the major browsers. We are able to stop using the legacy WebAudio API ScriptProcessorNode in the majority of cases now and use the more powerful AudioWorkletNode instead.

In short this means Superpowered will now work on a dedicated audio thread, away from the UI main thread in the latest versions of Chrome, Firefox, Edge and Safari, on mobile and desktop!

The ScriptProcessorNode is still used where AudioWorklet's are not supported, which roughly equates to browsers that haven't been updated in the last year or so and some other niche open-source browser edge cases. You should note that where the AudioWorklet is not supported, audio processing takes place on the main Javascript thread, so suffers from sub-optimal performance, although it will still work.

High performance audio on the web is a fast moving space and we expect full AudioWorklet to come across 100% of browsers soon. Current estimates peg the feature coverage to be around 93% of global users, over at https://caniuse.com/?search=audioworklet

However, we're not quite in clear just yet! Firefox currently has a bug/limitation which prevents the use of ES6 module imports in AudioWorkletProcessor scripts. This is an issue because when you write AudioWorkletProcessor scripts with Superpowered, you'll need to import the Superpowered library so you can extend the SuperpoweredWebAudio.AudioWorkletProcessor class.

Specifically, we're referring to this import at the top of AudioWorkletProcessor

import '../Superpowered.js'; // <----- THIS LINE HERE

class SuperpoweredInputStageSamplerProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {

    onReady() {
        this.loaded = false;
        this.player = new this.Superpowered.AdvancedAudioPlayer(this.samplerate, 2, 2, 0, 0.501, 2, false);
        this.player.loopOnEOF = true;
        this.player.formantCorrection = 0;
        this.player.timeStretchingSound = 2;
    }
    ...

If you try to run the code above in Firefox you will receive an error

Error: Module resolve hook not set
// and/or
Uncaught (in promise) DOMException: The operation was aborted.

For more background on the issue, see here.

We expect this issue to be fixed in future versions of Firefox, but until then, we have two solutions available to you to solve the problems with Firefox, please read on.

Solution 1: Polyfill solution

While we're waiting for correct ES6 import support in Firefox, we've found a handy polyfill which can be included in your project to smooth out support across Firefox. We like this approach as it doesn't require as user-agent conditional coding within your Superpowered project.

1. Copy the polyfill_worklet_import.js polyfill here to an your hosts static files, eg the same location you server your css and assets.
2. Add <script type="module" src="[your public files]/polyfill_worklet_import.js"></script> as the first script that gets loaded in your application.

This is exactly how we've got Firefox support working on this documentation site.

The good thing about polyfill's are that they are only applied when required.

Solution 2: Bundling the ES6 module solution

You can also use a bundler like Rollup.js which can run at compile time rather than at runtime as suggested in Solution 1. This will remove the need for the polyfill, but it is at the cost of including the SP library in every AudioWorkletProcessor script file you write which increases the download size considerably as it currently also includes the Superpowered WebAssembly. Depending on your setup, this may not amount to an issue.

Here's an example rollup.config.js which lives in the root of your project. Please see here for more details on how Rollup works and how to set it up.

const fs = require('fs');

// Generate a config object for Rollup for a given filename
const createConfig = (filename) => ({
  input: `public/superpoweredProcessors/${filename}.js`, // The location of the input file
  output: [
    {
      file: `./public/superpoweredProcessors/bundled/${filename}.bundle.js`, // The location of the output file
      format: 'es',
    },
  ]
});

// Read all files in a folder to apply the transformation to
const files = fs.readdirSync('public/superpoweredProcessors/');
// Only bundle .js files, then remove the filename extension for createConfig
const configs = files.filter(fn => fn.includes('.js')).map((filename) => createConfig(filename.replace('.js', '')));

export default configs;

In the example, the folder public/superpoweredProcessors contains multiple processor scripts that we'd like to bundle to multiple output files with one command.

You can then run the transformation on the processor scripts with the following command in your project root via the terminal or add it to your projects package.json script commands.

rollup -c --bundleConfigAsCjs

When the bundling is complete, you refer to the bundled script when you call createAudioNodeAsync. More details are below.

Main script

Earlier we set up Superpowered and got it initialized in our main Javascript scope. Next we need to load our AudioWorkletProcessor toneProcessor.js into the WebAudio AudioContext to be scheduled.

We've created a handy wrapper function called createAudioNodeAsync for this in the SuperpoweredWebAudio helper class. We pass the location of the processor file (served statically), the name of the processor it gets registered with, and a reference to a function that will handle all incoming messages from the audio thread.

// ... webaudioManager defined above in previous steps

// The location of the processor from the browser to fetch.
const noiseProcessorUrl = 'http://localhost/static/processors/toneProcessor.js'

// Define a handler that will be called whenever this.sendMessageToMainScope is called from the AudioWorkletProcessor scope.
// Remember we called it with a ready event so expect to see it here.

const onMessageProcessorAudioScope = (message) => {
    if (message.event === 'ready') {
        console.log(message);
    }
}

// Now create the AudioWorkletNode, passing in the AudioWorkletProcessor url, its registered name (defined inside the processor) and the message callback.
const generatorProcessorNode = webaudioManager.createAudioNodeAsync(toneProcessorUrl, 'ToneProcessor', onMessageProcessorAudioScope);

// Connect the AudioWorkletNode to the WebAudio destination (main audio output by default, such as your speakers).
generatorProcessorNode.connect(webaudioManager.audioContext.destination);

See it all running

We created an ES6 sandbox with everything all hooked up for you. We'll be using this as the starting point for our guides later on. Fork the code sandbox and have fun!

Code examples

You can find example applications for common Javascript frameworks (React, Angular and Vue) having Superpowered loaded in and ready to roll here:

Superpowered example code

Now you've Superpowered up and running, so let's start doing some amazing things! Take a look at our guides for help with common tasks and setups you might require for your product:

Superpowered Guides

---