Superpowered Logo
Getting StartedGuidesAPI Reference
SDKLanguage
Getting StartedGuidesAPI Reference

Class: TimeStretching

Interactive example

Below is an example of the TimeStretching class being used with a realtime audio feed. Notice that we removed the rate property, because it's logically impossible to speed up a realtime audio source and slowing down requires ever growing memory. For a demo on how Superpowered time-stretching sound, please check the Advanced Audio Player interactive example.

Alternatively, for non-realtime processing examples, please see the Offline Time-stretching section.

Overview

TimeStretching is the process of altering the duration of a sound, tempo of a piece of audio without altering the pitch. This effect morphs the perceived passage of time in the audio signal without modifying its unique characteristics. However this class is also able to change the pitch of the audio independently from its tempo, so it's capable of high quality pitch shifting.

Tempo (time-stretching) can be controlled using the rate property. It's sonic behavior can be changed with sound : 0 represents slightly lower quality while saving CPU, 1 is best for DJ apps and “complete” music, and 2 is tuned for instrumental loops and single instruments. Please consider the types of device your app will be running on here.

Pitch-shifting can be controlled with pitchShiftCents, where cents are used to denote a unit of pitch based upon the equal tempered octave. The minimum and maximum values of this property are set between `-2400` (two octaves down) and `2400` (two octaves up) respectively. Pitch-shifting can sound unnatural for some vocals, and to mitigate that you can use the formantCorrection property with adjustable strength from 0.0 to 1.0.

One instance allocates around 220 kb memory.

Implementation example

Pitch-shifting real-time audio

The time doesn't change when using the pitch-shifting capability only, therefore the audio processing code is fairly simple as buffers are simply passing through.

class SuperpoweredTimeStretchingStageProcessor extends SuperpoweredWebAudio.AudioWorkletProcessor {


    onReady() {
        this.effect = new this.Superpowered.TimeStretching(
            this.samplerate, // The initial sample rate in Hz.
            1,   // The minimum value of rate. For example: if the rate will never go below 0.5, minimumRate = 0.5 will save significant computing power and memory. Minimum value of this: 0.01.
            2      // Valid values are: 0 (best to save CPU with slightly lower audio quality), 1 (best for DJ apps, modern and "complete" music), 2 (best for instrumental loops and single instruments).
        );
        this.effect.pitchShiftCents = 0;
        this.effect.formantCorrection = 1;
    }


    onDestruct() {
        this.effect.destruct();
    }


    onMessageFromMainScope(message) {
        if (typeof message.pitchShiftCents !== 'undefined') this.effect.pitchShiftCents = message.pitchShiftCents;
        if (typeof message.formatCorrection !== 'undefined') this.effect.formatCorrection = message.formatCorrection;
    }


    processAudio(inputBuffer, outputBuffer, buffersize, parameters) {
        // Ensure the samplerate is in sync on every audio processing callback
        this.effect.samplerate = this.samplerate;


        // Render the output buffers
        this.effect.addInput(
            inputBuffer.pointer, // Pointer to floating point numbers. 32-bit interleaved stereo input.
            buffersize  // Number of frames to process.
        );
        if (!this.effect.getOutput(outputBuffer.pointer, buffersize )) {
            for (let n = 0; n < buffersize * 2 ; n++) outputBuffer.array[n] = inputBuffer.array[n];
        }
    }
}
#include "SuperpoweredTimeStretching.h"


void initTimestretchingEffect() {
    timeStretching = new Superpowered::TimeStretching(
        48000,
        1,
        2
    );
    timeStretching->pitchShiftCents = 0;
    timeStretching->formantCorrection = 1;
}


void changePitch(int pitchShiftCents) {
    timeStretching->pitchShiftCents = pitchShiftCents;
}


void processAudio(float *interleavedInput, float *interleavedOutput, unsigned int numberOfFrames, unsigned int samplerate) {
    // Ensure the samplerate is in sync on every audio processing callback
    timeStretching->samplerate = samplerate;


    // Render the output buffers
    timeStretching->addInput(interleavedInput, numberOfFrame);
    bool success = timeStretching->getOutput(interleavedOutput, numberOfFrames);
    ...
}

Offline time-stretching

Using the time stretcher is more complex than using other effects, because it changes time and therefore buffering as well. It's fully integrated into the AdvancedAudioPlayer class, which is recommended for simple use. But if the time-stretcher has to be used alone, the logic could look like this:

// Pseudo code example


while (ts.getOutputLengthFrames() < numberOfFramesNeeded) {
    let timestretcherNeedsThisAmountOfFramesToProduceAudio = ts.getNumberOfInputFramesNeeded();
    if (timestretcherNeedsThisAmountOfFramesToProduceAudio can not be acquired) break;
    acquire timestretcherNeedsThisAmountOfFramesToProduceAudio into some Superpowered.Float32Buffer;
    ts.addInput(some Superpowered.Float32Buffer pointer, number_of_frames_acquired);
}


if (ts.getOutputLengthFrames() >= numberOfFramesNeeded) {
    // success, get output
    ts.getOutput(output buffer pointer, numberOfFramesNeeded);
} else {
    // could not produce enough audio
    // return with silence
}
// EXAMPLE: reading an audio file, applying a 4% rate increase time stretching and saving the result to WAV


int main(int argc, char *argv[]) {
    Superpowered::Initialize("ExampleLicenseKey-WillExpire-OnNextUpdate");
    // Open the input file.
    Superpowered::Decoder *decoder = new Superpowered::Decoder();
    int openReturn = decoder->open("test.m4a");
    if (openReturn != Superpowered::Decoder::OpenSuccess) {
        printf("\rOpen error %i: %s\n", openReturn, Superpowered::Decoder::statusCodeToString(openReturn));
        delete decoder;
        return 0;
    };


    // Create the output WAVE file.
    mkdir("./results", 0777);
    FILE *destinationFile = Superpowered::createWAV("./results/offline2.wav", decoder->getSamplerate(), 2);
    if (!destinationFile) {
        printf("\rFile creation error.\n");
        delete decoder;
        return 0;
    };


    // Create the time stretcher.
    Superpowered::TimeStretching *timeStretch = new Superpowered::TimeStretching(decoder->getSamplerate());
    timeStretch->rate = 1.04f; // 4% faster


    // Create a buffer to store 16-bit integer audio up to 1 seconds, which is a safe limit.
    short int *intBuffer = (short int *)malloc(decoder->getSamplerate() * 2 * sizeof(short int) + 16384);
    // Create a buffer to store 32-bit floating point audio up to 1 seconds, which is a safe limit.
    float *floatBuffer = (float *)malloc(decoder->getSamplerate() * 2 * sizeof(float));


    // Processing.
    while (true) {
        int framesDecoded = decoder->decodeAudio(intBuffer, decoder->getFramesPerChunk());
        if (framesDecoded < 1) break;


        // Submit the decoded audio to the time stretcher.
        Superpowered::ShortIntToFloat(intBuffer, floatBuffer, framesDecoded);
        timeStretch->addInput(floatBuffer, framesDecoded);


        // The time stretcher may have 0 or more audio at this point. Write to disk if it has some.
        unsigned int outputFramesAvailable = timeStretch->getOutputLengthFrames();
        if ((outputFramesAvailable > 0) && timeStretch->getOutput(floatBuffer, outputFramesAvailable)) {
            Superpowered::FloatToShortInt(floatBuffer, intBuffer, outputFramesAvailable);
            Superpowered::writeWAV(destinationFile, intBuffer, outputFramesAvailable * 4);
        }
    };


    // Close the file and clean up.
    Superpowered::closeWAV(destinationFile);
    delete decoder;
    delete timeStretch;
    free(intBuffer);
    free(floatBuffer);


    printf("\rReady.\n");
    return 0;
}

Properties

formantCorrection

PROPERTY
Amount of formant correction, between 0 (none) and 1 (full).
TypeMinMaxDefault
Number
010

formantCorrection

PROPERTY
Amount of formant correction, between 0 (none) and 1 (full).
TypeMinMaxDefault
float
010

outputList

PROPERTY
The AudiopointerList storing the audio output. To be used with the advancedProcess() method. Read only.
TypeMinMaxDefault
Superpowered::AudiopointerList

pitchShiftCents

PROPERTY
Pitch shift cents, limited from -2400 (two octaves down) to 2400 (two octaves up). Examples: 0 (no pitch shift), -100 (one note down), 300 (3 notes up). When the value if a multiply of 100 and is >= -1200 and <= 1200, changing the pitch shift needs only a few CPU clock cycles. Any change in pitchShiftCents involves significant momentary CPU load otherwise.
TypeMinMaxDefault
Number
-240024000

pitchShiftCents

PROPERTY
Pitch shift cents, limited from -2400 (two octaves down) to 2400 (two octaves up). Examples: 0 (no pitch shift), -100 (one note down), 300 (3 notes up). When the value if a multiply of 100 and is >= -1200 and <= 1200, changing the pitch shift needs only a few CPU clock cycles. Any change in pitchShiftCents involves significant momentary CPU load otherwise.
TypeMinMaxDefault
int
-240024000

preciseTurningOn

PROPERTY
Maintain precise timing when the time-stretcher turns on. Useful for all use-cases except when the audio is heavily manipulated with some resampler (scratching).
TypeMinMaxDefault
Boolean
true

preciseTurningOn

PROPERTY
Maintain precise timing when the time-stretcher turns on. Useful for all use-cases except when the audio is heavily manipulated with some resampler (scratching).
TypeMinMaxDefault
bool
true

rate

PROPERTY
Time stretching rate (tempo). 1 means no time stretching. Values above 2 or below 0.5 are not recommended on mobile devices with low latency audio due high CPU load and risk of audio dropouts.
TypeMinMaxDefault
Number
minimumRate in Constructor41

rate

PROPERTY
Time stretching rate (tempo). 1 means no time stretching. Values above 2 or below 0.5 are not recommended on mobile devices with low latency audio due high CPU load and risk of audio dropouts.
TypeMinMaxDefault
float
minimumRate in Constructor41

samplerate

PROPERTY
Sample rate in Hz. High quality pitch shifting requires 44100 Hz or more, the sound is "echoing" on lower sample rates.
TypeMinMaxDefault
Number

samplerate

PROPERTY
Sample rate in Hz. High quality pitch shifting requires 44100 Hz or more, the sound is "echoing" on lower sample rates.
TypeMinMaxDefault
float

sound

PROPERTY
Default: 1.
TypeMinMaxDefault
0: save CPU with slightly lower audio quality
1: DJ apps, modern and "complete" music
2: instrumental loops and single instruments
021

sound

PROPERTY
Default: 1.
TypeMinMaxDefault
0: save CPU with slightly lower audio quality
1: DJ apps, modern and "complete" music
2: instrumental loops and single instruments
021

Methods

  • constructor

    METHOD
    Creates a TimeStretching instance.
    Parameters
    NameTypeDescription
    samplerateNumberThe initial sample rate in Hz.
    minimumRateNumberThe minimum value of rate. For example: if the rate will never go below 0.5, minimumRate = 0.5 will save significant computing power and memory. Min: 0.01, max 0.75.
    Returns
    TypeDescription
    Superpowered.TimeStretchingThe constructed instance.

  • constructor

    METHOD
    Creates a TimeStretching instance.
    Parameters
    NameTypeDefaultDescription
    samplerateunsigned intThe initial sample rate in Hz.
    minimumRatefloat0.01The minimum value of rate. For example: if the rate will never go below 0.5, minimumRate = 0.5 will save significant computing power and memory. Min: 0.01, max 0.75.
    Returns
    TypeDescription
    Superpowered::TimeStretchingThe constructed instance.

  • addInput

    METHOD
    Add audio input.
    Parameters
    NameTypeDescription
    inputNumber (pointer in Linear Memory)32-bit interleaved stereo input.
    numberOfFramesNumberNumber of frames in input.
    Returns
    None

  • addInput

    METHOD
    Add audio input.
    Parameters
    NameTypeDefaultDescription
    inputfloat *32-bit interleaved stereo input.
    numberOfFramesunsigned intNumber of frames in input.
    Returns
    None

  • advancedProcess

    METHOD

    Processes audio. For advanced uses to:

    1. Save memory bandwidth (uses no memory copy) for maximum performance.
    2. Support up to 8 audio channels.
    3. Provide sub-sample precise positioning.

    It's never blocking for real-time usage. You can change all properties on any thread, concurrently with process(). Use it in the same thread with the other real-time methods of this class. Do not use this method with addInput() or getOutput().

    Parameters
    NameTypeDefaultDescription
    inputAudiopointerlistElement *The input buffer. It will take ownership on the input.
    Returns
    None

  • getNumberOfInputFramesNeeded

    METHOD
    Returns with how many frames of input should be provided to the time stretcher to produce some output. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    None
    Returns
    TypeDescription
    NumberThe result can be 0 if rate is 1 and pitch shift is 0, because in that case the time stretcher is fully "transparent" and any number of input frames will produce some output.

  • getNumberOfInputFramesNeeded

    METHOD
    Returns with how many frames of input should be provided to the time stretcher to produce some output. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    None
    Returns
    TypeDescription
    unsigned intThe result can be 0 if rate is 1 and pitch shift is 0, because in that case the time stretcher is fully "transparent" and any number of input frames will produce some output.

  • getOutput

    METHOD
    Gets the audio output into a buffer. It's never blocking for real-time usage. You can change all properties on any thread, concurrently with process(). Use it in the same thread with the other real-time methods of this class.
    Parameters
    NameTypeDescription
    outputNumber (pointer in Linear Memory)32-bit interleaved stereo output.
    numberOfFramesNumberNumber of frames to return with.
    Returns
    TypeDescription
    BooleanTrue if it has enough output frames stored and output is successfully written, false otherwise.

  • getOutput

    METHOD
    Gets the audio output into a buffer. It's never blocking for real-time usage. You can change all properties on any thread, concurrently with process(). Use it in the same thread with the other real-time methods of this class.
    Parameters
    NameTypeDefaultDescription
    outputfloat *32-bit interleaved stereo output.
    numberOfFramesunsigned intNumber of frames to return with.
    Returns
    TypeDescription
    boolTrue if it has enough output frames stored and output is successfully written, false otherwise.

  • getOutputLengthFrames

    METHOD
    Returns with how many frames of output is available. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    None
    Returns
    TypeDescription
    NumberFrames of output available.

  • getOutputLengthFrames

    METHOD
    Returns with how many frames of output is available. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    None
    Returns
    TypeDescription
    unsigned intFrames of output available.

  • removeFramesFromInputBuffersEnd

    METHOD
    Removes audio from the end of the input buffer. Can be useful for some looping use-cases for example. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    NameTypeDescription
    numberOfFramesNumberThe number of frames to remove.
    Returns
    None

  • removeFramesFromInputBuffersEnd

    METHOD
    Removes audio from the end of the input buffer. Can be useful for some looping use-cases for example. It's never blocking for real-time usage. Use it in the same thread with the other real-time methods of this class.
    Parameters
    NameTypeDefaultDescription
    numberOfFramesunsigned intThe number of frames to remove.
    Returns
    None

  • reset

    METHOD
    Reset all internals, sets the instance as good as new. Don't call this concurrently with process() and in a real-time thread.
    Parameters
    None
    Returns
    None

  • reset

    METHOD
    Reset all internals, sets the instance as good as new. Don't call this concurrently with process() and in a real-time thread.
    Parameters
    None
    Returns
    None

  • setStereoPairs

    METHOD
    This class can handle one stereo audio channel pair by default (left + right). Maybe you need more if you load some music with 4 channels, then less if you load a regular stereo track. Don't call this concurrently with process() and in a real-time thread.
    Parameters
    NameTypeDescription
    numStereoPairsNumberThe number of stereo audio channel pairs. Valid values: one (stereo) to four (8 channels).
    dontFreeBooleanIf true, this function will not free up any memory if numStereoPairs was bigger before, so no reallocation happens if numStereoPairs needs to be increased later.
    Returns
    None

  • setStereoPairs

    METHOD
    This class can handle one stereo audio channel pair by default (left + right). Maybe you need more if you load some music with 4 channels, then less if you load a regular stereo track. Don't call this concurrently with process() and in a real-time thread.
    Parameters
    NameTypeDefaultDescription
    numStereoPairsunsigned intThe number of stereo audio channel pairs. Valid values: one (stereo) to four (8 channels).
    dontFreeboolIf true, this function will not free up any memory if numStereoPairs was bigger before, so no reallocation happens if numStereoPairs needs to be increased later.
    Returns
    None
v1.0.32