--- title: "AudioSpectralTransformation" language: "en" type: "Symbol" summary: "AudioSpectralTransformation[f, audio] returns a modified version of audio by applying a time-frequency transformation f to its short-time Fourier transform. AudioSpectralTransformation[f, video] transforms the first audio track in video." keywords: - spectral filtering - spectral transform - audio effects - audio transform - audio spectral transform - time frequency transform - short-time Fourier transform - STFT - audio STFT - audio modification canonical_url: "https://reference.wolfram.com/language/ref/AudioSpectralTransformation.html" source: "Wolfram Language Documentation" related_guides: - title: "Audio Editing" link: "https://reference.wolfram.com/language/guide/AudioEditing.en.md" related_functions: - title: "SpectrogramArray" link: "https://reference.wolfram.com/language/ref/SpectrogramArray.en.md" - title: "AudioSpectralMap" link: "https://reference.wolfram.com/language/ref/AudioSpectralMap.en.md" - title: "AudioTrackApply" link: "https://reference.wolfram.com/language/ref/AudioTrackApply.en.md" - title: "TranslationTransform" link: "https://reference.wolfram.com/language/ref/TranslationTransform.en.md" - title: "ImageTransformation" link: "https://reference.wolfram.com/language/ref/ImageTransformation.en.md" --- # AudioSpectralTransformation AudioSpectralTransformation[f, audio] returns a modified version of audio by applying a time-frequency transformation f to its short-time Fourier transform. AudioSpectralTransformation[f, video] transforms the first audio track in video. ## Details and Options * Applying an arbitrary time-frequency transformation to the short-time Fourier transform of audio signals can create interesting audio modifications and effects for creative use. * ``AudioSpectralTransformation`` computes the short-time Fourier transform of audio, maps every value at position ``f[{time, freq}]`` to ``{time, freq}``, and computes the inverse using the overlap-add method. * The following options can be given: | | | | | --------------------- | --------- | --------------------------------------- | | DataRange | Automatic | ranges to assume for time and frequency | | Padding | 0 | padding scheme to use | | PartitionGranularity | Automatic | audio partitioning specification | | Resampling | Automatic | resampling method | * By default, ``DataRange -> {{0, dur}, {0, sr / 2}}`` is used, where ``dur`` and ``sr`` are duration and sample rate of ``audio``. --- ## Examples (16) ### Basic Examples (1) Modify the spectrogram of the signal: ```wl In[1]:= a = \!\(\*AudioBox["![Embedded Audio Player](audio://content-cnp9u)"]\); In[2]:= Spectrogram[a] Out[2]= [image] In[3]:= AudioSpectralTransformation[# / 2&, a] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-cni4k)"]\) In[4]:= Spectrogram[%] Out[4]= [image] ``` ### Scope (4) Transform only along the time axis: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time * .5, frequency}; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-vr4cg)"]\) ``` --- Transform only along the frequency axis: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time, frequency * .5} AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-erg1y)"]\) ``` --- Transform along both time and frequency axes: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"] Out[1]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-yzpmr)"]\) In[2]:= dur = QuantityMagnitude[Duration[a]];f[{time_, frequency_}] := {Tanh[time / dur]dur + frequency / 10000, frequency - time ^ 3} In[3]:= AudioSpectralTransformation[f, a] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-8ki1c)"]\) In[4]:= Spectrogram[%] Out[4]= [image] ``` --- Process the audio track of a video: ```wl In[1]:= f[{time_, frequency_}] := {time * .5, frequency}; AudioSpectralTransformation[f, \!\(\*VideoBox["![Video Player: ExampleData/fish.mp4](video://content-2sfji)"]\)] Out[1]= \!\(\*VideoBox[...]\) ``` ### Options (2) #### DataRange (1) By default, the data range is ``{{0, duration}, {0, samplerate / 2}}`` : ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= AudioSpectralTransformation[# - {1, 0}&, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-0eym0)"]\) In[3]:= Spectrogram[%] Out[3]= [image] ``` Specify a different data range for the function: ```wl In[4]:= AudioSpectralTransformation[# - {.5, 0}&, a, DataRange -> {{0, 1}, {0, 11025}}] Out[4]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-2008e)"]\) In[5]:= Spectrogram[%] Out[5]= [image] ``` #### PartitionGranularity (1) Use the ``PartitionGranularity`` option to change the quality of the result: ```wl In[1]:= a = ExampleData[{"Sound", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[x_] := x / 2; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-uytl8)"]\) In[3]:= AudioSpectralTransformation[f, a, PartitionGranularity -> {.05, .008, BlackmanWindow}] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-zta1r)"]\) ``` ### Applications (5) Compress the time and do a time-dependent frequency stretching: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"] Out[1]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-yzpmr)"]\) In[2]:= f[{time_, frequency_}] := {time * 1.2, frequency + time * frequency ^ .5}; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-i09j3)"]\) In[3]:= Spectrogram[%, 1024, 2048, ImageSize -> Medium] Out[3]= [image] ``` --- Apply a curvilinear distortion to the spectrogram: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[pt_] := With[{s = {.5, .1}}, Module[{r, a, an}, r = Norm[pt - s];a = ArcTan@@(pt - s);an = a + 2r; s + r{Cos[an], Sin[an]}]] AudioSpectralTransformation[f, a, DataRange -> {{0, 1}, {0, 1}}] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-cyl8z)"]\) In[3]:= Spectrogram[%, 1024, 2048, ImageSize -> Medium] Out[3]= [image] ``` --- Quantize the spectrogram both in frequency and time: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {.05Floor[time / .05], 100Floor[frequency / 100] + 1}; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-j8eqj)"]\) In[3]:= Spectrogram[%, 1024, 2048, ImageSize -> Medium] Out[3]= [image] ``` --- Create a time-dependent frequency shift: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time, frequency + 400Sin[2 Pi time]}; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-kycsf)"]\) In[3]:= Spectrogram[%, 1024, 2048, ImageSize -> Medium] Out[3]= [image] ``` --- Apply a fisheye transformation to the spectrogram: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[pt_] := With[{s = {2, 10000}}, Module[{r, a}, r = Norm[pt - s]^2 / Norm[s];a = ArcTan@@(pt - s); s + r{Cos[a], Sin[a]}]]; AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-avlfc)"]\) In[3]:= Spectrogram[%, 1024, 2048, ImageSize -> Medium] Out[3]= [image] ``` ### Properties & Relations (4) Performing an identity operation does not return the input sample values: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= AudioSpectralTransformation[Identity, a] == a Out[2]= False ``` Even in the case of an identity, a short-time Fourier transform followed by an overlap add operation is performed. The result only has minor discrepancies with the input: ```wl In[3]:= {Mean[#], StandardDeviation[#]}&[AudioSpectralTransformation[Identity, a] - a] Out[3]= {{0.00164342}, {0.0455284}} ``` --- Create a crude pitch-shifting effect: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time, frequency * .5} AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-erg1y)"]\) ``` Compare to ``AudioPitchShift`` : ```wl In[3]:= AudioPitchShift[a, 2] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-5825e)"]\) ``` --- Create a crude time-stretching effect: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time * .5, frequency} AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-vr4cg)"]\) ``` Compare to ``AudioTimeStretch`` : ```wl In[3]:= AudioTimeStretch[a, 2] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-ecnld)"]\) ``` --- Create a crude frequency-shifting effect: ```wl In[1]:= a = ExampleData[{"Audio", "Apollo11SmallStep"}, "Audio"]; In[2]:= f[{time_, frequency_}] := {time, 200 + frequency} AudioSpectralTransformation[f, a] Out[2]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-1znd6)"]\) ``` Compare to ``AudioFrequencyShift`` : ```wl In[3]:= AudioFrequencyShift[a, 200] Out[3]= \!\(\*AudioBox["![Embedded Audio Player](audio://content-bir4y)"]\) ``` ## See Also * [`SpectrogramArray`](https://reference.wolfram.com/language/ref/SpectrogramArray.en.md) * [`AudioSpectralMap`](https://reference.wolfram.com/language/ref/AudioSpectralMap.en.md) * [`AudioTrackApply`](https://reference.wolfram.com/language/ref/AudioTrackApply.en.md) * [`TranslationTransform`](https://reference.wolfram.com/language/ref/TranslationTransform.en.md) * [`ImageTransformation`](https://reference.wolfram.com/language/ref/ImageTransformation.en.md) ## Related Guides * [Audio Editing](https://reference.wolfram.com/language/guide/AudioEditing.en.md) ## History * [Introduced in 2017 (11.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn111.en.md) \| [Updated in 2024 (14.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn141.en.md)