Feature/peaks

doc/BufSineFeature.rst

@@ -0,0 +1,96 @@
+:digest: Buffer-Based Sinusoidal Peak Tracking
+:species: buffer-proc
+:sc-categories: Libraries>FluidDecomposition, UGens>Buffer
+:sc-related: Guides/FluidCorpusManipulation, Classes/SinOsc
+:see-also: SineFeature, BufSines
+:description: Interpolated Sinusoidal Peak Tracking on the Spectrum of Audio Stored in a Buffer.
+:discussion: 
+  This process is tracking peaks in the spectrum of audio stored in a buffer, then estimating an interpolated frequency and amplitude of that peak in relation to its spectral context. It is the first part of the process used by :fluid-obj:`BufSines`. 
+
+  The process will return two buffers containing time series that describes the interpolated frequencies and magnitudes changing over time in the source buffer.
+
+:process: This is the method that calls for the slicing to be calculated on a given source buffer.
+:output: Nothing, as the various destination buffers are declared in the function call.
+
+:control source:
+
+  The |buffer| to use as the source material. The channels of multichannel buffers will be processed sequentially.
+
+:control startFrame:
+
+  The starting point for analysis in the source (in samples).
+
+:control numFrames:
+
+  The duration (in samples) to analyse.
+
+:control startChan:
+
+  For multichannel sources, the starting channel to analyse.
+
+:control numChans:
+
+  For multichannel sources, the number of channels to analyse.
+
+:control frequency:
+
+  The buffer where the interpolated frequency of the peaks will be written.
+
+:control magnitude:
+
+  The buffer where the interpolated magnitude of the peaks will be written.
+
+:control numPeaks:
+
+  The number of peaks to search report back. It is capped at (fftSize / 2) + 1.
+
+:control detectionThreshold:
+
+  The threshold in dB above which a magnitude peak is considered to be a sinusoidal component.
+
+:control order:
+
+  How the reported peaks are to be ordered. By default (0), it is by frequencies (lowest first), and the alternative (1) is by magnitude (loudest first).
+
+:control freqUnit:
+
+  The units and scale used to report the frequency of the peaks. By default (0), it is in Hz (linear), and the alternative (1) is in MIDI (logarithmic).
+
+:control magUnit:
+
+  The units and scale used to report the magnitude of the peaks. By default (0), it is in amp (linear), and the alternative (1) is in dB (logarithmic).
+
+:control windowSize:
+
+  The window size. As sinusoidal estimation relies on spectral frames, we need to decide what precision we give it spectrally and temporally. For more information visit https://learn.flucoma.org/learn/fourier-transform/
+
+:control hopSize:
+
+  The window hop size. As sinusoidal estimation relies on spectral frames, we need to move the window forward. It can be any size, but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2).
+
+:control fftSize:
+
+  The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will default to windowSize. The -1 default value will default to the highest of windowSize and (bandwidth - 1) * 2.
+
+:control padding:
+
+   Controls the zero-padding added to either end of the source buffer or segment. Padding ensures all values are analysed. Possible values are:
+
+   :enum:
+
+      :0:
+         No padding - The first analysis window starts at time 0, and the samples at either end will be tapered by the STFT windowing function.
+
+      :1: 
+         Half the window size - The first sample is centred in the analysis window ensuring that the start and end of the segment are accounted for in the analysis.
+
+      :2: 
+         Window size minus the hop size - Mode 2 can be useful when the overlap factor (window size / hop size) is greater than 2, to ensure that the input samples at either end of the segment are covered by the same number of analysis frames as the rest of the analysed material.
+
+:control maxFFTSize:
+
+  How large can the FFT be, by allocating memory at instantiation time. This cannot be modulated.
+
+:control maxNumPeaks:
+
+  Up to how many peaks can be reported, by allocating memory at instantiation time. This cannot be modulated.

doc/SineFeature.rst

@@ -0,0 +1,56 @@
+:digest: Sinusoidal Peak Tracking
+:species: descriptor
+:sc-categories: Libraries>FluidDecomposition
+:sc-related: Guides/FluidCorpusManipulation, Classes/SinOsc
+:see-also: BufSineFeature, Sines
+:description: Interpolated Sinusoidal Peak Tracking on the Spectrum.
+:discussion: 
+   This process is tracking peaks in the spectrum, then estimating an interpolated frequency and amplitude of that peak in relation to its spectral context. It is the first part of the process used by :fluid-obj:`Sines`.
+
+:process: The audio rate version of the object.
+:output: An array of two control streams: [0] is the interpolated frequency of the peaks extracted in Hz or MIDI, [1] is their respective magnitudes in amp or dB. The latency between the input and the output is windowSize samples.
+
+
+:control in:
+
+   The input to be processed
+
+:control numPeaks:
+
+      The number of peaks to search report back. It is capped at (fftSize / 2) + 1.
+
+:control detectionThreshold:
+
+   The threshold in dB above which a magnitude peak is considered to be a sinusoidal component.
+
+:control order:
+
+  How the reported peaks are to be ordered. By default (0), it is by frequencies (lowest first), and the alternative (1) is by magnitude (loudest first).
+
+:control freqUnit:
+
+  The units and scale used to report the frequency of the peaks. By default (0), it is in Hz (linear), and the alternative (1) is in MIDI (logarithmic).
+
+:control magUnit:
+
+  The units and scale used to report the magnitude of the peaks. By default (0), it is in amp (linear), and the alternative (1) is in dB (logarithmic).
+
+:control windowSize:
+
+   The window size. As sinusoidal estimation relies on spectral frames, we need to decide what precision we give it spectrally and temporally. For more information visit https://learn.flucoma.org/learn/fourier-transform/
+
+:control hopSize:
+
+   The window hop size. As sinusoidal estimation relies on spectral frames, we need to move the window forward. It can be any size, but low overlap will create audible artefacts. The -1 default value will default to half of windowSize (overlap of 2).
+
+:control fftSize:
+
+   The inner FFT/IFFT size. It should be at least 4 samples long, at least the size of the window, and a power of 2. Making it larger allows an oversampling of the spectral precision. The -1 default value will default to windowSize. The -1 default value will default to the highest of windowSize and (bandwidth - 1) * 2.
+
+:control maxFFTSize:
+
+   How large can the FFT be, by allocating memory at instantiation time. This cannot be modulated.
+
+:control maxNumPeaks:
+
+  Up to how many peaks can be reported, by allocating memory at instantiation time. This cannot be modulated.

example-code/sc/BufSineFeature.scd

@@ -0,0 +1,59 @@
+strong::Plot the BufSineFeature curves one over the other::
+code::
+~oboe = Buffer.read(s,FluidFilesPath("Harker-DS-TenOboeMultiphonics-M.wav"),numFrames: 311000);
+
+(
+~freq = Buffer(s);
+~mags = Buffer(s);
+FluidBufSineFeature.processBlocking(s,~oboe, frequency: ~freq, magnitude: ~mags, numPeaks: 5);
+w = Window("FluidWaveform Test",Rect(0,0,1000,500));
+w.view.layout = VLayout(
+	FluidWaveform(~oboe,featuresBuffer: ~freq,standalone: false),
+	FluidWaveform(~oboe,featuresBuffer: ~mags,standalone: false));
+w.front;
+)
+::
+
+strong::A few didactic examples::
+code::
+
+//mono source of 3 sines
+b = {SinOsc.ar([440,789,535],mul: [0.01,0.03,0.02]).sum}.asBuffer(1)
+
+//listen
+b.play
+
+//make destination buffers
+~freq = Buffer(s); ~mags = Buffer(s);
+
+//process
+FluidBufSineFeature.process(s,b,frequency: ~freq, magnitude: ~mags, numPeaks: 4, action: {\done.postln})
+
+// retrieve the first 2 frames of 4 peaks
+~freq.getn(0, 8, {|x|x.postln})
+~mags.getn(0, 8, {|x|x.postln})
+
+// there are only 2 peaks... this is because the distance between 2 peaks has to be clearly segregated in the FFT world. At the default 1024 and the usual SC SR of 44100, this is 43Hz per bin, so 440 and 535 are too near each other... if we reprocess with a higher frame size, we get the right values
+
+FluidBufSineFeature.process(s,b,frequency: ~freq, magnitude: ~mags, numPeaks: 4, windowSize: 2048, action: {\done.postln})
+
+// first 2 frames of 4 peaks
+~freq.getn(0, 8, {|x|x.postln})
+~mags.getn(0, 8, {|x|x.postln})
+
+//here is another source for fun, stereo this time, and triangle waves
+b = {LFTri.ar([300, 500],mul: [-40, -45].dbamp)}.asBuffer(1)
+
+b.play
+b.query
+
+// asking for 2 peaks - first and third harmonic of each should pop out
+FluidBufSineFeature.process(s, b, frequency: ~freq, magnitude: ~mags, numPeaks: 2, magUnit: 1, action: {\done.postln})
+
+// retrieving - the stereo values are interleaved, 2 for left 2 for right.
+~freq.getn(0, 8, {|x|x.postln})
+~mags.getn(0, 8, {|x|x.postln})
+
+::
+
+

example-code/sc/SineFeature.scd

@@ -0,0 +1,34 @@
+
+CODE::
+
+// a didactic example: a cluster of sinusoids, sorted by magnitudes
+{var source = SinOsc.ar([440,789],mul: [0.05,0.1]).sum; FluidSineFeature.kr(source,numPeaks: 3, order: 1).poll; source.dup}.play
+
+// or in MIDI and dB
+{var source = SinOsc.ar([69,79].midicps,mul: [-40,-35].dbamp).sum; FluidSineFeature.kr(source,numPeaks: 3, order: 1, freqUnit: 1, magUnit: 1).poll; source.dup}.play
+
+// a more exciting example: resynthesizing audio input
+(
+var buf = Buffer.read(s,FluidFilesPath("Harker-DS-TenOboeMultiphonics-M.wav"));
+
+x = {
+	arg nbPeaks = 10, t_hold = 1;
+	var source = PlayBuf.ar(1, buf, loop: 1);
+	var analysis = Latch.kr(FluidSineFeature.kr(source,numPeaks: nbPeaks, maxNumPeaks: 50),t_hold);
+	var resynth = SinOsc.ar(analysis[Array.iota(50)], mul: analysis[Array.iota(50) + 50]).sum;
+	[source, resynth];
+}.play
+)
+
+// play with the number of peaks to track
+x.set(\nbPeaks, 5)
+x.set(\nbPeaks, 1)
+x.set(\nbPeaks, 50)
+
+// trigger the holder to hear the reconstruction on the right hand side
+x.set(\t_hold, 1)
+
+// or make it automatic
+r = Routine{x.set(\t_hold, 1);0.01.wait;}.loop.play
+r.stop
+::