you could model amplifiers, distortion effects, phasers and flangers, according to CoreAudioML project, which makes this possible
https://github.com/Alec-Wright/CoreAudioML
You can model most useful nonlinear functions with neural networks, this is unsurprising. You can also use Volterra series. You can even estimate/measure the Volterra kernel then train a NN to model it instead of dealing with the computational complexity of generalized convolution for nonlinear dynamic systems.
The hard part is that there are some fundamental limitations to deal with. The biggest is aliasing - distortion effects in particular deal with enormous amounts of distortion (> 100% THD) which creates spectrums far outside the range of hearing. Digital audio systems need to have high orders of oversampling to prevent audible aliasing (8-16x is not unheard of!).
After aliasing is memory. It's too early in the morning for me to do math but I'm almost certain you can't model a looper with a causal NN that has less internal state memory than the length of your loop. Doing so is dumb anyway, since loopers are pretty trivial and their biggest cost is memory. Same goes for digital delay and modulation effects, the algorithms are not expensive.
Now I wonder if a NN would be able to learn a nonlinear effect without aliasing, even if run at the original sample rate. Oversampling and filtering are, after all, things that could become part of the model too. Perhaps it can learn to approximate them with less CPU cost than doing it for real.
Oversampling requires producing more output information than input information. It would be incredible for a NN to realize a system that could do this without requiring more memory and CPU cycles than a good oversampling algorithm, which can be derived analytically with various definitions of "optimal."
A 3rd or 4th order polynomial interpolator is pretty darn good and doesn't need a NN to find the coefficients.
I'm not talking about using a NN for oversampling; I'm talking about whether an NN could learn to reduce aliasing when implementing nonlinear functions without any external oversampling.
Oversampled DSP algorithms work by oversampling, performing the nonlinear processing, then filtering to remove the higher harmonics, and finally downsampling again. We do it this way because it's convenient and easy to understand and based on proven mathematics. But nothing says these steps have to be distinct.
An oversampled DSP algorithm looks like a regular DSP algorithm from the outside, perhaps with some more state and latency required for it to perform the internal oversampling. You can also imolement such an oversampled algorithm entirely at the original sample rate clock; it just means the processing needs to internally process several samples per outer loop sample.
Since neural networks excel at modeling "black boxes" as one amorphous blob that we don't understand, I wonder if a NN could learn to model such an internally oversampled algorithm fairly accurately, and what the computational complexity would be.
Since you can model the oversampling/filtering/etc steps as linear convolutions with wider internal state at the original sample rate, I'm almost certain this will work with the right NN topology. It's obvious an NN can implement oversampling.
And so my question is: could treating the combined oversampled processing as one step, and training a NN on that, potentially result in a more efficient implementation than doing it naively? Especially for heavy distortion that needs high oversampling ratios.
