A few developers (including myself) have thought about a platform for prototyping modular DSP algorithms.
There are many scripting languages for audio DSP such as Faust, SuperCollider, Csound, ChucK, or Vult that would be ideal for interpreting and running in a Rack module.
But also many generic fast languages like LuaJIT and Python with Scipy, which is only fast when processing blocks of audio but basically has the power of MATLAB.
Frank Buss Formula exists, but I don’t consider it to be a prototyping platform for implementing advanced DSP algorithms and analog modeling, rather a tool for users to implement functions that aren’t available as modules.
Does this idea interest any plugin developers? The UI could simply load a script and watch it for changes. It could have some knobs, switches, inputs, outputs, and RGB LEDs that would be exposed as variables in the scripting language.
Edit: Plan
We will wait on Wes for the design. It will tentatively have 8 inputs, 8 outputs, 8 knobs, and 8 buttons with RGB LEDs. We plan to add the following backends:
To open a discussion, if I or someone else worked on this, what scripting language should it support, and why? I have no personal preference, just want it to be the best choice for the average script writer. It should be easy for beginner programmers to learn, yet powerful enough to do convolutions, analog modeling, state machines, and other nontrivial algorithms. It would need to have an embeddable C/C++ implementation.
I find python + scipy + jupyter notebooks a great way to prototype DSP. You can hear the sound and everything. If you use scipy correctly almost no time spent in python. But loading a full python engine into rack to run a fragment of code would be environmentally tricky. Like how are you going to interact with your virtualenv and stuff. So I always end up just re-coding in C++.
Perhaps another way to think about this, though, is (at least for prototyping) is to drop the embeddable C++ constraint. Seriously. Like have a module which sends a collection of inputs and pulls a collection of outputs on a blob of shared memory or jack bus (maybe), etc… then you can write python, haskell, javascript and whatever other adapters you want in the local language. You have a compact API to support in each scripting language and you send a message at a well-defined location. If the kernel isn’t running you just get no output back. Basically an audio-remote kernel plugin. More work but lets people explore more scripting languages. Just a thought.
That’s what I do most of the time. But I don’t think a non-embedded API is the way to go for this. Professionals will use what they want or just skip prototyping altogether.
But adapters and external solutions aren’t reliable, and this will kill the appeal. A self-contained language can be shared to users who have no idea about programming, and be used with 100% reliability by just loading the script. The module could even save the source code to its state so patches could be shared.
Gotcha. So your really want something where it is not just prototype but also distributable to non-dev users. That means you want a language which is fast small and embeddable. There aren’t a lot of great choices there, but people seem to come to lua quite often with that combination. Python is a better language but a beast to distribute and configure onto everyones machine right?
A quick review of options makes it appear that Faust might be the clear choice. You get filters, reverbs, physical models, and a lot of general math routines you’d expect in a prototyping platform. And it’s well-known. The problem is that interpreting is difficult. I’d need to embed LLVM to JIT.
Totally agree that the benefit here is from embedded scripts.
Languages:
I would vote for staying away from the general-purpose ones (at least initially) because I’m not sure that DSP programming is thaaaaaat much easier in Python or Lua than C++. Sure they’re friendlier languages in general, but imperative DSP code would probably look pretty similar in all three, right? (I’d defer to @baconpaul on this–I’m a heavy Python user but have never done DSP in it).
+1: Faust, Vult, and maybe PureData (note https://github.com/enzienaudio/hvcc ) offer something distinctly different, with major library support (at least in Faust).
0: Can’t speak to SuperCollider or ChucK (SC does look cool, though).
-1: I’d suggest not prioritizing Csound; I used to use it quite a bit but I think it may be showing its age (and there’s already a Csound plugin, right?)
Random thought:
I have and use a Rebel Tech Wizard (part of their post-Owl series) and it’s great, but I think their community does suffer from a certain amount of fragmentation (https://www.rebeltech.org/patch-library/patches/latest should give some impression) due to how many languages the platform supports. On balance a prototyping Rack plugin is a nice idea, but there is something elegant about having one plugin language, one plugin manager, one plugin format…
It is a very cool language and the toolchain is extremely well worked out. Heads up about one thing (although you’re probably already seen it). You mention that you wanted a language
Analog modeling and nontrivial stuff for sure, but as the FAQ states:
[Faust] doesn’t allow for the efficient implementation of algorithms requiring multi-rates such as the FFT, convolution, etc. While there are tricks to go around this issue, we’re fully aware that it is a big one and we’re working as hard as possible on it.
This may not be 100% current. There are definitely audio-rate (e.g. sliding) FFT routines, but AFAIK it’s still not multi-rate at least as of Feb 2018, though there’s some experimental stuff in the codebase and I think they really are actively working on it (interesting technical paper from 2016).
FAQ also notes:
Faust’s conciseness can sometimes become a problem too, especially for complex algorithms with lots of recursive signals. It is usually crucial in Faust to have the “mental global picture” of the algorithm to be implemented which in some cases can be hard.
This may go to the state machine requirement; I’m sure it could be done (Faust is Turing complete, at least in recent versions) and it shouldn’t be too inefficient, but it might not be a very natural fit language-wise.
Maybe the ideal language support sequence would be Faust first (for heavy audio rate DSP) and, despite what I said above, Lua next (for prototyping more sequencer-y, state-y stuff; it would be close to ideal for that, and much easier to embed and JIT than python+scipy).
Some time ago I tried to create an environment like this to prototype my modules. My idea was to use a backend from a language with JIT features and generate that from the Vult Language. After trying Js, LuaJIT and LLVM I decided to go with LuaJIT for the following reason:
LuaJIT is very easy to generate from other languages and has enough features. Converting Vult to LuaJIT was very easy.
LuaJIT is almost as fast as C++ (in most cases). In all my benchmarks the Vult code ran with LuaJIT is around 15% slower than C++ generated with Vult. There are exceptions, for example when making a FIR filter, the C++ code is vectorized.
LuaJIT is very easy to embed in an application and it’s easy to create multiple Lua VMs (to handle multiple plugins).
There’s only one catch. LuaJIT in Mac requires that the main application is compiled with some special flags. If I make a plugin for Rack that uses LuaJIT (in Mac), it will not run unless Rack is recompiled with those special flags. This will not be a problem if the feature is supported by Rack. In Linux and Windows this is not an issue.
Javascript (for the Vult -> Js code is about 4x slower than Vult -> C++).
I think Faust is nice. I’m a hardcore functional language programmer, that’s why I think is nice. However, I ended up not using Faust. My main issue was the difficulty of understanding my own code using just a text editor. For me, in order to be efficient with Faust I need to use the tool that generates the diagrams.
Faust is the main reason why I made Vult imperative.
I toyed with this approach a while back, using Python because that’s what I know. It’s not very hard to get something basic working, but I shelved the idea because I couldn’t get it to a reasonably-idiot-proof state (in other words: it’s trivial to make Rack do bad stuff with it). My concerns were mostly:
The Python API lets you do a whole ton of things that may undermine Rack’s stability, at the very least you could sys.exit(1) and close the whole app which didn’t seem desireable. It’s especially problematic because who knows how that library you imported handles errors.
More problematically, if you load a .vcv file and you have a Python interpreter running whatever code was saved in that file, you’re potentially opening malware with full access to your machine. While you can restrict the APIs available to Python scripts, that’s not generally considered a valid security measure against malicious code. Sandboxing would work but portability would become a huge pain in the neck.
Even if the .vcv file did not contain the script itself but merely a reference to a filename (so you’d have to download the .py file separately, and presumably you know that executing .py files you don’t trust is silly), I don’t know that there won’t be a way to abuse this. What if you could just append python code at the end of the JSON and somehow make both parsers accept the file, or something, you’d still get a fairly reliable way to execute arbitrary code on the machine of whoever downloads this.
I’ve also thought of a few countermeasures, in case that helps people:
The module has no to_json and from_json at all. Annoying if you’re prototyping over multiple days, but maybe manageable?
The module’s slug is randomized based on some machine-specific attributes, e.g. the hash of a MAC address on your machine, or of your VCV store username, something like that. It would probably work for most people, and break in really weird ways for others. Also, unless that information is a secret, others could still craft .vcv files targeted at you with the right slug in it. Maybe the slug should be randomized on first startup and saved somewhere in the Rack directory, idk.
Scripts have to be signed and the signature is checked against a white-list, and that whitelist is local to the machine and maintained by the user manually. Probably too cumbersome.
My hunch is that Lua and more specialized languages won’t have the same problem but I have not explored that at all.
Sandboxing is definitely something to think about and pretty much a requirement. Most embeddable interpreters allow this, like Javascript and Lua, so they can serve a purpose in secure contexts. Python is not a fun language to embed because the power of Python is in a large environment of external libraries and C extensions. Faust might be securely embeddable, but I see no mention.
PureData Vanilla, Purr Data, PureData Extended, or … ?
Pd is a mess thse days, I’d wait until things there settle into a single established version.
Both very cool, but SC3 is the more maintained and longer-lived system.
I’d suggest looking at Csound again if you haven’t recently. It’s continually maintained by a core of very talented developers and it’s got more features than you can shake sticks at. It also has a very nice API, a rather big plus for the lingo.
There is a plugin that has wrapped some Csound functions into VCV Rack modules, but it does not provide general access to the language. There are also the many VSTs created with the Cabbage program (an IDE for developing Csound programs), all based on Csound, but again without general access.
