VCV Prototype

Yeah, I’ve also found Soul is easy to break but the idea of it is really compelling…
Basically audio shader fragments that run in your overall program and target DSPs that would normally be off limits.
In his talk he predicts a future where sound designers will just swap code-graphs back and forth that will run independent of any specific plugin or application.

Any consideration given to Clojure?
Leonardo mentioned above he’s very into the Functional Paradigm and it seems to be on the rise all over but is kind of held back by kludges like you see in its python implementation… giving kind of a mixed impression. Sucks that Faust can’t work for more demanding tasks.

As far as alternative “languages”, yeah Supercollider would be amazing… used to use that way back and definitely woukd never complain about sound quality or documentation.

If nothing else I like this LuaJIT solution mentioned because it seems to work well everywhere that it’s been placed so far. Straddles the macro and micro timescales with aplomb.

On a quick look at the Faust grammar and first-class features I suspect it will be pretty low-risk to embed. (Of course safety/security depends on the threat model and I’m not sure what you have in mind for this…)

Building the AST and then traversing it before passing to the compiler should handle some of the basic risks. (The functional paradigm and relatively simple grammar makes this a LOT easier than for general languages).

If the concern is breaking out of VCV Rack into the OS, a whitelist around foreign functions would probably do it, and there seems to be one and only one way to call a foreign function.

If the concern is crashing VCV Rack, less so, but as far as memory is concerned there aren’t too many functions which do big allocations and they should all be identifiable in the AST.

Good to know! I will, thanks. When I last checked in with it, things seemed to be dragging a bit, but that was a few years ago and it does look like there’s a lot of work being done on it now. (I do like the language design, and I have a lot of fondness for it because it’s what I learned on…)

Just to clarify (because I’m the one who brought up the limitation); Faust can do extremely demanding tasks very easily–as I understand it there are just a few classes of algorithms that are hard to implement efficiently because it doesn’t have robust support for processing audio at below-sample rates. So, for example, the library FFT routines get calculated at every audio sample, which is way more than you need to do something like a spectrographic display.

1 Like

To more concretely explore options, someone should implement any or all of the following scripts in a language of their choice, which will likely represent what beginner programmers want to create. Correctness isn’t the goal, just rough code overview.

  • Attenuator: Set outputs[0] = inputs[0] * params[0].
  • Gate logic: Set outputs[0] = inputs[0] XOR inputs[1] using the 0/10V gate standard.
  • Integrator: Compute y += inputs[0] * deltaTime and then set outputs[0] = y.
  • LFO: freq = FREQ_C * 2^(params[0] + inputs[0]); phase += freq * deltaTime; outputs[0] = sin(2*pi * phase); outputs[1] = triangle(phase); etc
  • Filter
  • Echo: Delay the output by 1 second.
  • FFT tilt filter: Take the FFT of the last N samples of inputs[0], multiply the log-log plot by a diagonal line with slope params[0], take the IFFT, and write the output to outputs[0].
  • Karplus-Strong
4 Likes

Right, but this is the part that’s giving me pause… you mentioned that it cant efficiently implement FFT, Convolution… I’m sure that would rule out Wavelets and other slightly exotic stuff as well.

Some of my favorite effects right now are Unfiltered Audio Zip and their Sunder RE looks really cool as well…

Edit: Ok, it’s been awhile since I’ve looked at Faust… wow it’s really come on in leaps and bounds! The Mass-Interaction stuff looks really interesting as well…

And now I see from the paper you linked,

“The current version of Faust is monorate: all signals are
isomorphic to functions mappings integers (clock ticks) to
(scalar) sample values. In [9], an innovative extension to Faust
for handling both different clocks and multi-dimensional samples has been proposed: these features are of key import when targeting efficient spectral processing applications…”

This is like opening a whole can of worms… now I’m lusting after that Bela and reading more about the Starling platform… Radium as well - sigh, it never ends and I have no time for anything more due to Unreal and Houdini already :frowning:

1 Like

I’ll start the ball rolling on Faust. These are sketches and I have not tested them (so please expect errors!) but they do compile and I’ve included the block diagrams generated by the online tool. @Vortico, is this what you had in mind? If so, I’ll continue as time permits.

Faust Attenuator: takes an audio signal and a gain parameter and outputs silence if gain <= 0, the original signal if gain >= 5, and a linearly attenuated signal if 0 > gain > 5.

attenuator(audio, gain) = audio * (max(0, min(gain, 5)) / 5);
process = attenuator;

faust_attenuator

Faust XOR Gate: sets an output gate to 10V if either (but not both) inputs are 10V.

xor_gate(gate_in_1, gate_in_2) = ((gate_in_1 == 10) xor (gate_in_2 == 10)) * 10;
process = xor_gate;

faust_xor_gate

Faust (Leaky) Integrator/one-pole filter: @Vortico, this is what you were asking for, right? deltaTime made me think you might have wanted a more-than-one sample delay with no leak.

deltaTime = 0.99;
process = + ~ *(deltaTime);

faust_leaky_integrator

(This is the canonical way to do it, though it needs some explanation. In Faust, A~B feeds the output from block A back into block A, through block B, with an implicit one-sample delay. Here, block A is +; it is adding the feedback from B and the (implicit) input. Block B is multiplying the output from A by a coefficient (presumably <= 1), and then sending it back (after one sample) to A.)


Notes on the above:

Faust does implicit connection between input and output streams. The Faust passthrough program is process = _;, which connects an implicit input to an implicit output using a straight wire (_).

For this exercise, I’m assuming that the Rack wrapper for Faust DSP is implicitly making all Rack inputs available as streams in order and taking all output streams and writing them to outputs[0], outputs[1], etc. This is typically done via architecture specifications but that’s more like tooling than core language stuff. Range conversion can also be done with UI primitives but I’m avoiding that above for simplicity.

1 Like

Just a quick thought: It may be profitable to contact the FAUST team to see about adding VCV Rack plugins to the list of targets for their on-line compiler. It spits out VSTs for Linux with a user-selected GUI toolkit, might be even easier for Rack plugins.

Yes, that’s exactly what I’m after. deltaTime is equal to the timestep duration, e.g. 1/44100s.

Could you elaborate more on how multiple inputs, outputs, and parameters would work in Faust? Say we’d have 8/8/8, and maybe 8 toggle switches. Instead of writing gate_in_1, would a user type predefined variables like input1 or input[0]?

@dlphillips: I’m actually about half done with a source-to-source translator that reads the default Faust C++ output (including UI primitives) and generates a Rack plugin. Only a few substantive steps, and nothing complicated. Baking it into Faust itself would be very easy. I think that once V1 has stabilized it would definitely be worth approaching the Faust team, and I’d be happy to share my translator before that.

@Vortico: Cool! Got it. I’ll fill in the other examples when I get a little time.

The naming thing is interesting in Faust since we’re in high functional territory and since Faust is used to defining its own UIs and interactions with its environment. I’ll explain what I mean and then make a draft proposal for how to handle it.

I’ll try to get a demo of source translation up in a separate thread; when I do, I’ll link it here. It may help clarify the below.

Background: In the attenuator example above, I used formal function parameters to put names in for clarity, but the attenuator could just have well been written process = _ * (max(0, min(_, 5)) / 5); (which would just grab the first two channels from the environment and match them against the two “wires”).

A typical Faust program, though, wouldn’t use the second full-speed channel as a parameter; instead, you would actually write process = _ * (max(0, min(hslider("gain", 0.5, 0, 1, .01), 5)) / 5); which sets up a horizontal slider in the execution environment (Faust WASM, JUCE VST, a standalone Jack/Qt application, whatever). Predictably, the slider I defined is labeled “gain”, starts at 0.5, ranges from 0 to 1, and is quantized to steps of 0.01. (Note that despite the single-rate semantics of the language, Faust actually processes parameters at a lower sampling rate; when I convert Faust code to Rack, I strip this out and make everything audio-rate).

Obviously, a good Faust -> Rack module transpiler could create Rack parameters and widgets for these UI elements, handle scaling, etc. But I think you want fixed inputs and outputs in the prototyper (i.e. loading a new script couldn’t and shouldn’t change the panel).

Proposal: For inputs and parameters, here’s a simple way to do the 8/8/8 => 8 model (which would require existing Faust scripts to be adapted, but which would be straightforward for new prototyping):

Require prototype scripts to define a designated function as their entry point (say rack_process). This function takes between 0 and 24 formal parameters, but their names are restricted to input[1..8] | param[1..8] | gate[1..8] (using 1-indexing since we don’t have array access syntax in Faust and this isn’t aimed at programmers anyway). Sequence and composition are not specified (so rack_process(input8, gate5, input1) is perfectly legal). Prototype scripts can also define other functions, but they cannot define a process function, since that’s the main Faust entry point.

The prototype plugin then wraps its entry point in Faust (process = rack_process;), compiles (LLVM via libfaust); uses the names and order of the rack_process parameters to establish data flow and standards conversion in C++ (== 10 ? 1 : 0 for gates, etc.), and then passes the function pointer (or however you want to do it) to Module.process().

There is a way of naming outputs but it’s clunky and limited, and not really needed. I would just let the first eight outputs consume the outputs of the block, and (probably) fail to compile if the prototype Faust code defined more than eight outputs.

2 Likes

Quick update:

I finished the translator for V0.6… fifteen minutes before the V1 dev build was released :stuck_out_tongue:

Anyway, with that out, this isn’t on anyone’s mind (including mine) but after I’ve absorbed the new API I’ll rework what I have and put it out. It’s definitely not feature-complete but I think I’ve flushed out (and worked out draft solutions to) some issues and choices that will come up in pretty much the same form for source-to-source conversion, Rack as a formal Faust target, and Faust as a prototyping language.

Anyway, other than updating the examples above (at some point) I won’t keep talking about Faust stuff in this thread, since @Vortico’s idea was to evaluate multiple languages. I’ll create a new topic (and put a link here) when the time comes.

If you’d like, I could ask Wes to design a panel (although he’s pretty busy with Grayscale stuff right now) and incorporate it into VCV’s “official” plugins like Fundamental or VCV Recorder, as long as it’s released under some open-source license.

I think Faust is a no-brainer at this point. It might even be possible to transparently support multiple languages, and if so, Faust would be a favorite among prototypers.

1 Like

Neat! I’ve come to think that a prototyping plugin would be a great official option, and I’d be happy to help out.

Just to be clear, what I’ve been working on isn’t really a prototyper, and it isn’t anywhere near production-ready; it’s a VERY crude offline source-to-source translator (in the form of a Python script) that chops Faust’s C++ output into a Rack module subclass which can then be built in the ordinary way. Useful but definitely for devs only. Some of its approaches may be a good starting point for an eventual prototyping plugin but there’s a lot of work left to do.

1 Like

So, PureData with a panel designer attached? :wink:

PD is embeddable, though I’ve never tried and it might require a spruced up GUI. That’s going to be the most approachable I can think of since clicky/graph editor stuff seems a lot more intuitive than writing code blobs but I don’t know what the effort budget is here.

Tangenially related, but strange: https://github.com/hundredrabbits/Orca

3 Likes

Orca is amazing. One of the users over in surge-synth land has used it to make some incredible (in many senses) sequences and we are already talking about how it could interact with rack as surge sort of rackifies bits of itself.

2 Likes

Orca: Woah. That’s like ASCII Monome. Fascinating!

Agree that PD or another graphical block language would be better for non-devs. I think the original impetus of this idea was either for devs (in their own prototyping) or fairly passive users of shared scripts (in which case code blobs is OK) but a Host-like embed of a PD/block environment (imagining a separate modeless window popping up) would also be an interesting thought.

Faust (as you probably know) actually is a block language, it’s just specified through text. I think there’s some Faust GUI stuff but I’ve never played with it.

This probably far exceeds the effort budget being proposed, but the idea still got me thinking.

prototyper.pdf (169.2 KB)

That involves defining the API from the script’s perspective, so you’re more than halfway there.

It seems that any embeddable languages that can be wrapped into a simple class with load(const char *script) and process(float *inputs, float *outputs, float *params), and with a reasonable build system, are welcome to be hosted by this plugin. If you’re willing to create this “backend” for Faust, and if others are interested in creating backends for other languages, I’ll create the frontend (handler code for loading and serializing scripts and calling the above methods). It’ll just be a while, possibly a few months.

1 Like

@Vortico–IDK halfway :slight_smile: but at least some of the way…

Anyway, this sounds good–sign me up! I’m doing some actual dev work in Faust anyway so it’ll be a nice opportunity to give back on the side (all open source on whatever license and with whatever copyright assignments you want). A few months sounds about right for my schedule, as well.

I’d love to solicit public comments as this develops–should I create a Faust Prototyper thread in Development and link back here?

@Skrylar–also really interesting. To some extent it seems like a sur-problem of the single-script prototyper so I can probably best support those ideas by working on the Faust sub-problem, but I’d love to hear further thoughts you have and am curious to see where it goes!

2 Likes

I feel dumb for not bringing up Julia in my original post. Used to work with that at UCLA when it was v0.2 and completely forgot about it. (In fact, I’m going to rewrite Rack in Julia now :stuck_out_tongue: ).

If we support Julia and Faust, I believe we’d cover more than enough ground for prototypers. And maybe throw in Javascript with duktape which would probably take 30 minutes once we have the abstractions in place.

Anyway, can we decide on a language-agnostic plugin API?

process(inputs[8], knobs[8], buttons[8], outputs[8], lights[8])
processBlock(N, inputs[8][N], knobs[8], buttons[8], outputs[8][N], lights[8])

On API:

Wouldn’t you definitely want an init method in the API and probably a stream/restore handler also into the data block? I could easily imagine these prototypes wanting their own persistent state of some form.

I know when I’m in a plugin I use rack::INFO() a quite often and see messages in a log; a method for foreign language to add a log entry could be super useful either to standard logging or perhaps to an in-module console? I think there’s some other debugging tools you may want in the wild but I’m not sure what they are yet.