BASICally: a new module for writing code within Rack

[StochasticTelegraph (Mahlen Morris)]

The first version of BASICally is now available in the library, and there is documentation as well.

While other modules with a similar idea (writing code/math in a window within VCV) already exist (Formula, Formula One, Teletype), my approach and emphasis is different enough that I thought it worth doing. Here are some notable features:

• It intentionally bears a visual resemblance to the BASIC language (albeit a quite limited version of BASIC). BASIC is a language that many people know, once knew, or can pick up by looking at examples.
• The right side of the module is a resize bar; pull it to the right or left, and the code window changes size. Handy for reading those long comments without line breaks and for shrinking the module down to a small size when you don’t wish to edit the code.
• Four different run “STYLES” (see Controls), giving it the ability to act on a RUN trigger, or to run the most recent working version continuously as you type, or only run while a button or trigger is pressed.
• Edits in the text window become part of the VCV Rack Undo/Redo system.
• You can pick from a (small) number of screen color schemes in the menu.

I will note that for sheer CPU efficiency, Formula One probably still has an edge over BASICally, at least when every sample requires a lot of math. I like to think that BASICally is more approachable and perhaps more versatile, which is what I was going for. I find it useful even just for using less space than a series of modules each doing just a bit of math, or for testing out ideas that I have for other modules.

I have several improvements (more inputs/efficiency/scrolling editor/additional features) for the next version already written, but I am super curious what the community thinks of it.

Enjoy!

[cosinekitty (Don Cross)]

I’m a sucker for writing parsers and interpreters. So I’m curious, do you parse into some kind of bytecode, or create an executable parse tree, or some other thing? There are so many fun ways to implement a programming language.

[k-chaffin]

As mentioned in your other topic on this, I tried it out for CV control and it works fine. Also as mentioned there, I couldn’t figure out how to scroll and you replied that that will be for a future release.

Are you still thinking of using this for a “tape recorder” control type of functionality? I’m just thinking out loud as to what I can do with BASICally, basically. I typically use clock divisions and multiple for doing automation in one way or another. Is there a way to “reset” the script for sync purposes?

[main.tenant]

£600 for this?

Nope! People of planet Earth, I give you Poundshop Misha

poundshop misha.vcv (2.5 KB)

And this version uses STROKE to let the A,S,D and F keys control the buttons

poundshop misha - with stroke.vcv (2.8 KB)

[Andre_M (Yier Sansi)]

I love the trend. First it was C sound (it was too hard for me), now Basic. That’s cool! VCV is becoming its own thing, kinda different from just modular emulation.

[StochasticTelegraph (Mahlen Morris)]

This was more the “controller” part, not the tape loop recorder itself. A controller that, I quickly realized, could have just been any sequencer, but by then the uses for a BASIC interpreter within Rack had become impossible for me to ignore.

As to a “reset”, it depends. I’ll point you to two features:

• There’s a STYLE setting that starts the script from the top when RUN gets a trigger.
• You could, within a loop, test an input and do CONTINUE ALL, which goes back to the start.

But neither of these will interrupt a WAIT, and the latter is kind of clumsy. Is that what you’re looking for? That could be a new STYLE setting. I’ve also been pondering making little subroutines that activate on events, like:

ON TRIGGER (IN4) … END ON

Do either of these work in your context?

[k-chaffin]

I’ll have to think about it. I set up a patch with BASICally in it along with one of my typical automated setups. I’ll let you know what I find.

[StochasticTelegraph (Mahlen Morris)]

@cosinekitty, It would be a LOT faster if it was native bytecode, but doing that in a multiplatform way is way beyond my comfort or interest level.

I admit that having a reason to implement a domain-specific language again (and possibly the first time when doing so wasn’t a mistake) carried me through the effort required to make this happen. I too am a sucker for such things

Since it’s personally enjoyable to describe something I’ve been working on in solitude for two months, here’s how it actually works:

1. When the module notices that the code has changed, it invokes a Flex+Bison routine that emits a sequence of Line objects. Lines can be an ASSIGNMENT, or a WAIT, or an IF-THEN-ELSE, and so on. These are nested; a FOR loop inside a FOR loop will be a Line inside the other Line.
2. Mathematical expressions are stored in a nested Expression object. Expression has a Compute() method that knows how to get a value from itself and its sub-Expressions.
3. Line’s would be hard to execute as they are, so once I know the program as a whole compiles, I then convert them into a vector of PCode objects. PCode’s are what get executed by Basically.cpp. The Expressions stay as they are, running Compute() on them when needed. PCode objects are as stupid as I could make them.
4. There’s some tricky/stupid bits getting Expressions to interact efficiently with the IN and OUT ports. The next version will be a fair bit faster.

Some surprises:

1. I tried replacing some of the ‘case’ statements in Expression with function pointers, and it made them slower/less efficient. I note that ExprTk (the computation engine behind Formula One) uses ‘case’ statements as well, so while I found this result surprising, I’m less worried that it was me doing it wrong.
2. Wow, Flex/Bison is fast. And while I’d been tempted to write the lexical analyzer/parser by hand, I’m really glad I gave myself the time to figure Flex/Bison out, because adding new features is much more faster now. But I would caution anyone against using my Flex/Bison code as an example; it’s still very much in the “I don’t understand much of this, but it does what I want and doesn’t appear to leak memory” territory.
3. No, there is no good reason that PCode has a relative_jump instead of an absolute line number to jump to.
4. One of the weirder bits was making WAIT statements (which really keep the CPU usage down) efficient even when the length of the wait depends on a value that can change (e.g., WAIT 100 * in1). I realized at some point that musical instruments are less surprising when they are responsive to user changes, so I opted to do it that way.

Interestingly to me, I’ve likely taken as much time working on the visual aspects of the module as the “runs code” part. I’m particularly happy that I got the module resizing stuff (grab the right edge and move it back and forth) to work; not having to release multiple modules to accommodate different line lengths was a relief. The next release will include more in this UI vein; highlighting the first line the compiler can’t compile, vertical scrolling, slightly more color choices, a couple more IN ports, and so on. And at least one addition to the language (teaser: OUT1 = C#3).

Sorry for the long spew; I’ve been at it full-time for weeks now, and it’s just nice to write about the points I’m proud of.

[cosinekitty (Don Cross)]

@StochasticTelegraph

It would be a LOT faster if it was native bytecode

I think your pcode approach is pretty much what I meant by “bytecode”. You make your own simple virtual machine with its own instructions and execute them. It would probably raise some eyebrows at VCV Rack if you were generating machine code on the fly and executing it.

That would be possible if performance mattered that much, but you would have to handle both Intel and ARM separately. One time I did write a Mandelbrot Set imager for Windows that allowed entering alternative formulas by the user. It literally compiled the complex-valued formula into native Intel x86 math coprocessor instructions. It really helped because the formula has to be iterated hundreds or thousands of times per pixel on the screen.

There is a Windows function called VirtualAlloc that allows allocating memory into which your code can programmatically write executable machine instructions. I have no idea how to do that in Linux or Mac though.

So it sounds more like a hybrid approach for pcodes as executable statements and parse trees (?) as expressions. Pretty cool!

I think the reason case statements are faster than function pointers is because of the way pre-fetch and pipelines work on modern processors. The pipeline is racing ahead, pre-fetching and speculatively executing instructions. When it hits a call through a function pointer, that might stall it, because it can’t be sure what the value of the pointer is going to be when it gets there. (Total speculation on my part.) But switch/case are really a series of conditional jumps, and it will make a guess based on recent results. In repeated loops, most of the time it will guess right and already have the statement pre-executed before it gets there.

@StochasticTelegraph

I’m particularly happy that I got the module resizing stuff (grab the right edge and move it back and forth) to work;

I’m definitely going to take a look at that part of your code. I’m sure that was interesting, to put it mildly. When I saw how the new MindMeld PatchMaster literally melds adjacent controllers into a seamless unit, it really opened my mind to new UI possibilities inside VCV Rack.

[FiroLFO (Firo Lightfog)]

Oh, what a nice module! Any performance improvement is welcome as I see myself using it heavily. It just ‘feels’ better than FB Formula or docB’s Formula One.

• Is elseif going to be available?
• Is there a way to check if a cable is plugged in (see isconnected())?
• Any trick to include a string variable (i.e. seq1=“10101100”)?

[main.tenant]

Sounds good. In the limited tests I did I missed having some kind of array or list of objects structure, I was wanting to build a scale of tied voltage values to navigate. Obviously access to the n’th value and length of the list/array would help.

For what I was doing a % modulo operator would also simplify things.

It’s interesting you went the hand rolled Basic interpreter route rather than leveraging Lua support libraries.

Oh and one more ask - could you think about expanders that give access to extra ins and outs?

[k-chaffin]

Well, as it stands, I cannot get any type of reset behavior without scripting the behavior as you describe. Personally, I think that in order to be usable for me, there must be a reset input jack. Upon reset, I would expect your module to immediately terminate any loops or waits and start the script running from the beginning, immediately.

I like your event triggering considerations.

I was about to ask about nested FOR loops, but I see those are supported. Is there a limit to the nesting depth?

It seems to me that BASICally really needs array types with indexing. Probably a 1D array would be sufficient. Array initialization/assignment would be needed.

The module seems to hold promise. I like the editor, whereas I have not always liked the editor in some other formula modules.

[DaveVenom (Dave Benham)]

I think some knobs providing user selectable constants might be more useful than additional inputs/outputs.

I think support for polyphony is important. Your programmable module would be unique if you could figure out how to let a line of code reference multiple channels at the same time. The other programmable modules support polyphony, but the formulas are restricted to a single channel, and the channels are iterated implicitly. I think automatic channel iteration is important, so cross channel formulas could be tricky - perhaps you could simply have a rule that code can only access inputs from other channels, but not the outputs. You would need a read only iteration index variable.

If possible, allowing resize to make the editor completely disappear would be welcome.

There is a lot of real estate taken up by the good/bad indicator. I think all that is needed is a small LED. Perhaps constant green if good, and flashing red if bad (need to support color blind users). That would leave more room for other inputs/outputs/controls. Also the mode selector seems rather large when it is likely to be set once and forgotten for any given script.

[k-chaffin]

Personally, I would prefer that you to remove the STYLE knob and put the options in a right click menu. That would free up panel space for at least a reset input and perhaps button. I spent several minutes trying to find the options in a menu before realizing that was what STYLE is for.

FYI, for testing, I am using BASICally to trigger my new (unreleased yet) Meander smart randomization capabilities.

[cosinekitty (Don Cross)]

@DaveVenom

Perhaps constant green if good, and flashing red if bad (need to support color blind users).

I’m so glad you mentioned this. I am not color perception impaired, but about 10% of males are. (This is linked to the X chromosome, and men only get one of those, so much higher incidence than females.)

Far too often I see UI designs that blithely assume the user can tell the difference between green and red, which is the worst assumption to make. The difficulty in telling green apart from red is the most common form of color impairment.

It’s fine to use color to enhance a UI for the majority of users, but there should always be auxiliary cues that make use possible for this sizeable minority.

[StochasticTelegraph (Mahlen Morris)]

Thanks for trying it out, all of you, and all the thoughtful comments! I was kind of expecting a bit of “no, no, no, modular is finding ways to do math with circuits, not code” gatekeeping, so if those people exist, I’m glad they haven’t tried this yet

I’m going to answer some questions and probe for more details below, but none of my thoughts are final, to be clear.

@FiroLFO

• Is elseif going to be available?

I’ve bumped into wanting this as well. I’ll investigate.

• Is there a way to check if a cable is plugged in (see isconnected())?

Like isconnected(out1) or isconnected(in1)? I’m surprised this is desirable, since I feel like the author of the code would also be the person connecting cables to it (or not). Can you describe the situation you’re thinking of a bit more?

• Any trick to include a string variable (i.e. seq1=“10101100”)?

At first glance, this looks more like an array of booleans than a string. I’m thinking about arrays (see below), but would love to know more about how you want to use them in this case.

@main.tenant

Yes, arrays are of interest. In BASIC (at least the BASIC I grew up with), one would do a DIM[10] to make an array, but I think the need for explicitly setting array sizes is firmly in the past. But here’s a divisive question: 0-indexed or 1-indexed? I think 0-based, but that requires more explanation to new programmers.

There is a mod(a, b) function, which is really doing std::fmod(). I tend to think of ‘%’ as a modulo operator for integers, and integers aren’t really a thing in BASICally. Also, approachability of the language is also of value to me (I’m assuming most VCV Rack users have never written a program), and I worry that punctuation marks are more off-putting than mod() or pow(). I like that mod(foo, 1) returns the part of foo after the decimal point.

Yeah, why BASIC? It’s not like I’ve written any since, uh, 1983-ish. Well, at first I was struck by the joy in Hainbach’s voice when he realizes that he can program a tape machine with the language of his C64 childhood. And then I thought, you know, BASIC is still very approachable, in that I suspect that you can have no experience with writing software and be able to tweak examples to do what you want. BASIC was, after all, created to introduce inexperienced students to programming. I know there are academics who wish it hadn’t been, though.

I’m not suggesting that I would want to write longer programs in this tiny subset of BASIC, but the context (inside a modular system with a tiny UI) doesn’t really lend itself to long programs anyway*. There’s plenty of environments that do code->music well (CSound, ChucK, Pure Data, live coding languages, etc.), but this doesn’t call for that, in my opinion; this is, like Perl and AWK, more of a glue language, enabling other modules to work together in a novel manner. Or at least a gesture in that direction.

But I’d be fascinated if someone else decided to cram some significant portion of Lua in Rack. And, well, I think one could slap a different parser on it and generate the same PCode from a pseudo-Python or what have you. I just didn’t feel like explaining Python indentation in the docs.

More IN’s and OUT’s: I quickly noticed the need for more INs myself, so the current version of the module in development has 6 INs and 4 OUTs.

I’ve been considering pushing the edit window to the right enough to add another column of ports, which is probably enough for ~seven more ports. I’ve been thinking that 11 INs and 4 OUTs would be pretty good (with room for a couple more controls). Thoughts? Have you found yourself needing more OUTs in particular? That (so far) is less compelling to me. I personally dislike expanders, as they seem terribly undiscoverable to me (I’ve been using Rack for a year and only recently heard of them).

@k-chaffin I think this “restart” behavior is a really good thing to point out. Since I’ve been thinking that one-time-per run code could go into an “ON RUN…END ON” block as well (in part so they don’t get run every sample), I need to think more on how to make this essentially a pseudo-multithreaded system, and how these ON blocks might interact.

There is no set limit to how deep FOR-NEXT loops can nest. But do bear in mind the slightly surprising detail about FOR loops.

Arrays, yes, I agree…

Thank you for the kind words. It feels like I’ve spent more time on the editor than on the compiler. I wish I had gone ahead and started with making my own version of TextField instead of subclassing it for as long as I did. The next version will have simple scrolling, which I think you’ll all like.

@DaveVenom I know what you mean about some knobs, but then I feel like every time I see a knob I think, wait, what if I want to modulate it? I’m kind of frustrated that I can’t find a thin module with knobs/outputs/renamable labels, so I could just put it next to BASICally and hook them in. PatchMaster is almost there. So is MentalKnobs. So I’m torn.

Polyphony is really deep, as you describe. I’d love to have some use cases from you in the form of code and desired results to ponder over, since it’s something I never use.

Make the editor disappear entirely; that seems very doable. Smaller STYLE knob is a good point.

As you can see from the screenshot above, the Good/Fix area will be different in the next version. But flashing is pretty distracting to me, so I’m disinclined to do that. I think the fact that the word changes (from Good to Fix) is helpful. I tried looking at the indicator via Edge’s color-blindness simulators, and while the difference is present, I’ll tweak the color and text shades a bit to clarify that they are different states.

Thanks again for sharing your experiences.

mahlen

*That said, I recall hearing that the authors of AWK were taken aback the first time they saw a 200-line AWK program.

[FiroLFO (Firo Lightfog)]

Hi,

@StochasticTelegraph

Like isconnected(out1) or isconnected(in1)? I’m surprised this is desirable, since I feel like the author of the code would also be the person connecting cables to it (or not). Can you describe the situation you’re thinking of a bit more?

Using IN1 as a clock input I’d like to decide if the

• 0V means no cable (= code providing some internal clock) or
• 0V means a cable with no current clock pulse (= code looking for the next pulse)

Also using inputs for quad-min calculation I would like to know if IN3 is

• 0V because there is no cable in (= code ignoring for min() calculation)
• 0V cos of the incoming voltage (=code considering it for min() calculation)

VCV refers to this as isConnected() but basically a short ic() (read “I see”) function would be very handy.

@StochasticTelegraph

At first glance, this looks more like an array of booleans than a string. I’m thinking about arrays (see below), but would love to know more about how you want to use them in this case.

Main purpose: pre-programmed sequencer. Knowing that you are considering arrays you should ignore this suggestion.

@StochasticTelegraph

one would do a DIM[10] to make an array, but I think the need for explicitly setting array sizes is firmly in the past. But here’s a divisive question: 0-indexed or 1-indexed? I think 0-based, but that requires more explanation to new programmers.

I agree, the array size definition is annoying (specially for beginner coders). I’d also vote for 0-based arrays.

@StochasticTelegraph

Make the editor disappear entirely; that seems very doable.

I’d love that.

@StochasticTelegraph

Smaller STYLE knob is a good point.

I’d welcome that. Although I prefer @k-chaffin’s idea of moving STYLE into the context menu.

However it is, BASICally is a fun little module! I’m trying to figure out why I like this so much more than FB Formula or Formula One.

And finally one more question:

• Any hope introducing RND (like the rack::random::uniform() way)?

[main.tenant]

@StochasticTelegraph

here’s a divisive question: 0-indexed or 1-indexed? I think 0-based, but that requires more explanation to new programmers.

I always liked 0 based as the substring maths with LENgth() style functions always seems easier that way, plus 99% of third party code assumes zero based, but it’s not a big deal.

Yeah, other than a little bit of VB in MS Office and web pages I also haven’t used Basic since I had a BBC Micro back in the day.

I wouldn’t personally use DIM, just like you haven’t insisted on LET or line numbers, but again it’s not a big deal.

[StochasticTelegraph (Mahlen Morris)]

@FiroLFO Ah, I see why is_connected() would be needed here. Well, I still think you would know in your own patches, but maybe you’re thinking of handing out the code to others. Which would be cool.

I’m very happy to hear you’re enjoying it. I hope I’m able to keep the module enjoyable.

Oh, yes, a uniform random has been on my list for a while. Maybe even a Gaussian as well.

[demcanulty (Dan)]

Looks like a cool module, I know I’ve gotten a ton of usage out of max/msp’s javascript support. I still haven’t hit the point where I really needed to code something in vcv, but it might just be that I’m not thinking creatively enough. I love the idea of more codeable modules being available.

And just wanted to weigh in as supporting 0-indexing. Every time I have started supporting 1-indexing in that past in any project or product I’ve worked on, it just causes all kinds of problems down the road.

I would go so far as to say that 0-indexing is consistent and beautifu in it’s own way, it’s easily understandable once explained, and it reflects the underlying code in a way that is much easier to keep bug-free.

Whereas whenever I break this rule I always wind up with 1-indexing creeping all around and polluting the namespace with confusing variable names and causing me to write bugs.

Anyway, that’s just my ‘2 cents’