ModScript: Using scripts for a better and richer integration with MIDI controllers (beta soon!)

Hello there, today is announcement day!

I can finally introduce the result of one year of research and development! ModScript Lune and its accompanying helper is yet another scripting module for VCV Rack, it uses the Lua language. It focuses on integrating with MIDI controllers and enabling interactions that would be very tedious or even impossible to do with current MIDI modules as per my research (VCV’s, Stoermelder’s etc.).

Screenshot

From the readme:

ModScript’s approach differs from the usual “everything-is-a-module” paradigm: it allows you to write all the MIDI interaction you need using code that is easy to read and write and to remain focused on musical elements in the patch.

It is my opinion and my main drive from the start in using and developing for VCV Rack that MIDI controller interaction in computer music is too often bland and limited to 1:1 mappings between your fingers and the software. Most modern DAWs use some kind of scripting or lightweight code for this purpose, I only hope to help Rack step up to that level too!

The scripts used in ModScript are simple text files that are interpreted by the module directly, they are easy to read and to write. These scripts have the ability to read and write any parameter from any module in your patch, they can also read the values of lights. In Pleine Lune there is also MIDI input and output, you can select the device to use in the right click menu and process the received MIDI messages directly in the script.

This video demonstrates the features of the script I made for the MPK Mini mk3, the intent is to show how PhraseSeq16 reacts to the script, take note how the pads select the step and the joystick toggles the gate types; this is also reflected on the screen by sending SysEX messages to the device, although a little glitchy as the program name is not supposed to be updated on the fly.

Features

  • Script auto reload on change, errors and logging in terminal
  • Read parameters and lights
  • Add and remove cables (experimental)
  • Modules and parameters referenced by ids, easily accessible in helper module
  • Chose scripts found in the scripts/ folder in right click menu
  • Control parameters: normalized (0.0-1.0), raw values, relative to current value (for relative knobs and increment/decrement)
  • Library of useful functions (MIDI message type, inspection) and objects (trigger, boolean trigger, pulse generator) and known modules.
  • Insanely fast LuaJIT engine, forked from VCV Prototype

ModScript started literally as a fork of VCV Prototype, removing every language except Lua, a lot of code is kept and then I added many functions to the API and features like MIDI i/o.

I am very much into acid and a big fan of the Roland TB-303 and I have created a monster that’s arguably almost as unpractical to use as the original! The script has (almost) everything to never look at the computer: select step, toggle gate, accent, tie and slide for each step, select note with keys, knobs control the synth and filter, select the memory, change waveform, select octave shift, alter notes being played with the keys. Most of it is shown on the screen, current memory, selected step number, gate types and even the note name! The script (~360 lines) is not very complex but the MPK is such a tiny and limited controller that it feels a little extreme to do so much with its limitations!

Helper module

In VCV Rack, every module in your patch is assigned an id. This id is a very long integer and it is needed to reference the module whose parameters you want to modulate; parameters are also assigned a numerical index, starting from 0. The Lune Helper module continuously displays the ids of the modules and parameters you hover with your mouse, it can also copy the module id in hexadecimal notation or its “dump” to your clipboard. This makes it easy to build your script and your patch in parallel and control new parameters as you add modules. The dump button automatically generates the code needed to access parameters with their meaningful names (e.g setParam("Frequency", 0.5)) instead of obscure indices.

Check out this live coding video I made, starting from a basic example and binding an APC Mini to the excellent Voxglitch Groovebox, speed is 400% of realtime. The result is a script that controls volume, mute, solo and the selected memory, much more could be done!

ModScript is open-source and released under the terms of GPL v3.0 or later, the code is available here: GitHub - Simon-L/ModScript

Acknowledgements

Heartfelt thanks to @falkTX and @dreamer, also big thanks to @heffalump and the overall very welcoming messages in the #development channel on Discord as well as the developers of the wonderful modules featured in the videos: @marc_boule, @slimechildaudio, @clone45, the VCV Prototype contributors and the WrongPeople Lua module.

Since I posted here 2 years ago (!!) and there last year I have remained focused on the integration of controllers in VCV Rack, hopefully ModScript is going somewhere in that regard. Faya, a sequencer module designed specifically for the Akai Fire I am also working on should become available before the end of the year.
I wish I had a beta build to offer, sadly there are still a few bugs and the module needs more testing. I hope this post and the examples will be engaging still. I am genuinely interested in knowing if the community has interest in this like, what controllers do you have? What do you think it could be used for in Rack? So far scripts have allowed me to think big!
I would be very pleased to receive feedback and suggestions here. I hang out on Rack’s Discord server too. Thank you to anyone who read this far! :slight_smile:

PS. Please don’t move this to the Development forum, thanks.

21 Likes

Awesome result!

I vaguely remember thinking about that would be cool to have similar scripting capability within Rack, and do it in Python as module, that was also pre sysex fix. Why do you go with Lua, btw ?

And I have AirMini32 controller, at some point I tried to tinker with it on low level, AFAIK to make it work it involves sending some initial boot sysex sequence. I found that there are scripts for Ableton in Python with implementation for this and many other controllers, so I think this can be used as reference GitHub - gluon/AbletonLive11_MIDIRemoteScripts

2 Likes

Amazing!

1 Like

This is great! Excited to try it out. It’s good timing for me. I’ve been building some quadraphonic performance templates, and a good 2/3rds of the modules end up being utilities for converting various control voltages to the formats modules want.

For controllers, I’ve got a Maschine mk3, Maschine Jam and Novation LaunchControl XL as well as a few custom TouchOSC templates.

It’d be interesting to see if the Maschine Jam could be adapted to work instead of the Akai Fire for your sequencer.

1 Like

Very, VERY interesting! I think music tech is approaching an inflection point in hardware control and I think this is likely going to be a part of it. Can’t wait to try this out.

1 Like

Great!
14-bit midi CC I/O functions ?
OSC send/receive perhaps in the future ?

Looking forward to playing with this.

I think that it might be an evolutionary step in the right direction.

This is awesome. Another vote for OSC.

Awesome. Perfect timing. I own quite a few rather old MIDI controllers and like you said, experience a hard time integrating more advanced control functionalities beyond one to one mappings. Often I end up with a bunch of cross wired logic modules, Schmitt triggers, flip flops and what-nots that become really tedious to debug or modify. So I’m really looking forward to experiment with ModScript.

AirMin32 has a button that toggles between 4 modes, in first mode all controls send messages to MIDI, in other 3 modes messages going to HyperControl. I think needed ability to get messages from multiple devices within script.

Screenshot 2022-09-06 at 10.10.01

I am absolutely humbled by the replies here! Thanks to everyone for the support, all these messages really mean something to me! I have delayed presenting this module for too long but I’m at a point in the process where I need to get feedback to keep going in a useful direction, again big thanks everyone!

OSC: (cc @trevormeier @Jens.Peter.Nielsen @auxmux) It’s absolutely a thing I have considered already – it might actually already be possible to load an OSC pack/unpack lib in the scripts and use a socket to send and receive messages, I’ll try that and report! The actual plan is to add this feature in the module’s code directly as it would be much better (and faster!).
About OSC in general: I have been convinced for a long time that Rack would greatly benefit from a complete OSC interface, report about the current patch and to be able to reach any parameter just like with the scripts!

NI hardware: (cc @trevormeier ) Thanks for bringing this up! I love the hardware from NI in general but sadly all the advanced features are of no use in VCV Rack because they are not implemented using MIDI but through another mean called HID, simply put their controllers are like complex joysticks or keyboards with lots of buttons, leds and even screens. I have myself started working on that, thanks to existing open source work. A version of ModScript designed for NI/HID interaction instead of MIDI will happen some way or another!

I will consider working on this once the module has reached a beta state. I was busy the past 2 days banging my head against a wall trying to fix a damn bug causing a nasty crash!

And I too have a question for you: what operating systems are you on? I have developed exclusively on Linux so far and I’m working on ensuring Windows compatibility.

1 Like

I have both an APC40 and APC Mini, and neither is of much use except in Ableton Live, and even in Live, I can’t use both controllers at once. I’ve also found that they’re both pretty useless in VCVRack because of the way the MIDI works.

You can’t - for example - map the channel faders on an APC40 to the channel faders on a Mindmeld Mixmaster. Each of the channels uses the same CC/Note values in different MIDI channels.

EDIT: The APC40 is even screwier than that. The 8 channel faders are the standard VOLUME CC on channels 1-8 but the master volume is Controller 14 on channel 1.

I’ve just spent a little while looking at HyperControl, sadly as opposed to some other “smart” MIDI protocols it hasn’t been documented much nor was it reverse engineered. The Python scripts from Ableton Live do tell us how to engage it using SysEX though! My guess is that it’s like most other similar proprietary protocols, it’s just some undocumented SysEX to/from the host software. Given the Air Mini32 doesn’t have infinite/relative-mode knobs I don’t think there’s much to get from it except what the normal MIDI mode will allow.

Multiple MIDI devices is still relevant though, thanks for the suggestion!

Edit:

I would have picked Python if it was easier but there are currently known bugs notably with multiple interpreters running in parallel. More details here, I worked on embedding pypy (a faster Python interpreter) and got good results but sadly all of that was lost when my computer was stolen last year.

In the end, I don’t regret going for Lua mostly because it is really fast and efficient. I wanted to guarantee the best performance even with the most complex scripts, ie. ModScript isn’t made for prototyping, I want it to be trusted by musicians performing live using several controllers.
The engine used is called LuaJIT and although I do not understand the details, it is generally regarded as one of the fastest and often considered the fastest scripting engine.

1 Like

I’m on macOS 12.5 on an M1 Max here.

HID support would be incredible. That would be a massive leap forward in usability for VCV rack. In the meantime, I’m often using my Maschine mk3 and Maschine Jam in MIDI mode alongside a LaunchControl XL and TouchOSC. It’s not as powerful or flexible, but it’s still great to have the hands-on control.

Very happy to see this come to light!

Quick question: does the Maschine Jam allow any leds or the value of the 8 touchstrips-fader to be set using MIDI messages from the computer in MIDI Mode? If it’s possible to control the feedback on the controller, then it’s worth experimenting! The LaunchControl XL could be very capable too!

It makes me happy to see development in this direction. About 15 years ago I was a developer for a commercial 3D modelling and animation tool company. I “embedded” and “extended” Python into the runtime and the API. The results were amaizing, IMHO :wink: But, it was an absolutely challenging endeavor, and would not have been possible without being a company developer.

I’m looking forward to seeing how you progress and wish you the best.

2 Likes

Thank you very much, your support is very encouraging! It must have been a very different time for Python, for programming and even more so for embedding an interpreter for a dynamic language! This reminds me 15 years ago was about the same time that I started programming, and my first language was… Python! It was like, version 2.5 or 2.4 even?

re: OSC
I’ve just made some tests and although it will be rather easy to do directly in the scripts, there is a bug with the network library used. This issue is related to a broader milestone that I wish to work on soon™ so I will postpone OSC support until then, sorry about that! I opened an issue to keep track of it: luasocket broken · Issue #3 · Simon-L/ModScript · GitHub

In the meantime, I tracked down the bug that prevented multiple Lune/Pleine Lune modules to be instantiated. 2 scripts driving 2 controllers means more lovely modules from our favorite developers in the same shot :heart: :

2 Likes

I just now looked at my source code from 2008 and I was using Python 2.5 at the time. Probably the most useful thing I did at that time was to expose the entire SDK and API so that external “dot notation” object oriented languages could do just about anything that the application runtime could do… This enabled me to do applications and web browser plugins via Python, Javascript, etc. It was as powerful but not necessarily so useful for the main 3D application to be able to load, interpret and run Python files, which themselves might call back into the application DLL that I created.

By the way, I used UDP for inter-application and inter-user communications. I did a worldwide metaverse system for shared realities. All of this was realtime 3D, so quite the challenge. Along the way I did some MIDI communications such that I could control real-time 3D modelling and animation via a hardware MIDI controller, the Evolution U-Control UC-33 .

Within the last 4 years, I did a VR/AR app where I could talk to hardware via MIDI or OSC.

I believe in you but I respect the magnitude of the challenge.

Actually, looking at source code, my 2019 VR/AR effort utilized OSC over UDP.

1 Like