Creating VCV Rack patch files from software

This is not a request for help, just me dropping some notes about something I’m experimenting with. Maybe some of my fellow plugin developers will find this interesting.

I’m intrigued by the idea of writing programs that output VCV Rack patches. I have this vague idea that I could define “recipes” for patches based on my own experience as a modular synth enthusiast, then the program could generate variations on the recipe.

In cooking, making gravy is basically the same thing as making pudding: they both involve cooking flour in a little fat and gradually stirring in a water-soluble base. The result can be very different depending on whether you add savory stuff (gravy) or sweet stuff (pudding).

In the same vein, we modular synth people all have recipes, basic stuff like “VCO goes through VCA, with VCA level controlled by ADSR envelope, …”. We have conceptual patterns in our minds, from which we can derive an astronomical number of actual patches by substituting specific modules for general ideas.

So my first idea is to experiment with reading/writing the VCV file format. Here is a working Python program that opens a vcv file, extracts the patch JSON out of it, then turns around and re-creates another vcv file.

#!/usr/bin/env python3
import zstandard as zstd
import io
import tarfile
import json
from typing import Optional, Any, Dict, cast


def ExtractVcvPatchJson(inVcvFileName:str) -> Optional[Dict[str, Any]]:
    # Decompress the vcv file into a tar image.
    with open(inVcvFileName, 'rb') as infile:
        decomp = zstd.ZstdDecompressor()
        reader = decomp.stream_reader(infile)
        buffer = io.BytesIO(reader.read())

    # Find the json contents inside the tar image.
    with tarfile.open(fileobj=buffer) as archive:
        for member in archive.getmembers():
            if member.name == './patch.json':
                if extracted := archive.extractfile(member):
                    return cast(Dict[str,Any], json.load(extracted))

    # Unable to find patch.json inside the tar image.
    return None


class Patch:
    def __init__(self, patch: Dict[str, Any]) -> None:
        self.dict = patch

    def writeJson(self, outFileName:str) -> None:
        with open(outFileName, 'wt') as outfile:
            outfile.write(json.dumps(self.dict, indent=4))

    def writeVcv(self, outFileName:str) -> None:
        # Create in-memory tar archive
        tarbuf = io.BytesIO()
        with tarfile.open(fileobj=tarbuf, mode='w') as tar:
            # Add patch.json
            binary = json.dumps(self.dict, separators=(',', ':')).encode('utf-8')
            patch_info = tarfile.TarInfo(name='patch.json')
            patch_info.size = len(binary)
            tar.addfile(patch_info, io.BytesIO(binary))

            # Add empty modules/ directory
            modules_info = tarfile.TarInfo(name='modules/')
            modules_info.type = tarfile.DIRTYPE
            tar.addfile(modules_info)

        # Compress tar archive with Zstandard
        tarbuf.seek(0)
        compressor = zstd.ZstdCompressor(level=3)
        with open(outFileName, 'wb') as outfile:
            outfile.write(compressor.compress(tarbuf.read()))

    @staticmethod
    def ReadVcv(inFileName:str) -> 'Patch':
        pdict = ExtractVcvPatchJson(inFileName)
        if pdict is None:
            raise Exception('Cannot load VCV Rack patch from file: ' + inFileName)
        return Patch(pdict)


if __name__ == '__main__':
    patch = Patch.ReadVcv('input/beetle_breakfast.vcv')
    patch.writeJson('output/beetle_breakfast.json')
    patch.writeVcv('output/beetle_breakfast.vcv')

I thought I would share this in case it inspires anyone else to write programs to read or write vcv format.

Also, I’m interested to hear from anyone else who has written code to generate novel patch files. Again, I’m not yet sure where this is leading, but my intuition is strongly pulling me in this direction, and I’ve learned to trust that feeling…

10 Likes

Interesting approach!

Probably it all comes down to how many degrees of freedom the program has. Will it just reproduce some of the developer’s favorite patch layouts or will it actually create new things and new sounds? Sort of like “Randomize” for the whole rack, including choice of modules and cables

The latter is probably extremely difficult to achieve unless you are willing to accept a lot of misses between a few hits with patches that actually produce anything meaningful.

Thanks for sharing. I actually have been doing the same recently. With json and python, it’s all super straightforward to do actually and a lot of fun.

1 Like

While I haven’t gotten as far as trying to actually create whole patches I have been using python to analyze or modify patches, presets, or selections.

It started with a desire not to have to use the file manager with Voxglitch Wavbank. Every time you have to use windows’ directory picker

which always starts you in an inconvenient place so you have to drill down to the correct folder every time and then you also have to save the preset so I just made a blank preset and used python to walk the whole sample library and make presets for everything so now my wavbank preset menu mirrors the folder structure for my sample library.

(in case anyone else want to test it out, here’s the link: rack_scripts/mksamplerpresets.py at main · dustractor/rack_scripts – you just need to supply the location of your samples folder with the --path argument.)

I also used this approach of making presets for a couple other modules such as:

a script for voxglitch autobreak – it doesn’t walk a folder structure, you have to supply the path with loops and it makes a preset for five samples until it runs out of enough samples to make the full five. I need to go back and work on that one now that I figured out how to do drag-and-drop with tkinter.

a bunch of one-off scripts to make patterns for voxglitch onezero and sickozell trigseq+ where it makes patterns based off of decimal expansions of irrational numbers. (I was bored!)

a script that read midi files, extracted the pitch information ignoring timing or simultaneous notes and dumped that into text files and made presets for onepoint.

I toyed around with the idea that it would be cool to get graphviz to render a patch as a diagram so I did get as far as figuring out that vcv patches are compressed with zstandard but then I realized that you don’t get any information about the names of inputs or outputs so the diagram is not very informative:

In another attempt at reading vcv patches, I made a script that reads all the patches in a directory and looks for instances of the wavbank module and then reverse-engineers which sample is actually selected based off the position of the SampleSelectKnob (which is useless if you’re using the wave selection input which I usually do.) I wish there were a way to interact with a patch ‘live’ so that I could get this value.

I’m still learning c++ and the rack api but I’m hoping it will be possible to make a module that can hook into the dataToJson event (for other modules or the whole patch?) and divert that data to a python script for further processing.

The most advanced script I have made so far – last week I figured out how to get tkinter to accept a drag-and-dropped list of files/folders so I made a thing where you save a patch selection (in this case it was – big-surprise – a bunch of wavbank modules already hooked up to a mixer and a sequencer and a sample-randomizer) but none of the wavbanks had a folder selected yet. The script lets you drag some folders onto a window and it saves a patch (or strip) selection with those folders selected in the wavbanks. It worked for the patch selections but when I tried loading a strip it crashed.

I haven’t even begun to consider making something that actually constructed a patch by adding modules, connecting cables, positioning things. That seems like way too much to consider.

5 Likes

Funny enough, sickozell trigseq+ is also how I started down this rabbit hole. Oh cool, there’s a bunch of values in the vcvm, what if I use python to generate a bunch of these presets. Then started doing that for other modules to make vcvm files, similar to your example of pointing to a folder for modules that save a reference to a file path in the vcvm.

1 Like

FWIW, the starting folder of file pickers can be specified. If it’s always starting from the default location, then the plugin isn’t doing what it needs to do to make picking files friendlier. So, open an issue with the developer of plugins that aren’t remembering where you last picked things from.

1 Like

So in this case, both @dustractor and I are referring to having multiple folders mapped in presets, where there might be different sets of files like samples. This is a lot easier than navigating through the os file browser.

1 Like

Yes, this is similar to my line of thinking as well. The patch file tells VCV Rack just enough information to instantiate each module, put it in the right location in the rack, set all the parameter values, and hook up all the cables.

Here are some things the patch json doesn’t tell you:

  • How many HP units wide is each module?
  • What are the names of the parameters, what is their acceptable range of values, and in general, what do they mean?
  • What are the names of each input and output port?

In the json representation, all you see for a module is something like the following. I’m using the VCV LFO as a familiar example for everyone here. You can do the same thing by hovering your mouse over any VCV Rack module, press Ctrl+C, and now your clipboard contains the json text representation for that module. Paste in any text editor to see the result.

{
  "plugin": "Fundamental",
  "model": "LFO",
  "version": "2.6.3",
  "params": [
    {
      "value": 1.0,
      "id": 0
    },
    {
      "value": 0.0,
      "id": 1
    },
    {
      "value": -0.40963858366012573,
      "id": 2
    },
    {
      "value": 0.0,
      "id": 3
    },
    {
      "value": 0.0,
      "id": 4
    },
    {
      "value": 0.5,
      "id": 5
    },
    {
      "value": 0.0,
      "id": 6
    }
  ]
}

I’m coming to the conclusion that for what I need, I will also need to create a registry of known modules, to define the inputs, outputs, and parameters in a way that makes sense from a human perspective; I don’t want to memorize that parameter[17] is the volume knob, for example. I just want to operate on the semantic concept of parameter.volume. This is just like me wanting to program in C++, not hexadecimal machine code.

In order to use a module at a semantic level, I will have to define a mapping between the controls and labels visible on the screen with the integer identifiers present in vcv patch files. This means probably that I will have to manually create a database of modules, which will be tedious. Fortunately, once done, it can be used forever and shared with other people.

With a mapping like that, like @dustractor shows in the tree diagram, you could start to better represent what’s in the patch in a way that a modular synth person could understand.

3 Likes

Hopefully you wouldn’t need to manually build a database. In the API docs, VCV Rack API: rack::engine::PortInfo Struct Reference there is a getFullName() function. Maybe a module could (temporarily?) override either each module’s dataToJson() or the main rack::patch::Manager toJson() and run a custom function that added the output from getFullName to the usual json output?

1 Like

For that and also many other useful module functions you would have to actually load the plugin and possible feed it some dummy zero samples to trigger process(). But yes, a plugin loader just for the purpose of gathering info would make many things easier.

This is a really interesting idea. I wonder about the complexity of a plugin loader. I don’t know if I would write code from scratch or fork VCV Rack and mutate it into what I want.

The idea of creating a custom module that loads inside VCV Rack itself is less scary sounding, mainly because I’m already familiar with creating VCV Rack plugins and modules.

From my work on Sapphire Echo, I recently learned how straightforward it is to write code that walks through all the modules in a patch. I used it in Echo to undo/redo moving other modules around when I insert an expander. I could use the same iteration technique to extract and save information about modules to a file.

Lots of food for thought here. I hope other people find this brainstorming useful, even if it’s not for the same project goals.

5 Likes

If I were to write a plugin loader, I’d have a good look at Rack to see how it’s done. but would then write my own loader, that serves only the purpose of loading the plugin into some kind of sandbox where I could query parameter ranges and whatever information a “living” plugin offers.

Good thing I don’t have to actually do this =)

In general it’s pretty easy. I’ve never done a cross platform one, but there must some something easily stealable from VCV Rack.

1 Like

Off Topic since Squinky is here: just spent the evening exploring Kitchen Sink, best FM operator I have found so far. The controls (Octave, Ratio and Fine) are perfect for musical purposes and the wave form crossfade is kiiler! Thanks!

2 Likes

tx!

The code for the module browser in Rack might be a good starting point. It loads plugins and constructs instances of ModuleWidgets, possibly also Modules, without actually running them.

I’ve actually already done this for my own long-term purposes–the code needs a bit of an update but it outputs essentially everything about every module that can be gathered from the API. I’m also very interested in this semantic definition problem, and would be happy to collaborate (although my time is, as ever, fractured).

I might be able to refresh and re-run the code this weekend, but if you want to take a crack at it, the easiest entry point (IMO) is a custom Rack compile that builds out Window::screenshotModules (code here). As seen in standalone.cpp, this is called when you call Rack as rack --screenshot 1.0, and it already does the crawling and instantiation of all available modules. It’s as easy as adding some API calls once you’ve got the module instantiated, and then firing off some delimited output, and you get screenshots as a bonus!

4 Likes

Paul Bacon’s LintBuddy might also be of some help, either directly as is, or as example code. Patch a cable between any module and LintBuddy and it gives you a list of all parameters, inputs, and outputs with index and full name that can be exported.

It is also a great tool when developing a plugin to make sure you haven’t forgotten to label any parameters, inputs, or outputs.

6 Likes

I am highly interested in something like this! I started writing R code to read VCV patches as networks for hobby analyses (similar to what @dustractor seemed to be doing). This extra level of input/output port classification would be wonderful!

1 Like

clearly an online database that others could contribute to would be a good thing…

1 Like