SVG panel designer (with React) and NanoVG draw hot code reloading

I know there is Inkscape and other great vector tools. I took a slightly different approach.

I like to code and web is one of my many homes. So I used React to create an environment where I code my UI. This can and most likely will be extended to a full editing experience. The window on the right with the Rack Module is Chrome, Just running this on macos in splitview without Toolbar.

Would there be interest in something like this:

This allows to create interactive components. SVG if you take some time is not so hard to unterstand. You can till create components in vector tools and export to svg then import into your code.

This may be something for users who are more comfortable with code than with design tools. You can even import the ComponentLibrary, but I didn’t do that, create your own components or use a Free one like RUIL: Design Library for VCV Rack Modules

The red vertical lines are the rack units. The grey thick lines are 1cm, thin lines are 1mm. The display above is the current cursor position in mm when you hove the mouse inside the Rack Module.

The copy button copies the svg code and you can paste it into a svg file (download could also be implemented). text objects are convert with the commandline using inkscape in headless mode.

I didn’t implement a pan/zoom yet (matrix transform on the svg), but I have code I could use for this. Just using browserzoom right now.

Another Idea for this would be to create Interactive manuals

21 Likes

Looks interesting Rainer…

Thanks.

I’ve improved the workflow a bit, albeit I haven’t been able to configure automatic import of the svgs like I want them to have. But as workaround I convert the svgs to react components (which is roughly the same as svg +/- some syntax differences).

create-react-app is capable of doing this already. The problem is: I have to have an svg element in the svg file all the time, but I just want to use fragements like path or rect or g. In or to be able to modifiy the position i need to pass this file some attributes, but this is exactly what is failing.

I had a look into @svgr/webpack which is used under the hood by create-react-app but haven’t been successfull.

So what I did in the meantime is using svgr as a cli tool to convert the components.

Just in case here is a webpack expert on the board. Maybe he can help out with this.

This step is necessary in order to position the components, otherwise I can’t set cx,cy or the transform method on a group.

Nice touch to it: svgr uses svgo to optimize the SVG, but this also means you should always position your components at 0,0 otherwise svgo is optimizing the transform into a path … On the other hand you can deactivate optimization. Anyway.

TLDR;

If someone need want’s to colab or just doesn’t want to spent time on UI write me a message.

This allows me now to easily compose whatever parts of the UI is in different modules in almost no time.

1 Like

Proper

Just in case someone needs a reminder, how easy it is to work with SVG directly:

https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths.

I just extended the development experience with hot reloading for the draw function of the widgets. Now you can prototype your nanovg code very quick. I probably end up integrating this two tools.

For this I used https://github.com/mitsuba-renderer/nanogui/ and a package called reloadr that hot swaps functions or classes on save. This approach should be fairly save here, because I extracted the draw function, and just calling it on every redraw. So swapping this function out shouldn’t show side effects as long as you just draw and don’t do nasty stuff with it.

What does it mean? Quite easy: You change your code, save it (or let auto save do it, like in my case) and you get an instant change/draw of your code. I think this will very usefull, for scatching/testing/prototyping.

Maybe this is something we could even integrate into rack. I’ll have a look at whether I can create a plugin, that runs a python runtime and integrate it with rack. Maybe a custom build/fork … it that’s something Andrew can agree with. Let’s see how far I can get it.

Could be a good extension, or companion to https://vcvrack.com/Prototype. Or maybe Prototype could have a second modul with nanovg bindings to let you draw your own UI in Rack, that this would solutions would be obsolete.

Maybe it even would allow performant code in Combination with cython to prototype/develop full blown prototypes with the same or nearly the same performance (to be verified). https://notes-on-cython.readthedocs.io/en/latest/std_dev.html

I don’t know how far I’ll dig into this c/python cython thing, but if there will be some result I’ll let you know.

Now for the setup:

git clone git@github.com:mitsuba-renderer/nanogui.git
cd nanogui
pipenv install --editable .
pipenv install reloader

I’m using pipenv because I like to keep my env clean. But feel free to use pip or directly install into your system with pip and without any virutalenv.

Here is the rack test chamber:

import gc

import nanogui
from nanogui import Color, Screen, Widget, nanovg
from nanogui.nanovg import RGB, RGBA, NVGsolidity
from reloadr import autoreload

RACK_GRID_WIDTH = 15
RACK_GRID_HEIGHT = 380

def draw_rail(ctx, size, theme):
    hole_radius = 4.0
    rail_height = 15
    ctx.BeginPath()
    ctx.Rect(0.0, 0.0, 1000, 1000)
    ctx.FillColor(RGB(*theme['background']))
    ctx.Fill()

    for y in range(0, size.y, RACK_GRID_HEIGHT):
            ctx.FillColor(RGB(*theme['rail']['fill']))
            ctx.StrokeWidth(1.0)
            ctx.StrokeColor(RGB(*theme['rail']['stroke']))
            # Top rail
            ctx.BeginPath()
            ctx.Rect(0, y, size.x, rail_height)
        
            for x in range(0, size.x, RACK_GRID_WIDTH):
                ctx.Circle(x + RACK_GRID_WIDTH / 2, y + rail_height / 2, hole_radius)
                ctx.PathWinding(NVGsolidity.HOLE)
        
            ctx.Fill()

            ctx.BeginPath()
            ctx.MoveTo(0, y + rail_height )
            ctx.LineTo(size.x, y + rail_height)
            ctx.Stroke()

            # Bottom rail
            ctx.BeginPath()
            ctx.Rect(0, y + RACK_GRID_HEIGHT - rail_height, size.x, rail_height)
            for x in range(0, size.x, RACK_GRID_WIDTH):
                ctx.Circle(x + RACK_GRID_WIDTH / 2, y + RACK_GRID_HEIGHT - rail_height + rail_height / 2, hole_radius)
                ctx.PathWinding(NVGsolidity.HOLE)
            
            ctx.Fill()
            ctx.BeginPath()
            ctx.MoveTo(0, y + RACK_GRID_HEIGHT - 0.5)
            ctx.LineTo(size.x, y + RACK_GRID_HEIGHT - 0.5)
            ctx.Stroke()


@autoreload
def draw(rack, ctx):
    try:
        ctx.Scale(2,2)
        ctx.Translate(0,-RACK_GRID_HEIGHT/1.25)
        size=rack.size()
        light = {
            "background": (0x30, 0x30, 0x30),
            "rail": {
                "fill": (0xc9, 0xc9, 0xc9),
                "stroke": (0x9d, 0x9f, 0xa2)
            }
        }
        dark = {
            'background': (0x13, 0x13, 0x13),
            'rail': {
                "fill": (0x25, 0x25, 0x25),
                "stroke": (0x00, 0x00, 0x00)
            }
        }
        theme = light

        draw_rail(ctx, size, theme)

        # Panel
        rack_units = 10
        margin=2
        ctx.BeginPath()
        ctx.Rect(RACK_GRID_WIDTH*margin, RACK_GRID_HEIGHT, RACK_GRID_WIDTH*rack_units, RACK_GRID_HEIGHT)
        ctx.FillColor(RGBA(0xff,0x05,0x17,0x40))
        ctx.Fill()
    except expression:
        pass


class Rack(Screen):
    def draw(self, ctx):
        draw(self, ctx)



if __name__ == "__main__":
    nanogui.init()
    # create a fixed size screen with one window
    rack = Rack((800, 600), "Rack Test Chamber", resizable=True)
    rack.set_background(Color(0x13, 0x13, 0x13, 0xff))
    rack.draw_all()
    rack.set_visible(True)
    nanogui.mainloop(refresh=1 / 60.0 * 1000)
    del rack
    gc.collect()
    nanogui.shutdown()

Run the script: python rack.py and change the draw.

UPDATE: Ported the rack rail code and added some exception handling so the client doesn’t crash instantly when having a syntax error. Colors are adapted for my dark theme, but original theme is available. But feel free to use the original values

3 Likes

Good stuff!

It would be nice to have a built in scripting language for rendering in Rack! I dont use svgs and I wrapped nanovg in a way that makes writing draw code less verbose and have an editor module that lets me arrange ports on other modules and save the layout to be used by them but its very incomplete and messy at this point… it is all inside rack though.

For live reload I use a script that runs make and opens/closes rack on file changes. So it’s only as live as much it takes to build the plugin.

It’s interesting what you’re exploring here and I’m following this thread to see where the cython stuff goes.

In other news, Rack 3.0 might use another rendering lib.

I’ve been reading this news. And I think it will not happen in 2021 … so given we don’t even have v2 right now, I’m relaxed speding time on this topic. Having skia would be nice, so then I probably would use Flutter for sketching and drawing as its built ontop of skia and has hot code reloading already. Works like a charm. Or just use one of the many bindings available. Probably it would probably work exactly like the current solution. But I don’t think V3 will drop the current widget API, so I guess there will be a shim layer/emulation layer for plugins. And than later may deprecate this layer. But this is just gut feeling and guessing.

Is your module code checked in at github or some other location. I might have a look at it.

I also been thinking about the in-rack-approach. Probably the best solution I could come up with is to have one of the scripting engines already available in Prototype have the widget api exposed. So the idea is to have a module that exutes a draw.py or draw.lua … draw.*** you name it and pass the nvg context to it. To have hot code reloading the module would have a udp server and on the command linethere is a file watch script sending the script to rack when there is a change notification. I just wanted to have a scratchpad in the first place for learning nanovg and having fast edit continue cycles. And the first shot was already the best solution I could come up with. So I didn’t look further.

The translation is straight forward. You can even wirte the code without the pyhton naming conventions.

so in the python binding its ctx.LineTo in rack it is nvgLineTo. So translating your code back or forth between python an C is a matter of seconds/minutes.

Haven’t uploaded the editor module to anywhere yet, as it is not quite there to be used generally (and I have no released modules built with it) but what it does basically is to attach to a module by covering it with a layer and convert mouse click/drag events into selecting and moving widgets, while continously exporting the layout (a json file containing ids of params/ports paired with position data). I’ll post it when I get around cleaning it up. Still has some major problems with how it handles custom widgets that arent ports or parameters.

The problem I see with this is if I want to change something later, I have to go through manually translating back again or stick with cpp. Same as with the js prototyping. Personally, I’d rather not use two different langs or environments for prototyping and making actual modules, as these two phases aren’t distinct enough in my practice and I’m afraid it would result even more of my modules getting stuck permanently in the prototype section. I’d like to see python to be used in the rendering of non-prototype modules, or to have hot reloading for rendering bits in any way. Still, having tools like this can be useful for a lot of people, and can pave the way for future solutions, so don’t take my words as discouragement!

I see your point. And understand … So I’ve been extending my experiements and research. So I have a couple more options.

Drawing on a HTML5 Canvas. The API of NanoVG is modeled after the Canvas API, so there is probably no learning Curve at all. This is already working in the designer module. With animation and everything. Follow up would be to save the canvas as SVG. Which is also possible. There are different solutions for this working and I will just experiment a bit more to figure out a good workflow. Which kind of also makes the python NanoVG Draw unnecessary. I still quite like it and keep it, because of the instand changes it just takes something like between 200 and 600 ms roughly estimated to swap the code. So it’s instant. Another Idea would be to have a designer completely written in Rack or with just with the python solution I have right now. I also started to experiment with skia-python which looks promising. Skia has an experimental SVG render which translates all drawing code to svg, so I can store it as svg. (https://github.com/kyamagu/skia-python/blob/d7a3adb7a6e9cb1a92a59cb3f75c503a8259ba6f/notebooks/Canvas-Creation.ipynb) This also work s with the HTML5 Canvas. Just lets see where I get from here.

If I could whish for what I really wnat is a complete Designer Experience within Rack, also with DSP modules available so that the designer could compile an extension. Something like a visual programming Environment. Like in Unreal Editor. Something like this https://github.com/jagenjo/litegraph.js Still exploring… Here is another idea to include into the designer … which is kind of similar: https://github.com/mattdesl/canvas-sketch

And another option I’m exploring is konva.js where I can also generate SVGs from different layers. https://konvajs.org/docs/data_and_serialization/Serialize_a_Stage.html

I think the first step towards having an editor within Rack is to standardize a module layout format. This is what I’ve been poking at. Svg is good for draw data, but it might makes sense to have a format that is specific to modules, ie it would have a concept of parameters, input/output ports and lights, the rack grid size, labels and custom widgets, and could contain relations like labels paired with ports, groups of knobs, a line connecting two ports etc.

Such format could be used alongside or without svgs, but the point is to have the layout be the source of the panel instead of the graphics be the source of the layout, if that makes sense x)

Yes it makes sense. I already been searching for this kind of data, when I started. For example a json file. that describes the complete layout of the modul with,parameter ids, labels and positions, svgs that are used as assets, or to cut it short: A parsable, declarative file with all information describing the module.

Super interesting thread! I wanted to ask whether there were any updates in the meantime.

Some context for my interest in this: I’m looking in a way to quickly design custom modules that I can use as an interface to control others in the same patch via midi mappings. A bit more about this idea here: Interface module to control several modules at once - #8 by mosphaere

Two more specific questions:

@theshoemaker I wondered on which solution you converged (nanogui, skia-python, or something else) and whether you managed to more tightly integrate it (along the lines of the Prototype companion idea you mention above)?

@unlessgames, I wondered if perhaps your own workflow you’ve described above (the editor module you mention and the close/recompile/reopen rack on file changes workflow) have possibly reached a stage where you’d be up for sharing? I wouldn’t mind messy code, any snippets would be useful.

Hey, I’m not working on modules right now, stopped a while back and haven’t had the time/urge to come back to it.

I’ve made the editor module although it is very unstable and lacking features. I went the way I explained here, made up a layout format and you can drag around knobs/ports in Rack on the edited module, then it saves the layout as a json and the modules load it. I retro-fitted my previously released modules with this stuff, although the editor itself is not in my public repo, the layouts and their handling is.

Sorry, but not sure when will I have the time to publish the editor module (it is part of my private dev branch, and I won’t release that whole mess for sure), I wanted to make a separate repo from it that is more like a library for devs, but didn’t go through with it, also thought might be more wise to see what changes in v2 as it relies on some non-api stuff, and in general the whole thing is a sort of dance-around the svg workflow which might not be a good path… and it begs for many features that I don’t think I want to spend time making.

The auto compile stuff is just some node.js script, doesnt do much, just watches files and makes modules and makes and runs Rack, most of the code is just to handle toggling these “features” in the terminal :d Guess I can share this more easily if you want as it doesn’t tie into a bunch of other code and needs little explanation to use.

But what you are talking about is a bit different, my editor thingy is more geared towards making new “static” modules with fixed elements. Creating different control layouts would require a lot of manual work as the number of ports/knobs etc is tied into code still, which it doesn’t generate automatically. The only thing it helps is to position pre-coded elements on a panel inside Rack. If you want to add/swap/remove new elements often, it is clunky to use. The mapping inside Rack functionality is another “special thing” your modules would need to handle. Maybe it would be easier to find some touch-osc like thing that lets you create general midi control surfaces, with customizable faders etc. That runs outside of Rack, or base something on Stoermelder’s modules.

Since modules need to have fixed number of ports/knobs/sliders it could work to make one that has many of those and can position or hide the unneeded ones based on its preset, so your control layouts would be different presets to the same module, instead of having different modules each with their own source code. This way you wouldn’t have to compile modules to create new control layouts, which feels like a must-have freedom to your idea.

1 Like

Regarding the node.js autocompile, thanks a lot for offering to share it! Given that it’s this simple, I can probably easily cook it up if I go down the recompilation route for a first implementation. However, I agree it would be great to avoid recompilation for new controls altogether. The svg → compile → relaunch → repeat workflow could do for a minimal implementation but is likely to get annoying real quick.

Making a module with loads of spare controls, moving, and selectively hiding, as you suggested, is a neat alternative idea to go about it.

Thank you so much for all the details, cheers!