SVG panel designer (with React) and NanoVG draw hot code reloading

I know there is Inkscape and other great vector tools. I took a slightly different approach.

I like to code and web is one of my many homes. So I used React to create an environment where I code my UI. This can and most likely will be extended to a full editing experience. The window on the right with the Rack Module is Chrome, Just running this on macos in splitview without Toolbar.

Would there be interest in something like this:

This allows to create interactive components. SVG if you take some time is not so hard to unterstand. You can till create components in vector tools and export to svg then import into your code.

This may be something for users who are more comfortable with code than with design tools. You can even import the ComponentLibrary, but I didn’t do that, create your own components or use a Free one like RUIL: Design Library for VCV Rack Modules

The red vertical lines are the rack units. The grey thick lines are 1cm, thin lines are 1mm. The display above is the current cursor position in mm when you hove the mouse inside the Rack Module.

The copy button copies the svg code and you can paste it into a svg file (download could also be implemented). text objects are convert with the commandline using inkscape in headless mode.

I didn’t implement a pan/zoom yet (matrix transform on the svg), but I have code I could use for this. Just using browserzoom right now.

Another Idea for this would be to create Interactive manuals

20 Likes

Looks interesting Rainer…

Thanks.

I’ve improved the workflow a bit, albeit I haven’t been able to configure automatic import of the svgs like I want them to have. But as workaround I convert the svgs to react components (which is roughly the same as svg +/- some syntax differences).

create-react-app is capable of doing this already. The problem is: I have to have an svg element in the svg file all the time, but I just want to use fragements like path or rect or g. In or to be able to modifiy the position i need to pass this file some attributes, but this is exactly what is failing.

I had a look into @svgr/webpack which is used under the hood by create-react-app but haven’t been successfull.

So what I did in the meantime is using svgr as a cli tool to convert the components.

Just in case here is a webpack expert on the board. Maybe he can help out with this.

This step is necessary in order to position the components, otherwise I can’t set cx,cy or the transform method on a group.

Nice touch to it: svgr uses svgo to optimize the SVG, but this also means you should always position your components at 0,0 otherwise svgo is optimizing the transform into a path … On the other hand you can deactivate optimization. Anyway.

TLDR;

If someone need want’s to colab or just doesn’t want to spent time on UI write me a message.

This allows me now to easily compose whatever parts of the UI is in different modules in almost no time.

1 Like

Proper

Just in case someone needs a reminder, how easy it is to work with SVG directly:

https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths.

I just extended the development experience with hot reloading for the draw function of the widgets. Now you can prototype your nanovg code very quick. I probably end up integrating this two tools.

For this I used https://github.com/mitsuba-renderer/nanogui/ and a package called reloadr that hot swaps functions or classes on save. This approach should be fairly save here, because I extracted the draw function, and just calling it on every redraw. So swapping this function out shouldn’t show side effects as long as you just draw and don’t do nasty stuff with it.

What does it mean? Quite easy: You change your code, save it (or let auto save do it, like in my case) and you get an instant change/draw of your code. I think this will very usefull, for scatching/testing/prototyping.

Maybe this is something we could even integrate into rack. I’ll have a look at whether I can create a plugin, that runs a python runtime and integrate it with rack. Maybe a custom build/fork … it that’s something Andrew can agree with. Let’s see how far I can get it.

Could be a good extension, or companion to https://vcvrack.com/Prototype. Or maybe Prototype could have a second modul with nanovg bindings to let you draw your own UI in Rack, that this would solutions would be obsolete.

Maybe it even would allow performant code in Combination with cython to prototype/develop full blown prototypes with the same or nearly the same performance (to be verified). https://notes-on-cython.readthedocs.io/en/latest/std_dev.html

I don’t know how far I’ll dig into this c/python cython thing, but if there will be some result I’ll let you know.

Now for the setup:

git clone git@github.com:mitsuba-renderer/nanogui.git
cd nanogui
pipenv install --editable .
pipenv install reloader

I’m using pipenv because I like to keep my env clean. But feel free to use pip or directly install into your system with pip and without any virutalenv.

Here is the rack test chamber:

import gc

import nanogui
from nanogui import Color, Screen, Widget, nanovg
from nanogui.nanovg import RGB, RGBA, NVGsolidity
from reloadr import autoreload

RACK_GRID_WIDTH = 15
RACK_GRID_HEIGHT = 380

def draw_rail(ctx, size, theme):
    hole_radius = 4.0
    rail_height = 15
    ctx.BeginPath()
    ctx.Rect(0.0, 0.0, 1000, 1000)
    ctx.FillColor(RGB(*theme['background']))
    ctx.Fill()

    for y in range(0, size.y, RACK_GRID_HEIGHT):
            ctx.FillColor(RGB(*theme['rail']['fill']))
            ctx.StrokeWidth(1.0)
            ctx.StrokeColor(RGB(*theme['rail']['stroke']))
            # Top rail
            ctx.BeginPath()
            ctx.Rect(0, y, size.x, rail_height)
        
            for x in range(0, size.x, RACK_GRID_WIDTH):
                ctx.Circle(x + RACK_GRID_WIDTH / 2, y + rail_height / 2, hole_radius)
                ctx.PathWinding(NVGsolidity.HOLE)
        
            ctx.Fill()

            ctx.BeginPath()
            ctx.MoveTo(0, y + rail_height )
            ctx.LineTo(size.x, y + rail_height)
            ctx.Stroke()

            # Bottom rail
            ctx.BeginPath()
            ctx.Rect(0, y + RACK_GRID_HEIGHT - rail_height, size.x, rail_height)
            for x in range(0, size.x, RACK_GRID_WIDTH):
                ctx.Circle(x + RACK_GRID_WIDTH / 2, y + RACK_GRID_HEIGHT - rail_height + rail_height / 2, hole_radius)
                ctx.PathWinding(NVGsolidity.HOLE)
            
            ctx.Fill()
            ctx.BeginPath()
            ctx.MoveTo(0, y + RACK_GRID_HEIGHT - 0.5)
            ctx.LineTo(size.x, y + RACK_GRID_HEIGHT - 0.5)
            ctx.Stroke()


@autoreload
def draw(rack, ctx):
    try:
        ctx.Scale(2,2)
        ctx.Translate(0,-RACK_GRID_HEIGHT/1.25)
        size=rack.size()
        light = {
            "background": (0x30, 0x30, 0x30),
            "rail": {
                "fill": (0xc9, 0xc9, 0xc9),
                "stroke": (0x9d, 0x9f, 0xa2)
            }
        }
        dark = {
            'background': (0x13, 0x13, 0x13),
            'rail': {
                "fill": (0x25, 0x25, 0x25),
                "stroke": (0x00, 0x00, 0x00)
            }
        }
        theme = light

        draw_rail(ctx, size, theme)

        # Panel
        rack_units = 10
        margin=2
        ctx.BeginPath()
        ctx.Rect(RACK_GRID_WIDTH*margin, RACK_GRID_HEIGHT, RACK_GRID_WIDTH*rack_units, RACK_GRID_HEIGHT)
        ctx.FillColor(RGBA(0xff,0x05,0x17,0x40))
        ctx.Fill()
    except expression:
        pass


class Rack(Screen):
    def draw(self, ctx):
        draw(self, ctx)



if __name__ == "__main__":
    nanogui.init()
    # create a fixed size screen with one window
    rack = Rack((800, 600), "Rack Test Chamber", resizable=True)
    rack.set_background(Color(0x13, 0x13, 0x13, 0xff))
    rack.draw_all()
    rack.set_visible(True)
    nanogui.mainloop(refresh=1 / 60.0 * 1000)
    del rack
    gc.collect()
    nanogui.shutdown()

Run the script: python rack.py and change the draw.

UPDATE: Ported the rack rail code and added some exception handling so the client doesn’t crash instantly when having a syntax error. Colors are adapted for my dark theme, but original theme is available. But feel free to use the original values

3 Likes

Good stuff!

It would be nice to have a built in scripting language for rendering in Rack! I dont use svgs and I wrapped nanovg in a way that makes writing draw code less verbose and have an editor module that lets me arrange ports on other modules and save the layout to be used by them but its very incomplete and messy at this point… it is all inside rack though.

For live reload I use a script that runs make and opens/closes rack on file changes. So it’s only as live as much it takes to build the plugin.

It’s interesting what you’re exploring here and I’m following this thread to see where the cython stuff goes.

In other news, Rack 3.0 might use another rendering lib.

I’ve been reading this news. And I think it will not happen in 2021 … so given we don’t even have v2 right now, I’m relaxed speding time on this topic. Having skia would be nice, so then I probably would use Flutter for sketching and drawing as its built ontop of skia and has hot code reloading already. Works like a charm. Or just use one of the many bindings available. Probably it would probably work exactly like the current solution. But I don’t think V3 will drop the current widget API, so I guess there will be a shim layer/emulation layer for plugins. And than later may deprecate this layer. But this is just gut feeling and guessing.

Is your module code checked in at github or some other location. I might have a look at it.

I also been thinking about the in-rack-approach. Probably the best solution I could come up with is to have one of the scripting engines already available in Prototype have the widget api exposed. So the idea is to have a module that exutes a draw.py or draw.lua … draw.*** you name it and pass the nvg context to it. To have hot code reloading the module would have a udp server and on the command linethere is a file watch script sending the script to rack when there is a change notification. I just wanted to have a scratchpad in the first place for learning nanovg and having fast edit continue cycles. And the first shot was already the best solution I could come up with. So I didn’t look further.

The translation is straight forward. You can even wirte the code without the pyhton naming conventions.

so in the python binding its ctx.LineTo in rack it is nvgLineTo. So translating your code back or forth between python an C is a matter of seconds/minutes.

Haven’t uploaded the editor module to anywhere yet, as it is not quite there to be used generally (and I have no released modules built with it) but what it does basically is to attach to a module by covering it with a layer and convert mouse click/drag events into selecting and moving widgets, while continously exporting the layout (a json file containing ids of params/ports paired with position data). I’ll post it when I get around cleaning it up. Still has some major problems with how it handles custom widgets that arent ports or parameters.

The problem I see with this is if I want to change something later, I have to go through manually translating back again or stick with cpp. Same as with the js prototyping. Personally, I’d rather not use two different langs or environments for prototyping and making actual modules, as these two phases aren’t distinct enough in my practice and I’m afraid it would result even more of my modules getting stuck permanently in the prototype section. I’d like to see python to be used in the rendering of non-prototype modules, or to have hot reloading for rendering bits in any way. Still, having tools like this can be useful for a lot of people, and can pave the way for future solutions, so don’t take my words as discouragement!

I see your point. And understand … So I’ve been extending my experiements and research. So I have a couple more options.

Drawing on a HTML5 Canvas. The API of NanoVG is modeled after the Canvas API, so there is probably no learning Curve at all. This is already working in the designer module. With animation and everything. Follow up would be to save the canvas as SVG. Which is also possible. There are different solutions for this working and I will just experiment a bit more to figure out a good workflow. Which kind of also makes the python NanoVG Draw unnecessary. I still quite like it and keep it, because of the instand changes it just takes something like between 200 and 600 ms roughly estimated to swap the code. So it’s instant. Another Idea would be to have a designer completely written in Rack or with just with the python solution I have right now. I also started to experiment with skia-python which looks promising. Skia has an experimental SVG render which translates all drawing code to svg, so I can store it as svg. (https://github.com/kyamagu/skia-python/blob/d7a3adb7a6e9cb1a92a59cb3f75c503a8259ba6f/notebooks/Canvas-Creation.ipynb) This also work s with the HTML5 Canvas. Just lets see where I get from here.

If I could whish for what I really wnat is a complete Designer Experience within Rack, also with DSP modules available so that the designer could compile an extension. Something like a visual programming Environment. Like in Unreal Editor. Something like this https://github.com/jagenjo/litegraph.js Still exploring… Here is another idea to include into the designer … which is kind of similar: https://github.com/mattdesl/canvas-sketch

And another option I’m exploring is konva.js where I can also generate SVGs from different layers. https://konvajs.org/docs/data_and_serialization/Serialize_a_Stage.html

I think the first step towards having an editor within Rack is to standardize a module layout format. This is what I’ve been poking at. Svg is good for draw data, but it might makes sense to have a format that is specific to modules, ie it would have a concept of parameters, input/output ports and lights, the rack grid size, labels and custom widgets, and could contain relations like labels paired with ports, groups of knobs, a line connecting two ports etc.

Such format could be used alongside or without svgs, but the point is to have the layout be the source of the panel instead of the graphics be the source of the layout, if that makes sense x)

Yes it makes sense. I already been searching for this kind of data, when I started. For example a json file. that describes the complete layout of the modul with,parameter ids, labels and positions, svgs that are used as assets, or to cut it short: A parsable, declarative file with all information describing the module.