Which plugin (if any) is crashing Rack 2.1.0 standalone patch?

@baconpaul

Does anyone know if Surge does crash Rack these days? I have been working on trying to track down what is crashing Rack 2.1.0 Standalone in several of my patches. This morning I removed the Surge Rotary module that I had added when I created this patch series a few weeks ago. So far I cannot induce a crash with this patch now. It is still too early to tell but, after I found this GitHub issue from January that is still open and no apparent plugin updates since then, I thought I would ask here if anyone has had problems with Surge under V2 and/or is still having problems.

The plugin is still in the library.

I use the rotary in every patch as a send effect for bassy stuff. Never an issue. This is Rack 2.0.6, as 2.1.0 was freaking out all of the sudden with no apparent reason, 2.0.6 restores sanity :wink: , but hence tested on both and crashing it hasn’t been it’s main issue ;). Also this is MAcOS 12.3.1. I get a crash log for Rack if it ever does go south, which is rare. Mostly on startup and then second try works fine.

1 Like

Well, I did just get a crash without Surge in the patch, so that may not be the problem. My symptoms are the same as yours that when I do get a crash it is usually at startup and then the 2nd try works fine. It is tedious to track down because by the time that Rack tells me that “Rack crashed during the last session…”, Rack has already started the new log file and it almost always ends at:

[4.631 info src/network.cpp:147 requestJson] Requesting JSON GET https://api.vcvrack.com/version?edition=Pro
[5.000 info src/network.cpp:147 requestJson] Requesting JSON GET https://api.vcvrack.com/user
[5.269 info src/network.cpp:147 requestJson] Requesting JSON GET https://api.vcvrack.com/library/manifests?version=2
[5.726 info src/network.cpp:147 requestJson] Requesting JSON GET https://api.vcvrack.com/modules

But that must be the point at which Rack reports that a crash occurred in the previous session. So, I never see the log for the bad session. I have been copying the log to a temporary name file before I exit Rack and again before I try to launch Rack, but only the first time after exiting. Sometimes I have to load, exit and load several times before the crash and it is not clear why, but I miss the crash log if a crash occurs. This is why I posted a question here about the JSON lines above yesterday, thinking that this was a crash log, but it was not.

I have seen similar behaviour but not with the Surge modules. I get a crash which involves the Squinky Comp II and RPJ modules and it sounds suspiciously similar to yours.

2 Likes

Interesting, I do have the SquinkyLabs plugin Organ 3 module in this patch, several instances in fact, as well as the Squinkytronix plugin Harmony and Arpeggiator test releases. This gives a new direction to look. I’m already working with Squinky since I thought the crash was related to Arpeggiator and it may be in some way that the two Squinky plugins are… interacting(?).

Thanks for posting. I will probably rename this topic to “Which plugin is crashing Rack”.

1 Like

Yes, there is definitely an interaction across plugins which makes it really difficult to narrow down. It does sound like it might be the same thing we’re experiencing.

1 Like

I finally captured a crash log if anyone would like to look at it and see if you can understand what caused the crash. There is a stack trace but no module implicated. log.txt (74.0 KB)

It was really difficult to catch this log since the error would be reported on startup and most often the log would be overwritten before I knew it had crashed. The actual crash occurs on exit.

I can upload the patch file if anyone needs to see it.

1 Like

The occasional startup crash seems to always be my self-compiled Annuli, with Rack trying to step it before it was all ready and fully allocated by the looks of it. Maybe the order of startup steps can change and cause this somehow? In any case, trying 2nd time always opens it fine and it functions fine. Oh, one was the AI Resonator too, so not just Annuli, lends more beef to my theory of startup order of steps that sometimes steps a module before it’s ready on start loading from autosave…

Interesting. It is still confusing to me as to when Rack is crashing. I almost always see it at startup if it occurs. But in my case, it appears that the crash actually occurs on exit, but it could be that it is on exit from a crashed start. In the log I posted, I had started Rack and almost immediately terminated it. As a result, the log shows successful startup (I think) but there are no auto save lines as those only occur every 15 seconds or so and I did not give it enough time to do one auto save. That has turned out to be the best way to cause a crash but it may only happen 1 time out of 10 to 20 times. That might fit in with the observations you have made and I might be terminating Rack before the start finishes initializing. There have definitely been some crashes that occurred on exit and notified on exit and even a few that crashed during running, usually after some mouse cursor interaction with some module knob, but no pattern to a specific module.

Maybe I am making too much out of these crashes. My primary interest was in making sure that it is not my Meander plugin/module that is causing the crash. As you say, a 2nd start try always succeeds and nothing is corrupted.

What are other’s thoughts? If the log never implicates a plugin, should we assume that the crash was not caused by a plugin but rather something in Rack itself?

This question may deserve its own topic, but since it is intimately related to this current topic, I will ask it anyway.

Can anyone give a very brief overview about how auto save works?

I ask because the longer I run my test patch, the more likely I am to get a crash on exit or on the next start (and it is hard to catch which). So, the log shows auto save saving every 15 seconds. Where is it auto saving to? Is it \autosave\patch.json ?

How is the auto save information used? At the next startup, does Rack use the auto save file(s) to reconcile the patch if any changes were saved? Does it only do this in memory such that one still has to explicitly save the patch in order to save the reconciled patch info?

Does it make sense that for a long running patch, say 30 minutes, where there are 120 auto saves logged, and Rack is exited without saving the patch, will the next launch have to process 120 auto save entries and perhaps take so long that the Rack starts stepping before the reconcile completes and results in a crash?

Thanks for any wisdom anyone has about auto save.

Yes, autosave occurs every x seconds as per setting in settings.json. And Rack uses that autosave saved before exit to open when starting it cold. Always been like that. I am not sure the autosave is entirely to blame, as it could be that it tries to free a resource already deallocated, again due to how it shuts down. So same thing in reverse methinks… I have never had a failure to open a patch, just needed 2 times. And I only remember one crash on exit, but that was still on windows 10, not since moving to MacOS again. so mileage on different platforms may of course differ

First of all, when discussing crashes it’s always super important to say whether it’s Rack stand-alone or VST/DAW/Pro. Secondly it’s very important to say whether it’s crashing the browser or a patch.

For what it’s worth Surge is not crashing my Rack Standalone 2.1.0 on Mac, not in the bowser and not when adding a Rotary to a patch.

Fair enough. I edited the topic title to reflect the type of crash.

Thanks. I also do not think it is crashing my Rack Standalone 2.1.0 patch on Win. I was just going by the open Github issue for Surge opened by a moderator. That’s why I asked.

Is it correct to say that autosave saves the entire patch.json file every 15 seconds rather than any type of incremental save that would need to be reconciled at the next load?

correct, overwrites the autosave every save

1 Like

If you find anything that points to any of the squinklab modules let me know. Preferably create an issue in github if possible.

I’ve done a ton of testing in the last few days running the Linux version in under address sanitizer and with valgrind. Found nothing super interesting, and no hits at all on anything with squinky in the name.

I see a lot of plugin UIs trigger valgrind to say “branching on uninitialized memory” - these are all on startup. I do also see VCV recorder accessing uninitialized memory in the process call when Rack closes (calling a log10 function). of course any of these could be benign or false positive.

I did forward a couple of semi-interesting valgrind hits to Andrew.

I also run on windows in gdb so I can get a good stack if something bad happens, but it never does (for me).

I did have a 100% repro crash if I turned off my USB audio interface while rack was using WSAPI, but that seems fixed in 2.1.0.

Lastly, I run all my squinkytronix unit tests with Microsoft’s port of address sanitizer for windows. It never complains either.

Something appears to be wrong, but there’s not much info to go on, or even a clear suspect.

2 Likes

It is really impossible to blame the crashes on any plugin or module when the crash log never implicates any module in the stack trace back. All I can do is consider every module in my patch and see if I can narrow it down by taking modules out and messing around until I get a crash even without that module, etc.

From my perspective, it appears that the crash is most likely not due to a module in the patch. it is very easy in this trial and error attempts to make the crash come and go to get false negatives and false positives since it is so inconsistent how many times I have to launch a patch, run it for a while, exit and relaunch, trying to look at the log file between each step in order to see a crash before it gets overwritten at the same time that a launch tells me Rack has crashed on the previous session.

I think I’m also going to pause this investigation. Perhaps will will get some feedback on the debug info Squinky sent to VCV.

1 Like

That makes it very difficult to interpret my patch crashes as related to autosave. Autosave was probably a false suspect caused by the Monday Win11 update that had some major hiccups leading me to have to delete the entire autosave folder in order to get running again.