By “instability” I assume you mean “sporadic crashes”.
There is a log always (not just on crashes). user folder/log.txt. The log doesn’t necessarily capture the offending plugin, as some plugin offenses causes crashes in Rack or other plugins.
Some plugins are written better than others, or with different stability goals.
If you are using my 8FACE mk2 module, there is an unfortunate bug in 2.4.0. Please update manually until the fixed version is arriving in the VCV Library.
In general I don’t think the plugin format has a lot to do with “stability”. I think in general it’s more about the reliability of individual plugins. Rack itself is extremely rarely the culprit for a crash. Crash from a plugin is far more likely.
Rack doesn’t have process isolation for plugins (for performance) so if a plugin crashes, so does Rack.
If you’re seing a stability issue, the questions is almost immediatly a question of which plugins you’re using, and how you’re using them. Some are rock solid, and some are finicky or just not very good in some scenario.
Lot’s of help here if you have a specific patch and the Rack log if you have a crash.
If the issue is a crash loading a patch and no other traces in a log, the last loaded is the first likely candidate, but not always the plugin at issue. A plugin can do stuff that doesn’t crash immediately in that plugin but causes others to crash.
While I do think it’s generally true that the plugin formats generally shouldn’t differ much, I have seen freezes in the CLAP in reaper that don’t occur in the other formats. I reported to VCV support at one point but I’m not sure if it’s been adressed or not. In what situation are you seeing the issue? For me the freezes happened when adding/removing a module.