I am also currently trying to move my work flow more towards the use of controllers. My ideal would be able to perform with no screen, keyboard or mouse, although I am not trying to patch from controllers.
I have been able to achieve most of what I have required using Stoermelder plugins, Midi-Cat, 8Face & Strip. The area I have yet to master is a step sequencer, while I have maped 2 X apc mini to a few trigger sequencers I have yet to get it working In a way I want, I have recently started to develop a module for this use, but I would expect it to be a few weeks away from a beta release.
As for prototyping I am not sure if the upcoming inclusion of PD into VCV prototype will be useful for this, but I am hopeful. I am currently looking at both the VCV core modules, and Stoemelder code to learn how to interface with Midi from within a module.
Using the Apc - mini I have found no requirements for SysEx, as all the pads can be read and written using standard NoteOn, NoteOff. I am unsure of other controllers you mention but am very tempted to purchase a launchpad to see what can be achieved.