Save/restore of module with multiple audio interfaces on MacOS

Development
1

I recently added a second audio interface to the Zoxnoxious synth module ( Using multiple audio interfaces ). The module now uses two objects derived from audio::Port. Functionally no issues on Linux and MacOS, I’m able to set both interfaces and a happy camper with playback.

What is not working is JSON save/restore of the objects on MacOS. This did work fine with a single interface, on patch load the single audio port is set. With two audio::Port objects save/restore also works fine in Linux. The issue is MacOS: with two audio::Port objects they do not get set on restore.

I took a look at the JSON between the two and am thinking it might be MacOS having the same name for the devices.

Here’s the relevant JSON on Linux:

        "audioPort0": {
          "driver": 1,
          "deviceName": "hw:Zoxnoxious MIDI and Audio,0",
          "sampleRate": 4000.0,
          "blockSize": 256,
          "inputOffset": 0,
          "outputOffset": 0
        },
        "audioPort1": {
          "driver": 1,
          "deviceName": "hw:Zoxnoxious MIDI and Audio,1",
          "sampleRate": 4000.0,
          "blockSize": 256,
          "inputOffset": 0,
          "outputOffset": 0
        }

On MacOS, I get the following:

        "audioPort0": {
          "driver": 5,
          "deviceName": "Playback Inactive",
          "sampleRate": 4000.0,
          "blockSize": 256,
          "inputOffset": 0,
          "outputOffset": 0
        },
        "audioPort1": {
          "driver": 5,
          "deviceName": "Playback Inactive",
          "sampleRate": 4000.0,
          "blockSize": 256,
          "inputOffset": 0,
          "outputOffset": 0
        }
      }

Note the non-unique deviceNames.

I took a look at audio.cpp and Port::fromJson does rely on string matching of deviceName:

	json_t* deviceNameJ = json_object_get(rootJ, "deviceName");
	if (deviceNameJ) {
		std::string deviceName = json_string_value(deviceNameJ);
		// Search for device ID with equal name
		for (int deviceId : getDeviceIds()) {
			std::string deviceNameCurr = getDeviceName(deviceId);
			if (deviceNameCurr == "")
				continue;
			if (deviceNameCurr == deviceName) {
				setDeviceId(deviceId);
				break;
			}
		}
	}

I’m wondering what the options are at this point. Can MacOS provide unique deviceNames? Or maybe something in Rack on getDeviceIds() could help?

2

Similar issues occur with MIDI devices. RTAUDIO and RTMIDI (the x-plat libraries used by Rack) use the same port management and identification, which is flawed. On all OSes there are more unique identifiers available that can be used, but they are not human-friendly, so you’d need to match based on os id, but show users the friendlier name. This still has issues because device manufacturers are not required to provide unique device names/identifiers, and often don’t because that costs more.

Not sure why the RT* APIs “unique-ify” on some platforms and not on others. Also, the device ids reported by the api aren’t reliable. They change depending on what devices are connected at any given moment, and the rt* apis will give different numbers for the same device if an earlier device has been added or removed - even during the same Rack instance. If you add or remove devices, Rack’s port support easily gets confused, as shown in this screen snip:

Confused MIDI-CV

So, I don’t use Rack’s port serialization.

The strategy I’ve adopted is to use a combination of the device name and the order (first, second) in which devices with the same name appear in the list of devices. While this isn’t perfect, it is more robust than using just the name or using the name in conjunction with the (volatile) RT* API device ids.

1 Like
3

I can see value there, if I knew how to find the device name. Given my module is looking for specific devices / my custom hardware, it’d actually be a benefit to the user to have the module find it. It’s kind of annoying to force the user to select a device when the module should already know what to look for, if I knew how to look for it. Do you have any code pointers on examples? I may dig into that tonight if I get a chance.

4

When I’m at my computer tomorrow, I’ll post a link to my repo where my code resides.

The device identification is tied into my plugin’s extended-extender protocol where I don’t use Rack’s adjacency-based extender protocol to connect related modules. I base it on identifying the midi devices I’m controlling.

1 Like
5

<breaking from the primary issue> I have a lot to learn and this sounds like a great model to review. I’m using the default extender mode: all modules send DAC values to a right-most module which serves as the primary. It’s all daisy chained, so break the chain and modules stop sending DAC values. The synth, or the particular cards installed, stop responding.

6

My MIDI device binding is in this header file and the .cpp file next to it:

pachde-hc-one/src/em_device.hpp at main · Paul-Dempsey/pachde-hc-one (github.com)

This is based on forming a string id (termed a “claim” in this code) naming the driver, device, and order-of-enumeration-of-same-named-devices. This string is what is serialized to identify the device that the module is bound to. There is a central plugin-wide singleton broker that lets an instance of the primary module to claim a midi device. This claim is also how the secondary modules find their primary, using a similar plugin singleton broker (see module_broker.hpp).

In my case, the module works with input/output pairs, with logic to find both end-points that is specific to the known Eagan Matrix devices out there. The code is reasonably generic and EM-specific parts should be easy to remove/adapt.

There’s also a custom ui widget for selecting midi devices. (widgets/em_picker.hpp).

This plugin doesn’t do any significant DSP: it’s all about interfacing with an external MIDI device. The current state of the code is quite loose about thread-safety, so it’s not a good example of inter-module communication. For DSP in linked modules, you may have few good options other than the Rack extender message passing mechanism. I haven’t investigated if the Rack message passing could be adapted to how I link related modules in this plugin.

7

Thanks for the great info and source @pachde. I’ll have to dive into this to really get it; seems like a viable solution.

8

Not completely sure yet ;-). Fixing a threading issue now in the module brokering right now. Need to find a safe place to hook up the modules before process() starts running — but the general idea is viable.

The persistence and hooking up of midi devices is working well.

There are still issues with rtmidi. I just opened one with VCV (with a suggested fix) where I’m occasionally crashing in outbound midi dispatch where it’s attempting to send a midi message that has been moved. I also see deadlocks in the rtmidi thread after midi devices are plugged in or unplugged, and it doesn’t have to even be a device you’re currently using. Rack will hang with an un-killable Rack.exe, but just unplug midi devices and Rack wakes up.

9

ok, it’s the best solution until another one comes along :slight_smile:

10

Yeah, I’ve been seing that since forever.

11

I did find a crashing bug in the Rack wrapper around RTMidi, reported to VCV, with a proposed fix. Hopefully, the fix will be in the next Rack update.

1 Like
12

For those of you curious about the RTMIDI problem:

<rack>/src/rtmidi.cpp:

void runThread() {
		system::setThreadName("RtMidi output");

		std::unique_lock<decltype(mutex)> lock(mutex);
		while (!stopped) {
			if (messageQueue.empty()) {
				// No messages. Wait on the CV to be notified.
				cv.wait(lock);
			}
			else {
				// Get earliest message
				const MessageSchedule& ms = messageQueue.top();
				double duration = ms.timestamp - system::getTime();

				// If we need to wait, release the lock and wait for the timeout, or if the CV is notified.
				// This correctly handles MIDI messages with no timestamp, because duration will be NAN.
				if (duration > 0) {
					if (cv.wait_for(lock, std::chrono::duration<double>(duration)) != std::cv_status::timeout)
						continue;
				}

				// Send and remove from queue
				sendMessageNow(ms.message);
				messageQueue.pop();
			}
		}
	}

The problem is that this code takes a reference to the top item in the message queue, then waits for a timeout (which unlocks the lock). While waiting, other threads can post a message to the queue, which can cause the data in the queue to be reallocated, which makes the reference invalid. It now points to freed memory or memory used for something else.

When the timeout wait is signalled and this thread resumes, it is possible for the timeout condition to be satisfied, skipping the continue. In which case it passes the (invalid) message to sendMessageNow, which ends up a couple of functions deeper in code that dereferences the invalid pointer and crashing with a seg fault. If the queue data has not been reallocated, the reference may now be pointing to a different message which has the potential for the less disastrous consequence of messages being sent out of order.

A fix is to always continue after the timeout wait, so that the top queued item is re-acquired and used inside the lock so that the data is guaranteed to be valid.

				if (duration > 0) {
					cv.wait_for(lock, std::chrono::duration<double>(duration));
					continue;
				}

If nothing has changed the queue in the meantime and top is still the same message, when the duration is recalculated and the timeout has expired, duration will be < 0 and it is then sent immediately without any additional waiting.

Now, this is extremely timing-sensitive, so occurs relatively rarely. I was lucky enough to catch this in the debugger at a moment when my brain was un-fogged enough to see the problem. I had been working on and off for weeks trying to catch this bug in the act. I had the same crash under the debugger several other times but had been focused on trying to find a problem in my code.

2 Likes
13

Good catch!