Best Practice for storing large amounts of Data with a patch

Development
1

Just released a Delay/Looper, and the first feature-request is to save/load the content of the buffer between sessions. We’re talking about 2 minutes at 16 bit resolution

In theory I could save it to the “dataToJson”/“dataFromJson”, but I wonder if there is a better way?

2

You can save module specific assets outside of the json file in the module onSave event. The Fundamental WaveTable LFO does this.

1 Like
3

Yes. When done right the asset(s) gets included in the autosave dir and packaged up with the patch. Andrew gave an example in here long time ago but otherwise probably look at the VCV Wavetable VCO code.

Ah, here it is in Fundamental/WTVCO.cpp at v2 · VCVRack/Fundamental · GitHub

	void onAdd(const AddEvent& e) override {
		std::string path = system::join(getPatchStorageDirectory(), "wavetable.wav");
		// Silently fails
		wavetable.load(path);
	}

	void onSave(const SaveEvent& e) override {
		if (!wavetable.samples.empty()) {
			std::string path = system::join(createPatchStorageDirectory(), "wavetable.wav");
			wavetable.save(path);
		}
	}
3 Likes
4

Many thanks!

1 Like
5

This is about 25MB at 48kHz, so big but not absurdly big. Definitely wouldn’t suggest trying to shove it in a JSON file, especially since you’d have to encode it as valid JSON (eg, Base64, which would blow it up by 33%). Loading the patch would probably be very slow as well. Smart of VCV to provide the capability to create other files for a patch.

6

There are more space-efficient methods of storing audio than .wav, if space is a concern.

7

Yes, but now we’re using CPU which is a resource we frequently run out of with Rack.

8

Meh, I support flac in my SFZ player. All the decompression happens off the dsp thread, so the cpu hit is “fine”. Ant kind. Of compression done at load/save time shouldn’t hit the audio processing. I don’t think?

2 Likes
9

I’m not so much thinking of interference with the audio thread(s), I’m more thinking of the hit to total CPU usage of compressing 2 minutes of audio during the patch playing, and the ensuing heat generation, fan noise etc. But maybe it can be offloaded to a very low priority thread taking its time, if the audio doesn’t change often.

10

Its 8mb for “prince” and 16mb for “prism”… its just an fwrite/fread. Since its only on save/load I’m not concerned about cpu… if size becomes an issue i might look into compression…

I also might add an option in the menu for “retain buffer between sessions” for when you want to distribute a patch without the buffer…

11

Well, I have to admit I’m not crazy about the idea of storing sample data in the patch that way, but I don’t really know about the pros and cons…

12

Oh sure, I’m not concerned about writing that data either. My concern was about adding compression into the mix, and the ensuing CPU usage. I think you should just write the .wav and not think about compression. Our challenge in Rack is not disk I/O or disk usage, but mostly CPU usage.