Unofficial binaries are risky - should we keep posting them?

Development
21

Sapphire official releases now working

That is wonderful, thanks @Bloodbat!

One more change: I updated my GitHub Actions yml script to create releases for tags that start with “v2.” (official releases) or “rc2.” (release candidates). I verified by creating a release for Sapphire v2.5.7.

To make this release on GitHub, I updated the yml file only. The C++ code was not changed and that same code was used to build the plugins for Sapphire 2.5.7 in the VCV Rack Library.

Fun stuff for nerds

But do you believe me? What if I’m trying to trick you into downloading a plugin that doesn’t match the open source code I claim it is? Let’s put on the paranoid security aluminum foil hat and see how to confirm/refute my claim about the code matching.

  • Visit the GitHub issue I use to send plugin integration requests to VCV Rack. Scroll to the bottom and find where I requested an update version 2.5.7. You will see the git commit hash I requested a build for. That shows you exactly what commit hash was built for the the VCV Rack Library. You will see that for version 2.5.7 I asked them to build hash f838c2364366c2b258cd63dd5ec5158237c96ed1.
  • But maybe I edited that comment on GitHub after VCV Rack built the code? Let’s find out. Visit the VCV Rack Library repo list. Look for the subproject called “CosineKitty-Sapphire”. Click that link and confirm that it takes you to https://github.com/cosinekitty/sapphire/tree/f838c2364366c2b258cd63dd5ec5158237c96ed1 . Confirm that the hash in this URL matches the one I requested in my assigned issue page. The hash in the URL is what the VCV Rack library most recently published for Sapphire.
  • Clone the sapphire repo from GitHub to your local machine.
  • Change into the directory where you cloned the repo. Execute: git diff v2.5.7 f838c2364366c2b258cd63dd5ec5158237c96ed1.
  • Confirm that the only changes were made to the file .github/workflows/build-plugin.yml. Confirm that no other files were changed.
  • If you are a developer, at your leisure, do a git checkout f838c2364366c2b258cd63dd5ec5158237c96ed1 and study the code. See if anything looks suspicious or dangerous!
  • Close the circle by going back to my GitHub release page we started with. You can see the tag linked from that page is v2.5.7, and verify its hash code (9ac911be09bf1c8960c39974d42c609d4a68ab42) matches the output in your cloned repo when you run git show 2.5.7.)

In my mind, this is proof that the same source code was compiled into the plugins by the VCV Rack maintainers. They used that git commit hash to build the code for their library. Anyone can audit the exact source code that matches that published plugin version to look for anything suspicious.

3 Likes
22

This is great stuff

Sadly it leaves me in a difficult position; if I agree with you that privately built binaries are a risk (and I do agree) then we should not post them or expect Rack users to use them.

But my plugin is closed source (due to employment contract reasons) so I cannot use this method to validate a binary. A user can only trust my plugin when it is officially released in the Rack library.

How can I conduct any sort of pre-release user testing in a safe way?

23

This is a great question, and I don’t think I have a good answer. I’m a big fan of your DanTModules plugin, especially Bright Idea and Purfenator.

I think it all boils down to trust, and each individual person’s judgment. I personally would feel fine beta testing your pre-release plugins because you have a long-term positive presence on this forum, and your plugin has been in the VCV Rack Library for years.

I definitely don’t want to throw shade at closed source plugins, because each developer has their own goals and strategies for what they are trying to accomplish. Although Sapphire will always be free and open source, it’s possible I might create a separate closed-source plugin someday. At that point I would run into the same situation.

A good option would be to solicit volunteers for testing here on the forum, but only provide the binary links through a more private channel such as private message on this forum, or by email. This makes the distribution a little more contained than a link that is visible to the whole world.

Alternatively, include some wording like “volunteer testers assume risk of using this software” so that there is some level of informed consent.

If nothing else, I hope this conversation raises awareness so that people at least think about security and are conscious of risks. Again, the vast majority of people are trustworthy, but if nothing else, if something feels suspicious, trust your gut and be careful!

2 Likes
24

Just for a totally different perspective, here’s what I used to do with Squinky Labs. It worked well.

I had a small number of beta testers who really tested something. I made them use Slack to communicated, and I sent the builds to slack channels.

Sure, it was impossible to know for certain what was in those builds, but in this case if the people trusted me they could trust the build.

Many people refused to get a free slack account and put up with all that. I understand, but that meant we couldn’t work together.

Anyway, it worked out quite well.

1 Like