Catch2 Unit testing

Development
1

While searching the forum for ‘unit testing,’ I found quite a few ideas and pointers, but no clear, step-by-step instructions. So, I’ll share the solution I came up with.

It’s a minor patch that adds Catch2 as a submodule to Rack and an extra command-line option, which finds all tests inside the loaded plugins and runs them.

To get this running download this patch to your development rack dir: Catch2_patch.txt (updated April 27)

cd your_dev_rack_dir
git submodule add -f https://github.com/catchorg/Catch2.git dep/Catch2
git apply Catch2_patch.txt
make dep
make

And you can start running tests like so:

./Rack -d -r
./Rack -d --runtests='-?'
./Rack -d --runtests='--rng-seed=234 --order=decl [Shared]'

Which generates output like:

[2.536 info src/tests/runtests.cpp:29 runTests] Running tests with arguments --rng-seed=234 --order=decl [Shared]
Filters: [Shared]
Randomness seeded to: 234
==============================================================================
All tests passed (92 assertions in 3 test cases)

On the plugin side:

Download and add these files from the Catch2 project to your own project:

And start writing tests in your plugin:

#include "catch_amalgamated.hpp"
#include "ClockTracker.hpp"

TEST_CASE("ClockTracker", "[Shared][ClockTracker][lowlevel]") {
  SECTION("isPeriodDetected") {
    REQUIRE(...)
    ...
    etc.

Only tested this on Linux using version 241 and 251.

1 Like
2

That looks cool. Years and years ago I published this about how I unit test VCV plugins: SquinkyVCV-main/docs/unit-test.md at master · kockie69/SquinkyVCV-main · GitHub

But your way seems better. I guess one reason I did mine like I did is that I wanted to debug in Visual Studio, but build with gcc. whatever?

Does your stuff require g-test? the macros look kind of gtest-like.

very nice!

3

Funny you’d ask. It was the fact that I kept running into your composites that I started searching for testing methods. That, and let’s be honest here, I’m quite fed up with running into my own numerous bugs, often without a clue where to start.

In an old thread, you asked for a way to link to Rack for testing purposes (before v2). @Vortico responded: “…you want to link to Rack with your own executable with its own entry point (main()) but not call Racks functions.”

This triggered the thought to use Rack’s own entry point. The upshot is that all of the Rack API is available and running. The tests run after the scene is loaded, so maybe you could even grab a fully patched module instance and use that for testing.

To answer your question: It’s not gtest, it’s Catch2

I picked Catch2 over gtest (and others) for several reasons.

One of them is the fact that it is already used by devs like @Baconpaul, @qno, @valley.audio to name a few.

Some people over at JUCE use Catch2. Here’s an example of a a sexy custom Matcher for audio blocks:

REQUIRE_THAT(myAudioBlock, isEqualTo (someOtherBlock))

with expansion:

Block is 1 channel, 480 samples, min -0.766242, max 0.289615, 100% filled

[0—⎻—x—⎼⎽_⎽⎼—]

is equal to

Block is 1 channel, 480 samples, min -1, max 1, 100% filled

[0—⎻⎺‾⎺⎻—x—⎼⎽_⎽⎼—]

That and some other reasons (learning curve, features, little boilerplate, GENERATORS()) made me pick Catch2.

I’m at the beginning of learning TDD, BDD, Unit testing, and its idiosyncrasies. But I figured if I shared this hack, perhaps others adopt/improve it and implement tests in their plugins I could learn from.

WARNING: These green lines yelling “All tests passed” at you, with ever-growing numbers, can become addictive to our monkey brains. Get out while you still can.

1 Like
4

Catch2 is great and we use it for all our tests. Integrates wonderfully with CLion and vscode too

5

I tried TDD for the first time more or less a a joke - it sounded so crazy. This was at a company in 2006 that didn’t do much testing. I was surprised, and it worked great. Since then I’ve done all my home programming that way. As you obviously know, it’s great for plugins where its pretty easy to test everything. Unlike with a big application where it can be challenging.

At work not we are required to have good test coverage, but still I’d say most ppl don’t do TDD. I do it at work every now and then, but usually there I do the tests after.

Your approach sounds very good. I would probably use it if I were starting over.

Just curious on one thing - can you do that block matching in the frequency domain? Back when I was doing filters and oscillators I spent a ton of time setting up tests to test those things in the frequency domain. Time domain wouldn’t be super useful.

1 Like
6

Yes, there are a few test helpers that operate in the frequency domain in that link I posted. Here’s the code.