Automated building and releasing Plugins on Github with Github Actions

Update 7: update the workflow actions to use the latest Rack-SDK version for Mac builds, see also this post.


Update 6: update the workflow actions to use Toolchain Docker image version 10, see also this post.


Update 5: update the workflow actions, added Mac ARM64 cross compile build job


Update 4: update the workflow actions to latest toolchain image version 6 changes.


Update 3: update the workflow actions to more recent versions (see also this post), update toolchain image version.


Update 2: for using the official Rack Plugin Toolchain example (Windows and Linux only) see this post.


Update 1: for Rack2 SDK example see this post.


Hello,

after I optimized the Azure pipeline definition for @Aria_Salvatrice, I decided to create an Github Actions Workflow for automated builds of VCV Rack plugins on Github. This is based on the Azure pipelines that are discussed mostly in this topic - Azure Pipelines for Github allows build of plugin

The setup of Github Actions is much simpler for a repository than to use 3rd party CI services (like Azure, Tracis-CI, etc.), as it requires no setup at all. All you have to do is create a worklow under .github/workflows inside the repository.

I created a demo repository from where you can copy and put the vcv-plugin-github-actions-example/.github/workflows/build-plugin.yml at main · qno/vcv-plugin-github-actions-example · GitHub into your own plugin repo and everything works than out of the box.

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-sdk-version: latest
  rack-plugin-toolchain-dir: /home/build/rack-plugin-toolchain

defaults:
  run:
    shell: bash

jobs:

  modify-plugin-version:
    name: Modify plugin version
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/cache@v3
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"

  build:
    name: ${{ matrix.platform }}
    needs: modify-plugin-version
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/qno/rack-plugin-toolchain-win-linux
      options: --user root
    strategy:
      matrix:
        platform: [win-x64, lin-x64]
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - uses: actions/cache@v3
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Build plugin
        run: |
          export PLUGIN_DIR=$GITHUB_WORKSPACE
          pushd ${{ env.rack-plugin-toolchain-dir }}
          make plugin-build-${{ matrix.platform }}
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          path: ${{ env.rack-plugin-toolchain-dir }}/plugin-build
          name: ${{ matrix.platform }}

  build-mac:
    name: mac
    needs: modify-plugin-version
    runs-on: macos-latest
    strategy:
      fail-fast: false
      matrix:
        platform: [x64, arm64]
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - uses: actions/cache@v3
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Get Rack-SDK
        run: |
          pushd $HOME
          wget -O Rack-SDK.zip https://vcvrack.com/downloads/Rack-SDK-${{ env.rack-sdk-version }}-mac-${{ matrix.platform }}.zip
          unzip Rack-SDK.zip
      - name: Build plugin
        run: |
          CROSS_COMPILE_TARGET_x64=x86_64-apple-darwin
          CROSS_COMPILE_TARGET_arm64=arm64-apple-darwin
          export RACK_DIR=$HOME/Rack-SDK
          export CROSS_COMPILE=$CROSS_COMPILE_TARGET_${{ matrix.platform }}
          make dep
          make dist
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          path: dist/*.vcvplugin
          name: mac-${{ matrix.platform }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs:  [build, build-mac]
    steps:
      - uses: actions/checkout@v3
      - uses: FranzDiebold/github-env-vars-action@v2
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.CI_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.CI_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: softprops/action-gh-release@v1
        with:
          tag_name: ${{ github.ref }}
          name: Release ${{ env.CI_REF_NAME }}
          body: |
            ${{ env.CI_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.CI_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v3
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true

The workflow does this:

Build plugin

  • On each push to Github and pull request a build pipeline is triggered that compiles the plugin for Windows, MacOS and Linux with the Makefile from Rack-SDK
  • Each pipeline stores the built plugin variants as artifacts that can be accessed via the Actions tab under the according workflow run
  • A pipeline build modifies the plugin version on the fly by prepending the short git commit hash to the version in plugin.json (e.g. version 1.2.3 becomes 1.2.3-4a94fba) Note: This is only done when the build is not triggered from a tag!
  • The plugin.mk file in the RACK-SDK gets patched to use 7zip on Windows instead if zip command

Creating a Release

To create an automated Release on the Github repository, a developer must create and push a tag to the repository. This follows the convention which is described at Plugin Manifest - VCV Rack Manual. A tag must be named vX.Y.Z, where X.Y.Z represents the current version in the plugin.json, e.g. v1.2.3.

If the build is triggered by tag then the workflow creates und publish a new Release based on the tag version. The built plugin variants are attached as assets to the release. The release job contains a check that verifies if the created tag matches the current plugin version in plugin.json If the version and tag are not matching the Release job gets canceled.

9 Likes

This is awesome, I just used it for GitHub - NikolaiVChr/Autinn: Autinn VCV Rack plugin

How do I adapt it to build Rack v2 plugins? The rack sdk has different name for each OS in v2. Like ends with: “2.beta.1-win.zip”

Add new variables into the matrix strategy section, and then use them in the ‘Get Rack SDK’ section

1 Like

Got it to work, thank you.

1 Like

The build script have started to fail with some of these:

E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/m/mesa/libegl-mesa0_21.0.3-0ubuntu0.3~20.04.4_amd64.deb 404 Not Found [IP: 52.147.219.192 80]

[42](https://github.com/NikolaiVChr/Autinn/runs/4421531032?check_suite_focus=true#step:7:42)Reading package lists...

[43](https://github.com/NikolaiVChr/Autinn/runs/4421531032?check_suite_focus=true#step:7:43)E: Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgl1-mesa-dev_21.0.3-0ubuntu0.3~20.04.4_amd64.deb 404 Not Found [IP: 52.147.219.192 80]

Do anyone know how to fix?

looks like that host isn’t finding package requested (libegl-mesa0_21.0.3-0ubuntu0.3~20.04.4_amd64.deb)

404 error is ‘Page not found’

I modified the ubuntu prepare to update first:

sudo apt-get update && sudo apt-get install -y libglu-dev

and that fixed a similar issue for me.

Hi,

thank you for pointing out the issues. I fixed them and updated the workflow definition at the example repository.

The updated workflow definition for the current Rack2 SDK:

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-sdk-version: 2.0.3

defaults:
  run:
    shell: bash

jobs:
  build:
    name: ${{ matrix.config.name }}
    runs-on: ${{ matrix.config.os }}
    strategy:
      matrix:
        config:
        - {
            name: Linux,
            sdk-platform: lin,
            os: ubuntu-latest,
            prepare-os: sudo apt update && sudo apt install -y libglu-dev
          }
        - {
            name: MacOS,
            sdk-platform: mac,
            os: macos-latest,
            prepare-os: ""
          }
        - {
            name: Windows,
            sdk-platform: win,
            os: windows-latest,
            prepare-os: export CC=gcc
          }
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: recursive
      - name: Get Rack-SDK
        run: |
          pushd $HOME
          curl -o Rack-SDK.zip https://vcvrack.com/downloads/Rack-SDK-${{ env.rack-sdk-version }}-${{ matrix.config.sdk-platform }}.zip
          unzip Rack-SDK.zip
      - name: Modify plugin version
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"
        run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
      - name: Build plugin
        run: |
          ${{ matrix.config.prepare-os }}
          export RACK_DIR=$HOME/Rack-SDK
          make -j dep
          make -j dist
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: dist
          name: ${{ matrix.config.name }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v2
      - uses: FranzDiebold/github-env-vars-action@v1.2.1
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.GITHUB_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.GITHUB_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body: |
            ${{ env.GITHUB_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.GITHUB_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v2
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true
1 Like

Tested it this morning for another dev.

All I can say is Thank you for your service to the community!

Seriously, legendary effort.

I just used this to compile every hora and erica plugins, it took me a bit of time to figure out how it works exactly because I don’t use github very often but it seems to work like a charm. Thank you very much for that, really great job !

Just a quick comment. The Linux plugin must not be built with “ubuntu-latest”. The current recommendation per Andrew’s comment a while back, is that builds must be done on Xenial (16.04). This is done so the built plugins are relatively free of run time linker load errors and symbol resolution errors even when run on different/older distros.

Nice catch @netboy3!

Next you’ll be telling me you have a build of theXOR! :wink:

2 Likes

Has to be 18.04 then, as github actions don’t support 16.04 anymore.

It’s up to you, but the official toolchain that builds the plugins in the library is using Xenial.

@Raph read below.

I previously used force_link_glibc_2.23.h which I notice has been removed from SDK in v2, is this reflecting a change in requirements? Is there another header-only way to enforce compatibility?

I know this, but I had never any issues at all. I even built some stuff with selfwritten CMake build scripts, using also C++17 standard features and didn’t use the VCVRack Makefiles at all and it just worked fine.

Also the Building - VCV Rack Manual says for Linux: 16.04+, which I think means that 16.04 is the minimum required version to build.

You need to understand that there are two different issues here. One is local build and another is integration and public binary dissemination.

As a user or a developer, you can setup a dev environment and build a linux plugin on any distro. The recommended distro to build is Ubuntu at a minimum of 16.04 as you stated. As long as you use the plugin on the same distro you’ve built it on, any Ubuntu of 16.04 version and beyond will work fine.

The other issue is integration and binary dissemination. Once you decide to provide binary versions of the linux plugin to other users, you are facing issues with dynamic linked library mismatching. This issue is not reviewed in the manual as most developers don’t deal with it and leave binary dissemination to the VCV library integration. The only ones that have to worry about it are the closed source developers.

As a general rule with linux dynamic linking, most distros will maintain backward library and symbol naming compatibility. This means that a dynamically linked binary that is built on an older distro will usually run fine on a newer one, but not vice-versa. This is why the recommendation and the official build toolchain in the VCV library uses Ubuntu 16.04 which is old enough for compatibility, but still under the Ubuntu ESM life-cycle (until 2026Q1).

4 Likes

As a real life example, Prok Modular’s linux plugins used to get compiled on 20.04, but I’m on 18.04 so I’d get linker errors and they’d fail even though they looked good and worked for everyone with a modern system. When I pointed it out, it took them a few minutes to spin up a 16.04 VM and resubmit and it fixed everything.

Thank you for creating this example repository, I used it to successfully automate the build of computerscare modules without much trouble!