Automated building and releasing Plugins on Github with Github Actions

I know this, but I had never any issues at all. I even built some stuff with selfwritten CMake build scripts, using also C++17 standard features and didn’t use the VCVRack Makefiles at all and it just worked fine.

Also the Building - VCV Rack Manual says for Linux: 16.04+, which I think means that 16.04 is the minimum required version to build.

You need to understand that there are two different issues here. One is local build and another is integration and public binary dissemination.

As a user or a developer, you can setup a dev environment and build a linux plugin on any distro. The recommended distro to build is Ubuntu at a minimum of 16.04 as you stated. As long as you use the plugin on the same distro you’ve built it on, any Ubuntu of 16.04 version and beyond will work fine.

The other issue is integration and binary dissemination. Once you decide to provide binary versions of the linux plugin to other users, you are facing issues with dynamic linked library mismatching. This issue is not reviewed in the manual as most developers don’t deal with it and leave binary dissemination to the VCV library integration. The only ones that have to worry about it are the closed source developers.

As a general rule with linux dynamic linking, most distros will maintain backward library and symbol naming compatibility. This means that a dynamically linked binary that is built on an older distro will usually run fine on a newer one, but not vice-versa. This is why the recommendation and the official build toolchain in the VCV library uses Ubuntu 16.04 which is old enough for compatibility, but still under the Ubuntu ESM life-cycle (until 2026Q1).

4 Likes

As a real life example, Prok Modular’s linux plugins used to get compiled on 20.04, but I’m on 18.04 so I’d get linker errors and they’d fail even though they looked good and worked for everyone with a modern system. When I pointed it out, it took them a few minutes to spin up a 16.04 VM and resubmit and it fixed everything.

Thank you for creating this example repository, I used it to successfully automate the build of computerscare modules without much trouble!

@netboy3 yes, I think you are saying that these github builds are not 100% the same as a real toolchain build? Very good point.

I installed enough of the toolchain to make windows builds because I was finding that my own build worked fine, but sometime the library builds would crash. The only to replicate the crash was to use the rack toolchain.

Hi,

I just wanted to let you know that I added now the possibility to use the official Rack Plugin Toolchain for building a plugin with Github Actions. The Docker image is build at https://github.com/qno/rack-plugin-toolchain and published at https://github.com/qno/rack-plugin-toolchain/pkgs/container/rack-plugin-toolchain.

I updated the workflow definition at the example repository.

All you have to do is to use this workflow definition:

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-plugin-toolchain-dir: /home/build/rack-plugin-toolchain

defaults:
  run:
    shell: bash

jobs:
  build:
    name: ${{ matrix.platform }}
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/qno/rack-plugin-toolchain:v2
      options: --user root
    strategy:
      matrix:
        platform: [linux, mac, win]
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - name: Modify plugin version
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"
        run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
      - name: Build plugin
        run: |
          export PLUGIN_DIR=$GITHUB_WORKSPACE
          pushd ${{ env.rack-plugin-toolchain-dir }}
          make plugin-build-${{ matrix.platform }}
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: ${{ env.rack-plugin-toolchain-dir }}/plugin-build
          name: ${{ matrix.platform }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v3
      - uses: FranzDiebold/github-env-vars-action@v1.2.1
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.GITHUB_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.GITHUB_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body: |
            ${{ env.GITHUB_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.GITHUB_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v2
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true
2 Likes

That sounds very cool Silvio! I’ve never tried Github Actions - would you say that this is now very easy to use for anyone having their plugin source on Github? Could you give the “5-step cookbook for dummies” for automatically building a Github hosted plugin using this method?

For an basic overview what Github Actions are and how it works you can watch this tutorial - GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker - YouTube.

The explanation from the first post is still valid, so you can read the first post for more details.

To build a VCVRack plugin as Github Action with the provided workflow is very simple, you just need to to the following steps:

  1. In your plugin repository create a folder .github/workflows.
  2. Put the provided workflow definition build-plugin.yml into this folder.
  3. Make changes to your sources and push them to Github.
  4. In your Github repository navigate into the Action tab to see the progress and status of the Workflow run.
  5. To create a Github Release that contains the built plugin for all platforms you need to create and push a tag, e.g. like this:
    • git tag v2.0.7 -m "create v2.0.7"
    • git push origin --tags
    • Note: Make sure that your tag version number is the same as the version in the plugin.json and the tag starts with v (it is a convention), otherwise the the publish step will be canceled.

I think I will add this steps later to the Readme of the example repository.

Let me know if this helps and works.

1 Like

OK, I was made aware of legal issues for the usage of the MacOS SDK and to be 100% safe I removed the toolchain docker container and nobody can use the rack-plugin-toolchain image anymore.

Probably MacOS toolchain must be excluded from the container and MacOS plugin variant must only be build with the official available MacOS VM on Github.

I removed the MacOS toolchain from the rack-plugin-toolchain Docker image. Now it just contains the toolchains for Windows and Linux.

With this updated version of the workflow definition build-plugin.yml, the Windows and Linux variant of a Rack plugin gets build with the official Rack build toolchain and the MacOS versions gets build on a official macos-10.15 Github VM.

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-sdk-version: 2.1.2
  rack-plugin-toolchain-dir: /home/build/rack-plugin-toolchain

defaults:
  run:
    shell: bash

jobs:

  modify-plugin-version:
    name: Modify plugin version
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/cache@v2
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"

  build:
    name: ${{ matrix.platform }}
    needs: modify-plugin-version
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/qno/rack-plugin-toolchain-win-linux
      options: --user root
    strategy:
      matrix:
        platform: [win, linux]
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - uses: actions/cache@v2
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Build plugin
        run: |
          export PLUGIN_DIR=$GITHUB_WORKSPACE
          pushd ${{ env.rack-plugin-toolchain-dir }}
          make plugin-build-${{ matrix.platform }}
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: ${{ env.rack-plugin-toolchain-dir }}/plugin-build
          name: ${{ matrix.platform }}

  build-mac:
    name: mac
    needs: modify-plugin-version
    runs-on: macos-10.15
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - uses: actions/cache@v2
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Get Rack-SDK
        run: |
          pushd $HOME
          curl -o Rack-SDK.zip https://vcvrack.com/downloads/Rack-SDK-${{ env.rack-sdk-version }}-mac.zip
          unzip Rack-SDK.zip
      - name: Build plugin
        run: |
          export RACK_DIR=$HOME/Rack-SDK
          make -j dep
          make -j dist
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: dist
          name: mac

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs:  [build, build-mac]
    steps:
      - uses: actions/checkout@v3
      - uses: FranzDiebold/github-env-vars-action@v1.2.1
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.GITHUB_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.GITHUB_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body: |
            ${{ env.GITHUB_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.GITHUB_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v2
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true
4 Likes

In case anyone is interested, you can use the scrips above on free private GitHub repositories. You will just have to remove the parts related to CodeQL:

      #- name: Initialize CodeQL
      #  if: runner.os == 'Linux'
      #  uses: github/codeql-action/init@v1
      #  with:
      #    languages: ${{ matrix.language }}

and

      #- name: Perform CodeQL Analysis
      #  if: runner.os == 'Linux'
      #  uses: github/codeql-action/analyze@v1

For anyone curious, CodeQL scans code for vulnerabilities, so its not integral to building your plugin.

Does VCV use CodeQL in the library build toolchain? I hope so.

why do we need to remove those parts if using this on a private repo? is it against some TOS or something? because it certainly doesn’t prevent you from successfully building the plugin, as i just added it to my private repo and it worked flawlessly as is. am i missing something?

For me it failed in the “Initialize Code QL” section. I got the error “repository not enabled for code scanning” which lead me to this page which says “GitHub Advanced Security is available for enterprise accounts on GitHub Enterprise Cloud and GitHub Enterprise Server 3.0 or higher. GitHub Advanced Security is also included in all public repositories on GitHub.com

hmm, that’s strange. as far as i know, my account is just the normal free account, but i didn’t see any such error. ¯\(●_●)

See the 2nd bullet in the CodeQL License.

fundamental recorder seems to be built on a newer version of ubuntu, probably 22.04. it wont work with 18.04

That would be surprising. By the way, Recorder is not in Fundamental:

i used this the other day to test it out and it worked great, but i just added it to a new repo i created, followed all the same steps, and for some reason when pushing my new tag (“v2.0.0”), all the action’s steps run except the publish step, which is “skipped” for some reason. any ideas?

EDIT: well, whatever it was, it’s working now after creating a new tag for v2.0.1. not sure what i did wrong last time, but it’s working now.

QQ: do you plan to keep this repo up to date?

There are two issues that will require an update shortly:

Node.js 12 actions are deprecated. For more information see: GitHub Actions: All Actions will begin running on Node16 instead of Node12 - The GitHub Blog. Please update the following actions to use Node.js 16: actions/checkout, actions/upload-artifact, actions/checkout

macOS-latest pipelines will use macOS-12 soon. For more details, see https://github.com/actions/runner-images/issues/6384

Edit: Apologies if you have already addressed these issues, I just realised that I have not updated my actions recently, so these may not be a factor…