Automated building and releasing Plugins on Github with Github Actions

looks like that host isn’t finding package requested (libegl-mesa0_21.0.3-0ubuntu0.3~20.04.4_amd64.deb)

404 error is ‘Page not found’

I modified the ubuntu prepare to update first:

sudo apt-get update && sudo apt-get install -y libglu-dev

and that fixed a similar issue for me.

Hi,

thank you for pointing out the issues. I fixed them and updated the workflow definition at the example repository.

The updated workflow definition for the current Rack2 SDK:

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-sdk-version: 2.0.3

defaults:
  run:
    shell: bash

jobs:
  build:
    name: ${{ matrix.config.name }}
    runs-on: ${{ matrix.config.os }}
    strategy:
      matrix:
        config:
        - {
            name: Linux,
            sdk-platform: lin,
            os: ubuntu-latest,
            prepare-os: sudo apt update && sudo apt install -y libglu-dev
          }
        - {
            name: MacOS,
            sdk-platform: mac,
            os: macos-latest,
            prepare-os: ""
          }
        - {
            name: Windows,
            sdk-platform: win,
            os: windows-latest,
            prepare-os: export CC=gcc
          }
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: recursive
      - name: Get Rack-SDK
        run: |
          pushd $HOME
          curl -o Rack-SDK.zip https://vcvrack.com/downloads/Rack-SDK-${{ env.rack-sdk-version }}-${{ matrix.config.sdk-platform }}.zip
          unzip Rack-SDK.zip
      - name: Modify plugin version
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"
        run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
      - name: Build plugin
        run: |
          ${{ matrix.config.prepare-os }}
          export RACK_DIR=$HOME/Rack-SDK
          make -j dep
          make -j dist
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: dist
          name: ${{ matrix.config.name }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v2
      - uses: FranzDiebold/github-env-vars-action@v1.2.1
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.GITHUB_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.GITHUB_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body: |
            ${{ env.GITHUB_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.GITHUB_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v2
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true
1 Like

Tested it this morning for another dev.

All I can say is Thank you for your service to the community!

Seriously, legendary effort.

I just used this to compile every hora and erica plugins, it took me a bit of time to figure out how it works exactly because I don’t use github very often but it seems to work like a charm. Thank you very much for that, really great job !

Just a quick comment. The Linux plugin must not be built with “ubuntu-latest”. The current recommendation per Andrew’s comment a while back, is that builds must be done on Xenial (16.04). This is done so the built plugins are relatively free of run time linker load errors and symbol resolution errors even when run on different/older distros.

Nice catch @netboy3!

Next you’ll be telling me you have a build of theXOR! :wink:

2 Likes

Has to be 18.04 then, as github actions don’t support 16.04 anymore.

It’s up to you, but the official toolchain that builds the plugins in the library is using Xenial.

@Raph read below.

I previously used force_link_glibc_2.23.h which I notice has been removed from SDK in v2, is this reflecting a change in requirements? Is there another header-only way to enforce compatibility?

I know this, but I had never any issues at all. I even built some stuff with selfwritten CMake build scripts, using also C++17 standard features and didn’t use the VCVRack Makefiles at all and it just worked fine.

Also the Building - VCV Rack Manual says for Linux: 16.04+, which I think means that 16.04 is the minimum required version to build.

You need to understand that there are two different issues here. One is local build and another is integration and public binary dissemination.

As a user or a developer, you can setup a dev environment and build a linux plugin on any distro. The recommended distro to build is Ubuntu at a minimum of 16.04 as you stated. As long as you use the plugin on the same distro you’ve built it on, any Ubuntu of 16.04 version and beyond will work fine.

The other issue is integration and binary dissemination. Once you decide to provide binary versions of the linux plugin to other users, you are facing issues with dynamic linked library mismatching. This issue is not reviewed in the manual as most developers don’t deal with it and leave binary dissemination to the VCV library integration. The only ones that have to worry about it are the closed source developers.

As a general rule with linux dynamic linking, most distros will maintain backward library and symbol naming compatibility. This means that a dynamically linked binary that is built on an older distro will usually run fine on a newer one, but not vice-versa. This is why the recommendation and the official build toolchain in the VCV library uses Ubuntu 16.04 which is old enough for compatibility, but still under the Ubuntu ESM life-cycle (until 2026Q1).

4 Likes

As a real life example, Prok Modular’s linux plugins used to get compiled on 20.04, but I’m on 18.04 so I’d get linker errors and they’d fail even though they looked good and worked for everyone with a modern system. When I pointed it out, it took them a few minutes to spin up a 16.04 VM and resubmit and it fixed everything.

Thank you for creating this example repository, I used it to successfully automate the build of computerscare modules without much trouble!

@netboy3 yes, I think you are saying that these github builds are not 100% the same as a real toolchain build? Very good point.

I installed enough of the toolchain to make windows builds because I was finding that my own build worked fine, but sometime the library builds would crash. The only to replicate the crash was to use the rack toolchain.

Hi,

I just wanted to let you know that I added now the possibility to use the official Rack Plugin Toolchain for building a plugin with Github Actions. The Docker image is build at https://github.com/qno/rack-plugin-toolchain and published at https://github.com/qno/rack-plugin-toolchain/pkgs/container/rack-plugin-toolchain.

I updated the workflow definition at the example repository.

All you have to do is to use this workflow definition:

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-plugin-toolchain-dir: /home/build/rack-plugin-toolchain

defaults:
  run:
    shell: bash

jobs:
  build:
    name: ${{ matrix.platform }}
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/qno/rack-plugin-toolchain:v2
      options: --user root
    strategy:
      matrix:
        platform: [linux, mac, win]
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - name: Modify plugin version
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"
        run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
      - name: Build plugin
        run: |
          export PLUGIN_DIR=$GITHUB_WORKSPACE
          pushd ${{ env.rack-plugin-toolchain-dir }}
          make plugin-build-${{ matrix.platform }}
      - name: Upload artifact
        uses: actions/upload-artifact@v2
        with:
          path: ${{ env.rack-plugin-toolchain-dir }}/plugin-build
          name: ${{ matrix.platform }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v3
      - uses: FranzDiebold/github-env-vars-action@v1.2.1
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.GITHUB_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.GITHUB_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: actions/create-release@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          tag_name: ${{ github.ref }}
          release_name: Release ${{ github.ref }}
          body: |
            ${{ env.GITHUB_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.GITHUB_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v2
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true
2 Likes

That sounds very cool Silvio! I’ve never tried Github Actions - would you say that this is now very easy to use for anyone having their plugin source on Github? Could you give the “5-step cookbook for dummies” for automatically building a Github hosted plugin using this method?

For an basic overview what Github Actions are and how it works you can watch this tutorial - GitHub Actions Tutorial - Basic Concepts and CI/CD Pipeline with Docker - YouTube.

The explanation from the first post is still valid, so you can read the first post for more details.

To build a VCVRack plugin as Github Action with the provided workflow is very simple, you just need to to the following steps:

  1. In your plugin repository create a folder .github/workflows.
  2. Put the provided workflow definition build-plugin.yml into this folder.
  3. Make changes to your sources and push them to Github.
  4. In your Github repository navigate into the Action tab to see the progress and status of the Workflow run.
  5. To create a Github Release that contains the built plugin for all platforms you need to create and push a tag, e.g. like this:
    • git tag v2.0.7 -m "create v2.0.7"
    • git push origin --tags
    • Note: Make sure that your tag version number is the same as the version in the plugin.json and the tag starts with v (it is a convention), otherwise the the publish step will be canceled.

I think I will add this steps later to the Readme of the example repository.

Let me know if this helps and works.

1 Like

OK, I was made aware of legal issues for the usage of the MacOS SDK and to be 100% safe I removed the toolchain docker container and nobody can use the rack-plugin-toolchain image anymore.

Probably MacOS toolchain must be excluded from the container and MacOS plugin variant must only be build with the official available MacOS VM on Github.