Automated building and releasing Plugins on Github with Github Actions

qno · June 6, 2024, 3:04pm

I updated the workflow to use the up-to-date actions to get rid of the warnings.

name: Build VCV Rack Plugin
on: [push, pull_request]

env:
  rack-sdk-version: latest
  rack-plugin-toolchain-dir: /home/build/rack-plugin-toolchain

defaults:
  run:
    shell: bash

jobs:

  modify-plugin-version:
    name: Modify plugin version
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/cache@v4
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - run: |
          gitrev=`git rev-parse --short HEAD`
          pluginversion=`jq -r '.version' plugin.json`
          echo "Set plugin version from $pluginversion to $pluginversion-$gitrev"
          cat <<< `jq --arg VERSION "$pluginversion-$gitrev" '.version=$VERSION' plugin.json` > plugin.json
        # only modify plugin version if no tag was created
        if: "! startsWith(github.ref, 'refs/tags/v')"

  build:
    name: ${{ matrix.platform }}
    needs: modify-plugin-version
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/qno/rack-plugin-toolchain-win-linux
      options: --user root
    strategy:
      matrix:
        platform: [win-x64, lin-x64]
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive
      - uses: actions/cache@v4
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Build plugin
        run: |
          export PLUGIN_DIR=$GITHUB_WORKSPACE
          pushd ${{ env.rack-plugin-toolchain-dir }}
          make plugin-build-${{ matrix.platform }}
      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          path: ${{ env.rack-plugin-toolchain-dir }}/plugin-build
          name: ${{ matrix.platform }}

  build-mac:
    name: mac
    needs: modify-plugin-version
    runs-on: macos-latest
    strategy:
      fail-fast: false
      matrix:
        platform: [x64, arm64]
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: recursive
      - uses: actions/cache@v4
        id: plugin-version-cache
        with:
          path: plugin.json
          key: ${{ github.sha }}-${{ github.run_id }}
      - name: Get Rack-SDK
        run: |
          pushd $HOME
          wget -O Rack-SDK.zip https://vcvrack.com/downloads/Rack-SDK-${{ env.rack-sdk-version }}-mac-x64+arm64.zip
          unzip Rack-SDK.zip
      - name: Build plugin
        run: |
          CROSS_COMPILE_TARGET_x64=x86_64-apple-darwin
          CROSS_COMPILE_TARGET_arm64=arm64-apple-darwin
          export RACK_DIR=$HOME/Rack-SDK
          export CROSS_COMPILE=$CROSS_COMPILE_TARGET_${{ matrix.platform }}
          make dep
          make dist
      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          path: dist/*.vcvplugin
          name: mac-${{ matrix.platform }}

  publish:
    name: Publish plugin
    # only create a release if a tag was created that is called e.g. v1.2.3
    # see also https://vcvrack.com/manual/Manifest#version
    if: startsWith(github.ref, 'refs/tags/v')
    runs-on: ubuntu-latest
    needs:  [build, build-mac]
    steps:
      - uses: actions/checkout@v4
      - uses: FranzDiebold/github-env-vars-action@v2
      - name: Check if plugin version matches tag
        run: |
          pluginversion=`jq -r '.version' plugin.json`
          if [ "v$pluginversion" != "${{ env.CI_REF_NAME }}" ]; then
            echo "Plugin version from plugin.json 'v$pluginversion' doesn't match with tag version '${{ env.CI_REF_NAME }}'"
            exit 1
          fi
      - name: Create Release
        uses: softprops/action-gh-release@v2
        with:
          tag_name: ${{ github.ref }}
          name: Release ${{ env.CI_REF_NAME }}
          body: |
            ${{ env.CI_REPOSITORY_NAME }} VCV Rack Plugin ${{ env.CI_REF_NAME }}
          draft: false
          prerelease: false
      - uses: actions/download-artifact@v4
        with:
          path: _artifacts
      - name: Upload release assets
        uses: svenstaro/upload-release-action@v2
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: _artifacts/**/*.vcvplugin
          tag: ${{ github.ref }}
          file_glob: true

computerscare · June 6, 2024, 5:32pm

@qno Thanks a lot for creating the github repo for this, I just updated the gh actions for my plugin build and it was very helpful.

dan.tilley · August 28, 2024, 2:27pm

@qno

I have been trying to use Act to run my plugin’s action to build locally, but I am running into an error where the container does not have NodeJS in its Path.

Is there a reason why you have forked the Rack toolchain to build the container (other than just being faster to make changes to it)?

Any thoughts on how I might fix this NodeJS issue? The issue suggests something like:

- name: Hack container for local development
  if: ${{ env.ACT }}
  run: echo /home/runner/externals/node20/bin >> $GITHUB_PATH

then later suggests this will cause bugs, and possibly a custom env for the docker image like:

FROM blah...
ENV PATH "/home/runner/externals/node20/bin:$PATH"

but this is currently beyond my level of understanding

edit: ah I am assuming the above code would live in the Dockerfile, but not sure what blah... would be
2nd edit: hmm, well answered part of the question for myself, building the docker image from the rack repo fails, but building the docker image from your fork works…

thanks

Steve_Russell · November 15, 2024, 8:57pm

Is there a way of using the toolchain with Actions that can output unstripped debug versions as well as the release artifacts?

pachde · November 15, 2024, 11:56pm

Edit the build scripts to remove the symbol stripping.

cosinekitty · November 20, 2024, 4:15pm

With the release announcement of Rack 2.6.0, is there anything that needs to be updated in these build scripts? The one item mentioned in the announcement that might cause issues is:

Don’t combine SDK packages for Mac-x64 and Mac-arm64.

Today I’m going to start exploring what happens when I target my Sapphire plugin to the 2.6.0 SDK, and see if I can tell whether the existing build steps still work in GitHub Actions. I wonder if anyone else has already gone down that road and has anything interesting to share?

cosinekitty · November 22, 2024, 8:56pm

OK, I just pushed a change to my own Sapphire plugin’s documentation, which triggered running all my GitHub Actions unit tests as an expected side effect. It appears to still be using the 2.5.2 SDK:

--2024-11-22 20:35:06--  https://vcvrack.com/downloads/Rack-SDK-latest-mac-x64%2barm64.zip
Resolving vcvrack.com (vcvrack.com)... 107.191.126.116
Connecting to vcvrack.com (vcvrack.com)|107.191.126.116|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: Rack-SDK-2.5.2-mac-x64+arm64.zip [following]
--2024-11-22 20:35:07--  https://vcvrack.com/downloads/Rack-SDK-2.5.2-mac-x64+arm64.zip
Reusing existing connection to vcvrack.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 11511814 (11M) [application/zip]
Saving to: ‘Rack-SDK.zip’

If I’m reading this right, when you ask the vcvrack.com server for the latest SDK, it redirects you into downloading the 2.5.2 zip file. I believe this is because they are still calling Rack 2.6.0 “pre-release”, so it is not the official latest version yet.

So this got me thinking, what if I edit the GitHub Actions yml file to target the specific SDK 2.6.0. I did this and it fails:

--2024-11-22 20:49:00--  https://vcvrack.com/downloads/Rack-SDK-2.6.0-mac-x64%2barm64.zip
Resolving vcvrack.com (vcvrack.com)... 107.191.126.116
Connecting to vcvrack.com (vcvrack.com)|107.191.126.116|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-11-22 20:49:00 ERROR 404: Not Found.

The root cause is apparent when you look at the https://vcvrack.com/downloads/. As announced in the changelog, the MAC SDKs are now provided as separate files for arm64 and x64:

https://vcvrack.com/downloads/Rack-SDK-2.6.0-mac-arm64.zip
https://vcvrack.com/downloads/Rack-SDK-2.6.0-mac-x64.zip

I thought I would let other developers know ahead of time that your GitHub Actions scripts will need to be updated when 2.6.0 is officially released.

cosinekitty · November 22, 2024, 9:45pm

OK, one final update: this commit shows how to fix your yml script once 2.6.0 is released. I did this and my cross-platform plugin build now succeeds.

artificialcolors · November 23, 2024, 10:01pm

Thanks @qno for putting this together and keeping it current, and for everyone else who has been keeping things going. I’ve been using a LInux machine to build for Lin & Win, and a Mac for X86, but haven’t been able to get Mac ARM build working; this does the trick and is much easier.

In an effort to give back, if anyone is like me and constantly tinkering with readme files or other docs in the same repo and wasn’t already savvy with GitHub Actions, I updated my build script to only run the action on pushes to source code-related files (json, res, src) using “path filters”.

I replaced this:

on: [push, pull_request]

with:

on:
  push:
    paths:
      - '**.json'
      - 'src/**'
      - 'res/**'

Now it only triggers on pushes to plugin.json or any source/resource file, and I can tweak docs and doc images without all the plugins building. Obviously you can edit for your needs/files.

DaveVenom · November 23, 2024, 11:07pm

Thank you! I knew this was possible, but I haven’t studied GutHub Actions enough to figure it out. This will definitely help me!

dan.tilley · November 26, 2024, 11:45am

I’ve just seen this:

cosinekitty · March 12, 2025, 1:25am

Update: VCV Rack has released SDK 2.6.0. I noticed this by my GitHub Actions workflow getting a 404 error trying to download Mac SDKs. As expected, changes are required to your GitHub Actions workflows in 2.6.0. In my workflow yml file I also replaced “latest” with “2.6.0” because it looks like they no longer have a file that includes the name “latest”.

Here are my example changes for other developers who might benefit from this.

Bloodbat · March 12, 2025, 7:47am

Thanks Don!

Your work and your diff made updating my workflows a snap .

LarsBjerregaard · March 13, 2025, 9:02am

Just an FYI that the latest officially released version of Rack is still 2.5.2. v2.6.0 was a preview that never got released, because of problems I think.

dan.tilley · March 13, 2025, 9:05am

True there wasn’t an announcement, but the v2.6.0 is listed on the downloads page as 20th November

@Vortico are there issues with it?

LarsBjerregaard · March 13, 2025, 9:07am

All the previews are, that doesn’t mean it was released, because of e.g. problems. So I would personally caution against using SDK-2.6.0 for anything that should be released.

dan.tilley · March 13, 2025, 9:18am

I would like some clarity on this point if possible, the changelog posted by Don would suggest the version is considered ready, especially considering the commit changes from in development to 2024-11-20

My assumption was that the announcement was just forgotten about (but I concede that I might be making an ass of myself).

As a dev it would be nice if I were easily able to tell if a build on the downloads page is preview or not, or if there are known issues with the current in development next version, to have somewhere that could be documented.

LarsBjerregaard · March 13, 2025, 9:30am

I agree that Andrew should be a whole lot more stringent and clear about version numbers, previews, development blogs, changelogs and what is actually considered a release and what is not. Here’s my take:

What is on the Rack release/purchase page is what is actually officially released.
The changelog in the repository describes what was produced, and can be tested if you like, but not necessarily released.
Same goes for the downloads page.
Same goes for the development blog.

Of course, Andrew could have simply “forgotten” to release v2.6.0 but I doubt it, that hasn’t happened before, but what has happened many times before, is a preview version that was never released, because of problems people reported that needed to be fixed first, and I do remember people complaining about problems with 2.6.0.

In a really disciplined world you do it like this:

You have one or more development/testing channels/logs/blogs/pages, where you produce builds for people to try out and report back. They will be called things like product-3.9.1-alpha1, product-3.9.1-beta3, etc. When you think it’s ready you produce product-3.9.1-rc1 (rc == Release Candidate), which is the “last chance to find show-stoppers” build.

When you think it’s good enough you release product-3.9.1, which is ideally identical to product-3.9.1-rc1 except for the version number. This is the released version and you proclaim that loudly in your release channel/twitter/blog/whatever.

LarsBjerregaard · March 14, 2025, 7:30am

And Andrew has now produced

dan.tilley · March 26, 2025, 10:47am

Something new to consider