Drafting a github action to build Ctrlr for multiple platforms

[sgorpi]

Hi,

I started drafting a github action to build CtrlrX upon commits to master, based on the "post-commit" script. Although we can debate if it is better to use this script vs using explicit command-line statements in the .yml file, using the post-commit script allows you to test some stuff locally.

So far I've gotten the Linux build to work. If you go to the "Actions" tab, then select "Build Release", you can select a specific commit for which it executed. When you've selected a commit, at the bottom of the page is an "Artifacts" list. When the build is successful it'll post a zipfile there with the build/package result.

I could use some help with the Windows and MacOS X builds (github has Intel and ARM runners available), as I don't have experience with those...

• The windows build is failing on some include paths
• The MacOS build is failing on missing an xcode project...

Grtz,
H

[damiensellier]

wow that sounds awesome, I'll take a look at how it works. I can provide an xcode project file and a visual studio one for sure.
I never worked with online compiling , I need to learn from your PR and we will definitely install it.

We keep posted, I'll push an xcode file to the master this weekend

[sgorpi]

Thanks!

I got the windows build as far as the linker, which now fails, after a long time, saying

fatal error C1002: compiler is out of heap space in pass 2

If you have any hints on that, that would be great. I read online suggestions of switching to the 64 bit toolchain or of enabling/disabling link-time optimization. Might give that a go also, but if you have a more recent visual Studio project file that might address it, also welcome :) take care to define relative paths to include directories etc if possible

Cheers, H

[damiensellier]

I often get this problem on my mac bootcamp with VS.
We should ask John @dnaldoog if he could provide his Visual Studio file since he always deliver perfectly working Windows binaries.

Have a nice weekend ;) Thanks for your help!

Damien

[dnaldoog]

I had that problem, but when was it? And why was it? Was it on VS 2019 perhaps or when compiling a 32 bit version. I just can't recall, because on VS 2022 the process for [compiling CtrlrX is relatively easy these days](https://github.com/RomanKubiak/ctrlr/discussions/717s.

Does this screenshot help?

It wasn't clear to me how to export project files according to this page but attached are various project files from the builds folder.

various project files Visual Studio 2022.zip

[damiensellier]

Thanks John, I wonder if you have anything to change manually in VS to get this project file right?

We will need to find a way to create a working makefile from the projucer with the right flags and preprocessor declarations so that github desktop would push them to the CI/CD pipeline and get the server do its compiling job automatically. Same with Xcode (but it's ok without mods) and same with VS.

I'll remove those IDE files from the gitignore list so that the repo will have them from now on.

[damiensellier]

BTW boost needs to be unzipped for all compilers.

[damiensellier]

There's already a project that fully implemented Github CI with actions for JUCE plugins called PampleJuce, it has everything required if you need to get some inspiration : https://github.com/sudara/pamplejuce

[damiensellier]

Hi Hedde, I just approved the workflow and merged your PR.
Thanks for your hard work, that will really help with the releases.
I already noticed I'll need to update a few things about the plugin name and manufacturer ID in the makefile.
I used spaces at the end in the projucer settings to allow hacking the binaries in order to produce custom vst3 with the proper naming.
I'll work on it and test asap ;)

[sgorpi]

Awesome! Let me know if you need anything further. My username is my Gmail address.

I noticed that the macOS build broke after removing the Builds/ directory, but couldn't really understand the link. It could also be that the GitHub 'runner' for macOS changed and that it is missing some dependency now that you need to install in the GitHub workflow file...

Good luck with it!

[damiensellier]

Excellent, I guess it's because the process failed to complete the code signing step.

[damiensellier]

Ok I here is the solution offered by Gemini :

Based on the provided crash log, the GitHub Actions CI/CD workflow is failing because the Rez tool cannot find the AudioUnit/AudioUnit.r resource header file. This error occurs during the compilation of your AU (Audio Unit) plugin component, which is a key part of the build process for macOS audio software. The log shows a Fatal Error! and states, failed to find AudioUnit/AudioUnit.r. This suggests a missing or improperly configured dependency on the CI runner.
Why It's Failing 😥
The error is specifically related to the Rez tool, a part of Apple's developer tools used to compile resource files (.rsrc) for macOS applications and plugins. The AudioUnit.r file is a crucial resource definition header for building Audio Unit plugins.

• Missing System Header: The primary reason for the failure is that the CI/CD environment (the GitHub Actions runner) is missing the AudioUnit.r file. This file is typically located within the macOS SDK and is a standard part of Apple's developer tools.
• Deprecation of Carbon: Historically, AudioUnit.r was part of the Carbon framework, which Apple has been deprecating. However, the Rez tool still relies on these older resource files for compiling legacy-style Audio Unit components. The log shows the command is looking for headers in /System/Library/Frameworks/AudioUnit.framework/Headers, which is the correct path for older SDKs.
• Runner Environment Mismatch: Your local build works because you likely have a complete Xcode installation with all the necessary SDK components, including older headers. The GitHub Actions macOS runner, however, might have an updated, minimal, or differently configured SDK that no longer includes this specific file at the expected path. Apple has been moving away from these legacy resource compilation methods, so it's possible a recent update to the runner images removed this dependency.
  How to Fix It 🛠️
  To resolve this issue, you need to ensure the CI/CD runner has access to the required header file. Here are the most effective solutions:

1. Add the Legacy Headers to Your Repository
   This is the most reliable fix. You can download the necessary headers from an older macOS SDK and include them directly in your project's repository. This ensures the build is self-contained and not dependent on the specific configuration of the GitHub Actions runner.

• Step 1: Find a working macOS system (like your local machine). The required header files are typically located within the Xcode application bundle: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/AudioUnit.framework/Headers/.
• Step 2: Copy the AudioUnit.r and any other relevant files (like CoreAudioTypes.r) into a new directory within your project, for example, Resources/LegacyHeaders/.
• Step 3: Update your CMake build script to tell the Rez tool where to find these files. You need to add an include flag (-I) that points to this new directory. Your CMake file will look something like this:

Add the new include path to the Rez command

set(JUCE_AU_REZ_FLAGS ${JUCE_AU_REZ_FLAGS} -I "${CMAKE_SOURCE_DIR}/Resources/LegacyHeaders")

This will ensure the Rez tool always finds the necessary files, regardless of the CI runner's environment.
2. Install a Legacy SDK on the Runner
While less reliable due to potential changes in GitHub's runner images, you could try to install an older macOS SDK. This can be done by adding a step to your GitHub Actions workflow file (.yml) to download and install a specific version of the Xcode command-line tools that includes the legacy headers.
Note: This is often complex and prone to breaking with future runner updates. The first option is a more robust, long-term solution.

[sgorpi]

Sounds like chatgpt 😜 it's odd, because the build of a few days ago was all green. I guess something changed in the runner image. I saw a new release withchangelog at https://github.com/actions/runner-images/releases

I'll play a bit with the workflow on my fork, maybe it's something from your list or there's an easy solution like fixing the search path or installing some depen from a package management system... But if you find a fix in the meantime let me know !

[damiensellier]

Yes it's my buddy Gemini who was providing the support :)
I won't have free time for a week or so.
If you find the solution, let me know!

[damiensellier]

We also need to check if other setups for macOS can fix the issue. Now we use the latest-stable macOS setup with Maxim lobanov settings but maybe an older version or different setup could cover this audiounit problem.

https://github.com/maxim-lobanov/setup-xcode

[damiensellier]

I posted on the pamplejuce issue board :
sudara/pamplejuce#166 (comment)

Maybe we should remove the Rez command pointed by the user as described and test.

[sgorpi]

Hey Damien,

I've analyzed the github action logs between a working and later non-working run, and notice the main difference:
For the working run

Image: macos-14-arm64
Version: 20250901.1774

For the not-working runs after:

Image: macos-14-arm64
Version: 20250915.1804

Couldn't find obvious hints in the release notes of that image tho: https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20250915.1804

Further, It is something with the Rez command, but not specifically the on that this user points to, but the one in a JUCE CMake file:
https://github.com/damiensellier/CtrlrX/blob/5.6.34-Maintenance-Build-Workflow/JUCE/extras/Build/CMake/JUCEUtils.cmake#L422

I compared with a later JUCE but couldn't find this at all... What I did find are some hints that this has to do with "AudioUnit v1", and that later JUCE (and other plugins) have dropped support for this. I saw for example a PR that plainly removed the related code in an (old) fork of JUCE: Celemony/JUCE_ARA@10bef69

On the other hand, if I add enough "include directives" to the Rez command linked in the above file, we may get it working again as-is, because the required files are still somewhere on the macos image, for example I found the 'missing' AudioUnit.r file in /Applications/Xcode_16.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/AudioUnit.framework (and that directory can be found in some cmake variable to make it less dependent on compiler version). But, when adding this directory it complains about another missing file...

I'll first chase down the latter. If that works, then good. If it doesn't we might need to consider dropping AudioUnit v1 support similar to the PR I linked above...

[damiensellier]

Oh excellent ! That's definitely the main difference between the working and non-working setup.
We can just call for this particular setup with the xcode version parameter:
https://github.com/maxim-lobanov/setup-xcode?tab=readme-ov-file#available-parameters

https://devhints.io/semver

?

[damiensellier]

https://github.com/sgorpi/CtrlrX/actions/runs/17116456343/job/48548448436

The working action has :

Run maxim-lobanov/setup-xcode@v1
  with:
    xcode-version: latest-stable
  env:
    BUILD_TYPE: Release
    BUILD_DIR: build
    DISPLAY: :0
    HOMEBREW_NO_INSTALL_CLEANUP: 1
    SCCACHE_GHA_ENABLED: true
    SCCACHE_CACHE_MULTIARCH: 1
    IPP_DIR: C:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib\cmake\ipp
    HAVE_DEV_ID_APP_CERT: false
    HAVE_DEV_ID_INSTALLER_CERT: false
    HAVE_AZURE_CLIENT_SECRET: false
Switching Xcode to version 'latest-stable'...
0s
Run maxim-lobanov/setup-xcode@v1
  with:
    xcode-version: latest-stable
  env:
    BUILD_TYPE: Release
    BUILD_DIR: build
    DISPLAY: :0
    HOMEBREW_NO_INSTALL_CLEANUP: 1
    SCCACHE_GHA_ENABLED: true
    SCCACHE_CACHE_MULTIARCH: 1
    IPP_DIR: C:\Program Files (x86)\Intel\oneAPI\ipp\latest\lib\cmake\ipp
    HAVE_DEV_ID_APP_CERT: false
    HAVE_DEV_ID_INSTALLER_CERT: false
    HAVE_AZURE_CLIENT_SECRET: false
Switching Xcode to version 'latest-stable'...
Xcode is set to 16.2.0 (16C5032a)
9s

The later on Parking setup :
https://github.com/sgorpi/CtrlrX/actions/runs/17848835385/job/50835343223

##[debug]Evaluating condition for step: 'Use latest Xcode on system (macOS)'
##[debug]Evaluating: (success() && (matrix.name == 'macOS'))
##[debug]Evaluating And:
##[debug]..Evaluating success:
##[debug]..=> true
##[debug]..Evaluating Equal:
##[debug]....Evaluating Index:
##[debug]......Evaluating matrix:
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'name'
##[debug]....=> 'macOS'
##[debug]....Evaluating String:
##[debug]....=> 'macOS'
##[debug]..=> true
##[debug]=> true
##[debug]Expanded: (true && ('macOS' == 'macOS'))
##[debug]Result: true
##[debug]Starting: Use latest Xcode on system (macOS)
##[debug]Loading inputs
##[debug]Loading env
Run maxim-lobanov/setup-xcode@v1
Switching Xcode to version 'latest-stable'...
::group::Available Xcode versions:
Available Xcode versions:
##[debug]Xcode 16.2.0 (16C5032a) (/Applications/Xcode_16.2.app) will be set
Xcode is set to 16.2.0 (16C5032a)
##[debug]Node Action run completed with exit code 0
##[debug]MD_APPLE_SDK_ROOT='/Applications/Xcode_16.2.app'
##[debug]Set output version = 16.2.0
##[debug]Set output path = /Applications/Xcode_16.2.app
##[debug]Finishing: Use latest Xcode on system (macOS)

They both use Xcode 16.2.0 (16C5032a) . Strange!

[damiensellier]

We could then use the working setup like this :

jobs:
  build:
    runs-on: macos-14-arm64-20250901.1774 # !!!WRONG !!! Use the specific working image version

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Use latest Xcode
        uses: maxim-lobanov/setup-xcode@v1
        with:
          xcode-version: latest-stable

      # etc...

GitHub will probably deprecate the older system but it should be working short term. What do you think about that Hedde?

Edit: this sucks , the syntax for the macos version is definitely not valid. 😭

[damiensellier]

I'm not even sure GitHub has macOS 13 available. That could be a short term solution until they deprecate it.

[damiensellier]

The guy from pamplejuce tried with the latest macos 14 runner and it was successful apparently. He has a lead about our issue :

sudara/pamplejuce#166 (comment)

https://forum.juce.com/t/cmake-macos-commandlinetools-support/44676/6

[sgorpi]

A small update:

1. JUCE-based plugins that use version 6.1.0 or later will not have the issue, as they simply removed the whole Rez command (and other things). I traced it back to juce-framework/JUCE@cc7e4d1 which is a part of the 6.1.0 release...
2. I tried to manually add the missing resources (I had included a "Tmate" step in the workflow; if you manually dispatch the workflow with the debug flag enabled, it'll allow you to ssh to the github runner and do things in bash)... With that I could find a couple of include directories for the missing file, but I got stuck at failed to find CoreServices/CoreServices.r even though it does state -I /Applications/Xcode_16.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/AudioUnit.framework/../System/Library/Frameworks/CoreServices.framework/Versions/A/Headers now... You can see that in https://github.com/sgorpi/CtrlrX/actions/runs/17935382347/job/51000509244

Now maybe we have a couple of things to consider:

• we could roll back to the macos-13 image, but it gets deprecated early december: https://github.blog/changelog/2025-09-19-github-actions-macos-13-runner-image-is-closing-down/ so that'd be postponing the inevitable
• I can open a new PR to continue the discussion and make changes easier visible
• It seems that the issue may be solved by updating to a later JUCE version (also better support for ARM if I see the commits)... now I understand that's not an option due to the incopatibility of LUA with C++14 .. so what is then left is to selectively port back changes to the CMake scripts and other parts of the code until it works again...

[damiensellier]

Hi Hedde, thanks for looking into all the details.
We are not lucky because it was working last week :(
You're right, as you mentionned we cannot upgrade to JUCE 6.1 and above because they updated c++ to a higher version that is incompatible with LuaBind, it would require spending weeks of work to upgrade the whole software and if we replace by another lua<-->c++ framwork that would also deprecate all panels since the syntax to bind to cpp function would be slightly different. It's a no-no.

I already brought some JUCE 6.1+ code into our JUCE 6.0.8 lib. If you think we can bring the rez related changes from 6.1 to our 6.0.8 juce folder that would be a solution unless it uses c++ 17 instead of C++11

#41

I don't know if as Gemini suggested we could include in the repository the missing files from the sdk that were available in Image: macos-14-arm64 Version: 20250901.1774 . But bringing JUCE 6.1 changes for the cmake module to our 6.0.8 would be the best way IMHO.

You can definitely open a new PR to the same branch that would make things clearer.

Hope we will be able to find an easy solution.

Thanks, take care

Damien

[damiensellier]

NOTE : 6.1 vs 6.0.8

juce-framework/JUCE@6.0.8...6.1.0

juce-framework/JUCE@6.0.8...6.1.0#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a

[damiensellier]

Ok so I checked on my mbp macOS 13.7.5 install with xcode 14.2 and this is the actual path for CoreServices.r :
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Versions/A/Headers/CoreServices.r

The AudioUnit Framwork header folder is actually there :
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/AudioUnit.framework/Versions/A/Headers

We could add a folder in the resources for those deprecated files if needed.
Just tell me and I can upload a zip here.

[damiensellier]

Here are the zip folders of CoreServices and AudioUnits frameworks :

AudioUnit.framework.zip
CoreServices.framework.zip

[damiensellier]

I'm working on it right now.
I pushed a folder with the missing legacy frameworks for the AU Build and forcing REZ to use them.
I'll keep you posted.

[damiensellier]

I'm almost done with it! I'll post the solution later

[damiensellier]

I'm patching the missing file 1 by 1 which is time consuming, I'm progressing but need to add some framework folders from the old sdk containing tthe follwing dependencies :

To resolve the resource compiler errors on macOS, the following files need to be present in the JUCE directory structure.

File: ConditionalMacros.r

    Path: JUCE/modules/juce_audio_plugin_client/ConditionalMacros.r

File: CoreServices.r

    Path: JUCE/modules/juce_audio_plugin_client/CoreServices.r

File: AudioUnit.r

    Path: JUCE/modules/juce_audio_plugin_client/AudioUnit.r

File: AUComponent.r

    Path: JUCE/modules/juce_audio_plugin_client/AUComponent.r

File: AudioUnitCarbonView.r

    Path: JUCE/modules/juce_audio_plugin_client/AudioUnitCarbonView.r

These files are all located in older macOS SDKs and need to be placed in the specified directory to be found by the Rez compiler.

[sgorpi]

I think I have the solution... All these files do exist on the github runners. Specifically in /Applications/Xcode_16.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/ ... Notice the Xcode version is mentioned in this path...

After going down the same rabit hole as you (finding the locations of all these files), after a while i noticed the Rez argument isysroot wasn't set... But, to find the AudioUnit framework, an include path relative to /Applications/Xcode_16.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/ was autodetected by CMake.... So in manually setting isysroot to that directory, the build passed... I've just committed a change in my repo that sets this path, but relative to the automatically detected AudioUnit framework (so it should stay OK with future Xcode version updates...

What base-branch can I base a new PR on? Then it's easier to show the diff 😄

[sgorpi]

https://github.com/sgorpi/CtrlrX/actions/runs/18028322401/job/51299684813 macOS build is green again 🎉

[damiensellier]

Hi Hedde, that's really awesome 👍
I spent my afternoon yesterday doing revisions of the script to patch the missing files one by one and everytime I succeeded, another one was missing. I stopped yesterday at revision 70 :😭
That's so nice you found some traces of the old SDK to locate it.
Smart move!
I'll update my Branch with your mods and I'll also include a part of my script where we bypass the code signing for mac if the certificate is not available in the GH secrets.

Thank you so much 🙏

[damiensellier]

Hi Hedde,
So I have everything working, I'lll need to add my certificate for codesigning.
I updated the worflow so that the VST3 get the proper Category "Instrument|Tools" and also the custom VST3 Plugin Name and Manufacturer Name with trailing spaces so that the patching process for exported instances get their proper panel IDs.

if ("VST3" IN_LIST FORMATS)
    set_target_properties(${PROJECT_NAME} PROPERTIES
        # Overwrite the name property for VST3 compilation (includes trailing space)
        JUCE_PLUGIN_NAME "CtrlrX                          "
        # Overwrite the manufacturer property for VST3 compilation
        JUCE_COMPANY_NAME "CtrlrX Project                  "
    )
endif()

I had to patch JUCEUtils.cmake for the plugin category because the way JUCE cmake is setup for vst3 category automatically place "Tools" as an "Effect" such as "Fx|Tools" but the VST3 SDK allows "Instrument|Tools", it's not possible from CMakeLists.txt

It was a pain but it's working as it should be and exactly as if I was compiling on Xcode.

https://github.com/damiensellier/CtrlrX/actions/runs/18072964512/job/51425448166

The working branch is : https://github.com/damiensellier/CtrlrX/tree/5.6.34-Maintenance-Build-Workflow
I'll merge to the 5.6.34 branch when I'll have the codesigning working, then merge down to the Master branch for the release.

The commit is : 21faae5

You can update your local install from there if needed.

Thanks a lot Hedde for your help, I really appreciate it.

Damien

[sgorpi]

Awesome! Seems like you've gotten the hang of the github actions and cmake 👍 Do these additional spaces also solve the issue #176 that John was detailing?

I might play with some other github actions stuff like showing compile warnings in a PR as comments on code, if that proves interesting/useful I'll make another PR 😉

[damiensellier]

I removed the only source of warning on mac about ninja already being installed. I'm surprised how well and fast it's working. Less than 10 minutes for doing the process on each platform, the GH servers must be crazy powerful!

The ending spaces trail will only help if a search and replace process is added to the Linux export function.
I don't know how VST3 hosts determine the name of the plugin, if it's like on mac or windows it's definitely needed.
John will take a look at it because I don't have Linux and never worked with it.
I'd be really happy if everything could work on all platforms.