ELKS + Nano X github action

[toncho11]

This PR allows automatic generation of ELKS HDD image with Nano X applications integrated.

I made a new workflow that currently generates a minix-32MB hdd image with the latest version of Microwindows Nano X. Maybe there can be another way to do it, but I think for now we can use this one (because it works already).

[ghaerr]

Hello @toncho11,

Does this run automatically? And how do you test it? I don't know that much about exactly how CI's run on Github.

This is great, but I have a concern if it runs automatically but also builds ELKS from scratch in entirety (again), before building Microwindows. This would add another 3-4 minutes to each PR commit. If this is the case, it might be better to build Microwindows within the main CI build after normal completion, then add the contents to a special MINIX HDD.

If there was a way to somehow control how often it ran (possibly by inspecting changes to the external repos), then we might be able to use such a mechanism to build other external ELKS applications (like 8086-toolchain, Doom, etc) and automatically create an HDD image.

Thank you!

[toncho11]

Currently it takes 3 minutes, 22 seconds. I think it runs on every change. I used the "main" as a base for this "nanox" workflow.

The way it is designed now allows the nanox part to be simply appended at the end of "main" workflow. This is because it clones, compiles and then adds the binaries to an image of choice using mfs. So it is currently minix only, but it can be extended to mount the HDD fat images and copy the nanox binaries to them as well.

So yes, you can simply append after the run: ./build.sh auto allimages in the current "main" workflow:

      # ------------------------------------------------------------
      # Clone MicroWindows
      # ------------------------------------------------------------
      - name: clone microwindows
        run: |
          git clone https://github.com/ghaerr/microwindows.git
      # ------------------------------------------------------------
      # Build Nano-X for ELKS
      # ------------------------------------------------------------
      - name: build Nano-X for ELKS
        working-directory: microwindows/src
        run: |
          cd "$GITHUB_WORKSPACE"
          . ./env.sh
          echo "TOPDIR=$TOPDIR"
          test -d "$TOPDIR/elkscmd"
          cd microwindows/src
          make -f Makefile.elks
      # ------------------------------------------------------------
      # Copy Nano-X binaries into HDD Minix image (mfs cp)
      # ------------------------------------------------------------
      - name: install Nano-X into Minix HDD image
        run: |
          cd "$GITHUB_WORKSPACE"
          . ./env.sh
          
          IMAGE="$GITHUB_WORKSPACE/image/hd32-minix.img"
          
          test -f "$IMAGE" || {
            echo "ERROR: $IMAGE does not exist"
            ls -l "$GITHUB_WORKSPACE/image"
            exit 1
          }
          echo "which mfs:"
          which mfs || echo "mfs not found in PATH (expected in tools/bin)"
          
          MFS="$(command -v mfs)"
          
          echo "Using mfs at: $MFS"
          if [ -z "$MFS" ] || [ ! -x "$MFS" ]; then
            echo "ERROR: mfs not found or not executable"
            exit 1
          fi
      
          for f in microwindows/src/bin/*; do
            echo "Installing $(basename "$f") into /bin"
            "$MFS" "$IMAGE" cp "$f" "/bin/$(basename "$f")"
          done

          # Handle resource files:
          # Copy nxworld.map from host directly into /lib
          "$MFS" "$IMAGE" cp microwindows/src/bin/nxworld.map /lib

          # Remove it from /bin inside the image
          "$MFS" "$IMAGE" rm /bin/nxworld.map

Yes, this could be further optimized to detect if a repository was really updated. But this needs more experimentation.

[toncho11]

This is madness.
I am trying to make nanox workflow use an artifact from the "main". This way we can only compile microwindows and add the binaries and produce a new artifact with the new binaries.
But I can not make it work. It is late here and the whole process is too tedious.

[ghaerr]

But I can not make it work.

Thanks @toncho11 for trying, I know CI can be difficult. Lets leave this open and I'll try to play with it a bit and see if I can get it to work.

[ghaerr]

We may need to add the workflow directly into main.yml or reference it from main.yml like ow-libc.yml is.

[toncho11]

Reopened.

[toncho11]

This my last idea of nanox script that reuses main is too complicated and unreachable goal with github actions.

2 solutions:

1. What I proposed in: ELKS + Nano X github action #2562 (comment) where only one workflow is used.

2. Actually my original idea of rebuilding everything ELSK + nano X was also good - because it worked!!! Now it is lost somewhere in the history of github. Maybe this version: 25de2b8

[toncho11]

I reverted it to the original version.

We can add support for "on demand" execution, I used it already. Or maybe it is possible to make it only on demand. This way we decide when to produce a new elks+nanox image. But I do not see a problem to have separate workflow like this that produces this combined image, it does not block you from working on ELKS.

[ghaerr]

We can add support for "on demand" execution, I used it already.

How is that done?

I do not see a problem to have separate workflow like this that produces this combined image

Does this separate workflow rebuild ELKS from scratch each time, resulting in the additional 3-4 minute delay per push? As you may have noticed, I've already turned off additional PR builds for the same reason (in main.yml):

name: main

on:
  push:
    paths:
      - '**'
      - '!.github/workflows/cross.yml'
      - '!.github/workflows/ow-libc.yml'
      - '!tools/*'
#  pull_request:
#    paths:
#     - '**'
#     - '!.github/workflows/cross.yml'
#     - '!.github/workflows/ow-libc.yml'
#     - '!tools/*'

CI is definitely not my favorite programming as its hard to get right, as you've already found out. I'll try to understand more about what your "current" version does and how we might be able to get a Microwindows build integrated as we both desire.

[toncho11]

The code below is triggered when:

1. Triggered manually from WEBUI (workflow_dispatch)

or

workflow that triggered this was "main"
and
main concluded with "success"

on:
  workflow_run:
    workflows: ["main"]
    types:
      - completed

  workflow_dispatch:

jobs:
  nanox:
    if: |
      github.event_name == 'workflow_dispatch' ||
      (github.event_name == 'workflow_run' &&
       github.event.workflow_run.conclusion == 'success')

Yes, the current version rebuilds ELKS from scratch.

Again, it is possible to update main as explained here: #2562 (comment) or try to improve on this version where 71b2998 the HDD minix is downloaded from "main" and then you compile microwindows only, but I could not get it to compile only the build tools required to build microwindows. But some things do work in 71b2998 and if you can figure out why the microwindows can not be built in this case (build tools are missing I think) then it might be perfect. Again ... all the patience in the world will be needed.