Implement FHIR IG-based FML Execution Validation Test Suite with GitHub Pages Publishing

.github/workflows/ghbuild.yml

@@ -0,0 +1,141 @@
+# This is a simple workflow that runs the publisher and copies the output to https://<owner>.github.io/<repo>/index.html
+# Based on the idea by Carl Leitner
+# Change log:
+# 2021-06-18: (JCT): publish default branches to / , other branches branches/<branch>
+# 2021-11-26: (JCT): Reusable workflow
+# 2022-01-28: (JCT): add auto-create gh-pages if it doesn't exist
+# 2023-01-22: (JCT): use checkout action v3, and JamesIves/github-pages-deploy-action@v4
+
+# Make sure your repo has a branch called gh-pages
+
+name: CI
+
+# Controls when the action will run. 
+on: 
+  workflow_call: # Reusable by other workflows
+    inputs:
+      tx:
+        required: false
+        type: string
+  # Triggers the workflow on push or pull request events for any branch
+  push:
+    branches-ignore:    
+      - 'gh-pages'
+  pull_request:
+
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+    inputs:
+      tx:
+        description: 'Optional Custom terminology server URL'
+        required: false
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  # This workflow contains a single job called "build"
+  build:
+    # The type of runner that the job will run on
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+    # Steps represent a sequence of tasks that will be executed as part of the job
+    steps:
+      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Get branch name
+        run: echo "BRANCH_NAME=${GITHUB_REF##*/}" >> $GITHUB_ENV
+
+      - name: Echo branch name and check if it's the default branch
+        run: |
+          echo "Current Branch: $BRANCH_NAME"
+          DEFAULT_BRANCH=$(git remote show origin | sed -n '/HEAD branch/s/.*: //p')
+          echo "Default Branch: $DEFAULT_BRANCH"
+          if [ "$BRANCH_NAME" == "$DEFAULT_BRANCH" ]; then
+            echo "This is the default branch."
+            echo "IS_DEFAULT_BRANCH=true" >> $GITHUB_ENV
+          else
+            echo "This is NOT the default branch."
+            echo "IS_DEFAULT_BRANCH=false" >> $GITHUB_ENV
+          fi
+
+      - name: Update the image to the latest publisher
+        uses: docker://hl7fhir/ig-publisher-base:latest
+        with:
+          # Get the latest publisher - don't run the batch script but run the line directly
+          args: curl -L https://github.com/HL7/fhir-ig-publisher/releases/latest/download/publisher.jar -o ./input-cache/publisher.jar --create-dirs
+
+      - name: Create package cache folder
+        uses: docker://hl7fhir/ig-publisher-base:latest
+        with:
+          entrypoint: /bin/sh
+          args: -c "mkdir -p ./fhir-package-cache && chown 1001:127 ./fhir-package-cache"
+
+      - name: Run the IG publisher
+        uses: docker://hl7fhir/ig-publisher-base:latest
+        with:
+          entrypoint: /bin/sh
+          args: -c "mkdir -p /var/lib/.fhir && chown $(id -u):$(id -g) /var/lib/.fhir"
+
+      - name: Run the IG publisher with optional tx
+        run: |
+          echo "TX input: ${{ inputs.tx }}"
+
+          CMD="java -Xmx6g -jar ./input-cache/publisher.jar publisher \
+            -ig . \
+            -auto-ig-build \
+            -repo https://github.com/${GITHUB_REPOSITORY}/tree/${GITHUB_REF_NAME} \
+            -package-cache-folder ./fhir-package-cache"
+
+          if [ ! -z "${{ inputs.tx }}" ]; then
+            CMD="$CMD -tx ${{ inputs.tx }}"
+          fi
+
+          echo "Running command: $CMD"
+
+          docker run --rm \
+            -v ${{ github.workspace }}:/work \
+            -w /work \
+            hl7fhir/ig-publisher-base:latest \
+            sh -c "$CMD"
+
+      # Additional step to upload qa.json as an artifact
+      - name: Upload qa.json artifact
+        if: success()
+        uses: actions/upload-artifact@v4
+        with:
+          name: qa-json-artifact
+          path: ./output/qa.json  # Adjust the path based on where qa.json is located
+
+      - name: Delete files >100MB before deployment
+        run: |
+          echo "Removing files over 100 MB from ./output..."
+          find ./output -type f -size +100M -print -delete
+
+      - name: Get branch name
+        run: echo "BRANCH_NAME=${GITHUB_REF##*/}" >> $GITHUB_ENV
+
+      - name: Deploy candidate
+        uses: JamesIves/github-pages-deploy-action@v4.4.2
+        if: env.IS_DEFAULT_BRANCH == 'false'
+        with:
+          branch: gh-pages # The branch the action should deploy to.
+          folder: ./output # The folder the action should deploy.
+          commit-message: Deploy candidate branch
+          target-folder: branches/${{ env.BRANCH_NAME }}
+          single-commit: true
+          clean: false 
+
+      - name: Deploy main
+        uses: JamesIves/github-pages-deploy-action@v4.4.2
+        if: env.IS_DEFAULT_BRANCH == 'true'
+        with:
+          branch: gh-pages # The branch the action should deploy to.
+          folder: ./output # The folder the action should deploy.
+          commit-message: Deploy main branch
+          single-commit: true
+          clean-exclude: |
+            branches
+            sitepreview

.gitignore

@@ -18,6 +18,13 @@ lib-cov
 coverage/
 *.lcov
 
+# FHIR IG generated files
+output/
+fsh-generated/
+temp/
+template/
+input-cache/
+
 # nyc test coverage
 .nyc_output
 

README.md

@@ -12,6 +12,19 @@ FML Runner is designed as a library component for larger application frameworks,
 4. **Process** FHIR Bundles for bulk resource operations
 5. **Provide** REST API endpoints with FHIR-compliant CRUD operations
 6. **Optimize** performance with intelligent caching and FHIRPath integration
+7. **Validate** FML implementations using a comprehensive test suite with real-world test cases
+
+## FML Execution Validation Test Suite
+
+This repository includes a comprehensive FHIR Implementation Guide-based test suite for validating FML execution. The test suite documentation and TestPlan resources are automatically published to GitHub Pages.
+
+**📚 [View Published Test Suite Documentation](https://litlfred.github.io/fmlrunner/)**
+
+The published documentation includes:
+- Complete test suite overview and usage instructions
+- Real-world test cases from community FML projects  
+- FHIR TestPlan resources for structured validation
+- License compliance documentation for all imported content
 
 ## Installation
 

TEST_SUITE_README.md

@@ -0,0 +1,204 @@
+# FML Execution Validation Test Suite
+
+A comprehensive FHIR Implementation Guide defining a test suite for validating FHIR Mapping Language (FML) execution using real-world test cases.
+
+**📚 [View Published Documentation](https://litlfred.github.io/fmlrunner/)** - Complete test suite with FHIR TestPlan resources
+
+## Overview
+
+This test suite provides a standardized way to validate FML implementations using:
+- Real-world test cases from the ahdis/matchbox project
+- Official FHIR test cases from FHIR/fhir-test-cases repository
+- FHIR TestPlan resources for structured test execution
+- Comprehensive licensing compliance for all imported content
+
+The test suite is automatically built and published as a FHIR Implementation Guide to GitHub Pages, making the TestPlan resources and documentation easily accessible for implementers.
+
+## Quick Start
+
+### Prerequisites
+
+- [SUSHI](https://fshschool.org/docs/sushi/) (FHIR Shorthand IG generator)
+- [FHIR IG Publisher](https://confluence.hl7.org/display/FHIR/IG+Publisher+Documentation)
+- Node.js (for test data import scripts)
+
+### Building the Implementation Guide
+
+1. **Clone the repository**
+   ```bash
+   git clone https://github.com/litlfred/fmlrunner.git
+   cd fmlrunner
+   ```
+
+2. **Generate the IG**
+   ```bash
+   # Compile FSH files
+   sushi
+
+   # Generate the full IG
+   _genonce.sh  # or .bat on Windows
+   ```
+
+3. **Import additional test data** (optional)
+   ```bash
+   node scripts/test-data-import/import-all.js
+   ```
+
+## Directory Structure
+
+```
+input/
+├── fsh/
+│   └── tests/
+│       └── FMLExecutionValidationTestPlan.fsh
+├── examples/
+│   ├── matchbox/
+│   │   ├── qr2patgender/
+│   │   │   ├── qr2patgender.map
+│   │   │   ├── qr-input.json
+│   │   │   └── patient-output.json
+│   │   └── test-cases-metadata.json
+│   ├── fhir-test-cases/
+│   │   ├── tutorial-step1/
+│   │   │   ├── tutorial-step1.map
+│   │   │   └── tutorial-input.json
+│   │   └── test-cases-metadata.json
+│   ├── QuestionnaireResponse-qr-sample.json
+│   ├── Patient-patient-sample.json
+│   └── Bundle-tutorial-input.json
+└── pagecontent/
+    ├── index.md
+    ├── test-suite.md
+    ├── test-data.md
+    └── license-compliance.md
+```
+
+## Test Data Import
+
+The test suite includes scripts to import test data from external repositories:
+
+### Import from ahdis/matchbox
+```bash
+node scripts/test-data-import/import-matchbox.js
+```
+
+### Import from FHIR/fhir-test-cases
+```bash
+node scripts/test-data-import/import-fhir-test-cases.js
+```
+
+### Import all sources
+```bash
+node scripts/test-data-import/import-all.js
+```
+
+## Using the Test Suite
+
+### For FML Engine Implementers
+
+1. **Load the TestPlan**: Import the `FMLExecutionValidationTestPlan` resource
+2. **Execute Test Cases**: Run each test case against your FML engine
+3. **Validate Results**: Compare outputs against expected results
+4. **Check Assertions**: Evaluate FHIRPath assertions for pass/fail status
+
+### For Test Suite Maintainers
+
+1. **Add Test Data**: Place new test files in appropriate directories
+2. **Update Metadata**: Modify metadata JSON files to include new test cases
+3. **Regenerate TestPlan**: Run import scripts to update the FSH TestPlan
+4. **Rebuild IG**: Use SUSHI and IG Publisher to generate updated documentation
+
+## Test Case Structure
+
+Each test case includes:
+
+- **Map File**: FML mapping specification (`.map` or `.fml`)
+- **Input File**: Source FHIR resource (JSON or XML)
+- **Output File**: Expected transformation result (when available)
+- **Assertions**: FHIRPath expressions for validation
+
+### Example Test Case
+
+```
+input/examples/matchbox/qr2patgender/
+├── qr2patgender.map          # FML mapping
+├── qr-input.json            # Input QuestionnaireResponse
+└── patient-output.json      # Expected Patient output
+```
+
+## License Compliance
+
+The test suite incorporates content from multiple sources:
+
+- **ahdis/matchbox**: Apache License 2.0
+- **FHIR/fhir-test-cases**: HL7 FHIR License
+
+All imported files include proper attribution headers. See [License Compliance](input/pagecontent/license-compliance.md) for details.
+
+## Contributing
+
+To contribute to the test suite:
+
+1. **Add Test Cases**: Include proper licensing headers
+2. **Update Documentation**: Describe new test scenarios
+3. **Maintain Attribution**: Preserve all license information
+4. **Test Changes**: Verify IG builds successfully
+
+## File Naming Conventions
+
+Test files follow these patterns:
+- Map files: `*.map`, `*.fml`, `*-map.txt`
+- Input files: `*-input.json`, `*-input.xml`, `*source*`
+- Output files: `*-output.json`, `*-output.xml`, `*-expected.*`
+
+## Building and Testing
+
+### Build the IG
+```bash
+# Quick build
+sushi
+
+# Full build with publisher
+./_genonce.sh
+```
+
+### Validate FSH
+```bash
+sushi -s
+```
+
+### Validate structure
+```bash
+npm run test-suite:validate
+```
+
+### Update test data
+```bash
+node scripts/test-data-import/import-all.js
+sushi
+```
+
+## Support
+
+For questions or issues:
+- Review the [Published Test Suite Documentation](https://litlfred.github.io/fmlrunner/) 
+- Check existing [GitHub Issues](https://github.com/litlfred/fmlrunner/issues)
+- Create a new issue for bugs or feature requests
+
+## Published Documentation
+
+The FML Execution Validation Test Suite is automatically built and published to GitHub Pages:
+
+**🔗 [https://litlfred.github.io/fmlrunner/](https://litlfred.github.io/fmlrunner/)**
+
+The published documentation includes:
+- Interactive FHIR TestPlan resources
+- Complete test case specifications
+- Implementation guidance for FML engines
+- License compliance information
+
+## License
+
+This project is licensed under the MIT License. See [LICENSE](LICENSE) for details.
+
+Individual test cases are licensed under their original terms (Apache 2.0 or HL7 FHIR License) as documented in each file.