Separate regression tests into parallel CI job with continue-on-error

[Copilot]

Regression tests require significant work to pass and currently fail the entire CI, making it difficult to see core test status.

Changes

.github/workflows/ci.yml

• Split into two parallel jobs: build-and-test (core tests) and regression-tests
• Added continue-on-error: true to regression-tests job to prevent CI failures
• Removed run_pinesuite_tests workflow dispatch input (obsolete)
• Removed debug PINESUITE_TOKEN check step (obsolete)

packages/pine2ts/package.json

• Modified test script to exclude regression tests: vitest run --exclude='**/regression/**'
• Added test:all script to run complete test suite when needed

Result

Core tests (1496 total) now pass independently while regression tests still run for visibility but won't block CI. Both jobs execute in parallel.

Original prompt

Separate regression tests from main CI tests

Problem

The regression tests in the CI workflow require significant work to pass, and having them fail causes the entire CI to show as failed. This makes it difficult to see the status of the core tests.

Solution

Modify .github/workflows/ci.yml to:

1. Separate regression tests into their own job called regression-tests
2. Add continue-on-error: true to the regression tests job so it won't fail the overall CI status
3. Exclude regression tests from the main test run using --exclude="**/regression/**" or similar vitest pattern
4. Keep both jobs running in parallel for efficiency

Current State

The current ci.yml runs all tests together with pnpm test, which includes regression tests that may fail.

Desired State

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch:
    inputs:
      debug_enabled:
        description: 'Enable debug logging'
        required: false
        default: false
        type: boolean

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: pnpm/action-setup@v2
        with:
          version: 8
          
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'pnpm'
          
      - name: Install dependencies
        run: pnpm install
        
      - name: Build all packages
        run: pnpm build
        
      - name: Run unit tests (excluding regression)
        run: pnpm test -- --exclude="**/regression/**"
        env:
          PINESUITE_TOKEN: ${{ secrets.PINESUITE_TOKEN }}
        
      - name: Type check
        run: pnpm typecheck

  regression-tests:
    runs-on: ubuntu-latest
    continue-on-error: true
    steps:
      - uses: actions/checkout@v4
      
      - uses: pnpm/action-setup@v2
        with:
          version: 8
          
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'pnpm'
          
      - name: Install dependencies
        run: pnpm install
        
      - name: Build all packages
        run: pnpm build
        
      - name: Run regression tests
        run: pnpm test:regression
        env:
          PINESUITE_TOKEN: ${{ secrets.PINESUITE_TOKEN }}

Key Points

• The continue-on-error: true on the regression-tests job means it can fail without marking the overall CI as failed
• The main build-and-test job excludes regression tests so it focuses on core functionality
• Both jobs run in parallel for efficiency
• The existing test:regression script in package.json (cd packages/pine2ts && pnpm test:pinesuite) is used for the regression job
• Remove the now-unused run_pinesuite_tests workflow dispatch input since we're using continue-on-error instead

File to Modify

• .github/workflows/ci.yml

This pull request was created as a result of the following prompt from Copilot chat.

Separate regression tests from main CI tests

Problem

The regression tests in the CI workflow require significant work to pass, and having them fail causes the entire CI to show as failed. This makes it difficult to see the status of the core tests.

Solution

Modify .github/workflows/ci.yml to:

1. Separate regression tests into their own job called regression-tests
2. Add continue-on-error: true to the regression tests job so it won't fail the overall CI status
3. Exclude regression tests from the main test run using --exclude="**/regression/**" or similar vitest pattern
4. Keep both jobs running in parallel for efficiency

Current State

The current ci.yml runs all tests together with pnpm test, which includes regression tests that may fail.

Desired State

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch:
    inputs:
      debug_enabled:
        description: 'Enable debug logging'
        required: false
        default: false
        type: boolean

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: pnpm/action-setup@v2
        with:
          version: 8
          
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'pnpm'
          
      - name: Install dependencies
        run: pnpm install
        
      - name: Build all packages
        run: pnpm build
        
      - name: Run unit tests (excluding regression)
        run: pnpm test -- --exclude="**/regression/**"
        env:
          PINESUITE_TOKEN: ${{ secrets.PINESUITE_TOKEN }}
        
      - name: Type check
        run: pnpm typecheck

  regression-tests:
    runs-on: ubuntu-latest
    continue-on-error: true
    steps:
      - uses: actions/checkout@v4
      
      - uses: pnpm/action-setup@v2
        with:
          version: 8
          
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'pnpm'
          
      - name: Install dependencies
        run: pnpm install
        
      - name: Build all packages
        run: pnpm build
        
      - name: Run regression tests
        run: pnpm test:regression
        env:
          PINESUITE_TOKEN: ${{ secrets.PINESUITE_TOKEN }}

Key Points

• The continue-on-error: true on the regression-tests job means it can fail without marking the overall CI as failed
• The main build-and-test job excludes regression tests so it focuses on core functionality
• Both jobs run in parallel for efficiency
• The existing test:regression script in package.json (cd packages/pine2ts && pnpm test:pinesuite) is used for the regression job
• Remove the now-unused run_pinesuite_tests workflow dispatch input since we're using continue-on-error instead

File to Modify

• .github/workflows/ci.yml

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Copilot]

Pull request overview

This PR separates regression tests from core tests in the CI workflow to prevent failing regression tests from blocking the overall CI status. The changes introduce parallel test jobs where core tests must pass while regression tests run with continue-on-error: true.

Key Changes:

• Split CI into two parallel jobs: build-and-test (core tests) and regression-tests (allowed to fail)
• Modified the test script in pine2ts package to exclude regression tests from the default run
• Cleaned up obsolete workflow inputs and debug steps
• Fixed spacing issues in GitHub Actions secret references

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
.github/workflows/ci.yml	Added separate regression-tests job with continue-on-error: true, removed obsolete run_pinesuite_tests input and debug step, fixed spacing in secret references
packages/pine2ts/package.json	Modified test script to exclude regression tests, added test:all script to run complete test suite

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The single quotes in the --exclude argument may cause issues. In JSON strings, these quotes become part of the command and could be interpreted literally by the shell. Consider using double quotes with escaping instead: "test": "vitest run --exclude \"**/regression/**\"" or removing quotes entirely: "test": "vitest run --exclude **/regression/**"

Suggested change

	"test": "vitest run --exclude='**/regression/**'",
	"test": "vitest run --exclude=**/regression/**",