Testing .github/workflows/test-report.yml in public repos

[brianjmurrell]

Hello.

Per your description for Recommended setup for public repositories how does the artifact: option know which triggering run to go fetch the artifacts from?

More to the point however, is do you have any advise on how one can test/debug their implementation of .github/workflows/test-report.yml in a PR -- without having to iterate on landing updates to make it work to the master branch?

I have tried adding on: -> workflow_dispatch to my .github/workflows/test-report.yml and then running the workflow manually with:

But that manual run doesn't find artifacts to report on:

likely because it has no linkage back to a .github/workflows/ci.yml that uploaded artifacts?

[brianjmurrell]

I suppose it's this (NPI):

test-reporting/src/input-providers/artifact-provider.ts

Lines 53 to 56 in fcf3649

	const resp = await this.octokit.actions.listWorkflowRunArtifacts({
	...github.context.repo,
	run_id: this.runId
	})

When run from a workflow_run trigger, I am guessing that this is the triggering workflow run but when run from a workflow_dispatch trigger, it is it's self.

Any thoughts on being able to make this action simulate a workflow_run trigger by being able to provide a run id value for

test-reporting/src/input-providers/artifact-provider.ts

Line 55 in fcf3649

run_id: this.runId

when triggered from a workflow_dispatch with an input perhaps?

Or am I barking up the completely wrong tree here in trying to test my .github/workflows/test-report.yml before merging it to the default branch and there is a better way?

[IanMoroney]

One feature I've just started using myself is to create the artifact and to run the report for that artifact in the same CI pipeline, so there's no issue with locating the required artifact.
When a PR is created, the CI pipeline triggers, it runs the tests, generates the xml files, upload the test results, and then reports on them.
You could also choose to not use an artifact, and to run the report on the local files, if they are available in the same run, as I do on line 42 of the below.
(ignore the uses: block, it's my internal CI way of using this action)
https://github.com/phoenix-actions/test-reporting/blob/main/.github/workflows/ci.yml

[brianjmurrell]

@IanMoroney AFAIU, your solution doesn't work for PR's from forks for public repositories (which is what this entire issue is about).

It also doesn't work for workflows that might take longer than 24h (i.e. on self-hosted runners) to complete execution because the GITHUB_TOKEN for the run expires. The only solution for that is the same as above, running the reporter in a workflow_run: workflow.

[IanMoroney]

You should be able to get around the token issue by specifying your own PAT token instead.

As for the PR's from forks issue, it's a github limitation. The PR doesn't need to be merged though in order to get a report.

Example:
A PR triggers a workflow called CI, and that generates an artifact.
A test-report workflow operating on a workflow_run basis with a completed type will run as soon as the CI workflow has completed.

[brianjmurrell]

Example: A PR triggers a workflow called CI, and that generates an artifact. A test-report workflow operating on a workflow_run basis with a completed type will run as soon as the CI workflow has completed.

Of course, that is how it works once all of the kinks have been worked out and every is merged so that it runs like that.

What I am asking about though is testing the test-report workflow, (i.e. to be sure it's working as expected) in a PR, before merging it (i.e. untested), such that it has an artifact in a CI run to operate on. The only way I know of to do that is by running it manually, as I describe above but as you see, that manual run cannot find any artifacts, I suspect because it does not have whatever internal reference to a CI run it would have if it had run from a workflow_run trigger rather than a workflow_dispatch trigger.