-
Notifications
You must be signed in to change notification settings - Fork 675
Add AzDO pipeline to bump decoupled deps and update API docs after publish #5661
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+340
−0
Merged
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
d073e94
Add AzDO pipeline to bump decoupled local dependencies after publish
iclanton f934b71
Fix GitHub PR step to show API error responses on failure
iclanton e9dabf8
Use service connection credentials from git config for GitHub API calls
iclanton f206d7d
Extract push + GitHub PR steps into a reusable template
iclanton c384780
Address PR review: sanitize inputs, protect credentials, add security…
iclanton 0ac60c3
Rename to npm-post-publish, publish api artifact, add API docs update…
iclanton 86170f5
Rename to npm-post-publish, add api artifact, add API docs update sta…
iclanton 1e5760c
Extract api artifact publishing to separate branch
iclanton b7045fa
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 8d8bac6
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 8c803ef
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 63137c0
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton a33d3ce
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 1652350
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 10dade8
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 2797344
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 0ac30e7
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 728b415
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 59b9867
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 8963a0e
Only install to repo-toolbox.
iclanton 7bbe316
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton d5645af
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton a204ddd
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton b4be874
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton 4b5069d
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton ba040aa
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton d3bb564
fixup! Rename to npm-post-publish, add api artifact, add API docs upd…
iclanton f62258a
TEMP: download api artifact from latest npmPublish for testing
iclanton a4467e5
Use api-documenter-docusaurus-plugin.
iclanton 43b9ba8
fixup! Use api-documenter-docusaurus-plugin.
iclanton File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,221 @@ | ||
| parameters: | ||
| - name: delayMinutes | ||
| displayName: 'Minutes to wait for packages to propagate before running' | ||
| type: number | ||
| default: 5 | ||
|
|
||
| name: 'Post-publish $(Date:yyyyMMdd).$(Rev:r) (triggered by $(resources.triggeringAlias))' | ||
|
|
||
| variables: | ||
| - name: FORCE_COLOR | ||
| value: 1 | ||
|
|
||
| # This pipeline is triggered only by pipeline resources (npm publish pipelines), | ||
| # not by CI pushes or PR builds. | ||
| trigger: none | ||
| pr: none | ||
|
|
||
| resources: | ||
| pipelines: | ||
| - pipeline: npmPublish | ||
| source: 'rushstack NPM Publish' | ||
| trigger: | ||
| enabled: true | ||
| branches: | ||
| include: | ||
| - refs/heads/main | ||
| - pipeline: npmPublishRush | ||
| source: 'rushstack NPM Publish (rush)' | ||
| trigger: | ||
| enabled: true | ||
| branches: | ||
| include: | ||
| - refs/heads/main | ||
| repositories: | ||
| - repository: 1esPipelines | ||
| type: git | ||
| name: 1ESPipelineTemplates/1ESPipelineTemplates | ||
| ref: refs/tags/release | ||
| - repository: rushstackWebsites | ||
| type: github | ||
| name: microsoft/rushstack-websites | ||
| endpoint: GitHubProjects | ||
| ref: refs/heads/main | ||
|
|
||
| extends: | ||
| template: v1/1ES.Official.PipelineTemplate.yml@1esPipelines | ||
| parameters: | ||
| sdl: | ||
| sourceRepositoriesToScan: | ||
| exclude: | ||
| - repository: rushstackWebsites | ||
| pool: | ||
| name: Azure-Pipelines-1ESPT-ExDShared | ||
| os: windows | ||
| stages: | ||
| # ────────────────────────────────────────────────────────────────────────── | ||
| # Stage 0: Wait for packages to propagate to the npm registry | ||
| # ────────────────────────────────────────────────────────────────────────── | ||
| - stage: WaitForPropagation | ||
| displayName: 'Wait for npm propagation' | ||
| jobs: | ||
| - job: | ||
| displayName: 'Delay' | ||
| pool: server | ||
| timeoutInMinutes: 120 | ||
| steps: | ||
| - task: Delay@1 | ||
| displayName: 'Wait ${{ parameters.delayMinutes }} minute(s)' | ||
| inputs: | ||
| delayForMinutes: '${{ parameters.delayMinutes }}' | ||
|
|
||
| # ────────────────────────────────────────────────────────────────────────── | ||
| # Stage 1: Bump decoupled local dependencies | ||
| # ────────────────────────────────────────────────────────────────────────── | ||
| - stage: BumpDecoupledDeps | ||
| displayName: 'Bump decoupled local dependencies' | ||
| dependsOn: WaitForPropagation | ||
| variables: | ||
| BranchName: 'automated/bump-decoupled-deps' | ||
| CommitMessage: 'chore: bump decoupled local dependencies' | ||
| jobs: | ||
| - job: | ||
| displayName: 'Bump decoupled dependencies and create PR' | ||
| pool: | ||
| name: publish-rushstack | ||
| os: linux | ||
| steps: | ||
| - checkout: self | ||
| persistCredentials: true | ||
|
|
||
| - template: /common/config/azure-pipelines/templates/install-node.yaml@self | ||
|
|
||
| - script: 'git config --local user.email rushbot@users.noreply.github.com' | ||
| displayName: 'git config email' | ||
|
|
||
| - script: 'git config --local user.name Rushbot' | ||
| displayName: 'git config name' | ||
|
|
||
| - script: 'node common/scripts/install-run-rush.js install --to repo-toolbox' | ||
| displayName: 'Rush Install' | ||
|
|
||
| - script: 'node common/scripts/install-run-rush.js build --to repo-toolbox --verbose' | ||
| displayName: 'Rush Build (repo-toolbox)' | ||
|
|
||
| - script: 'node repo-scripts/repo-toolbox/lib-commonjs/start.js bump-decoupled-local-dependencies' | ||
| displayName: 'Bump decoupled local dependencies' | ||
|
|
||
| - script: 'node common/scripts/install-run-rush.js update' | ||
| displayName: 'Rush Update' | ||
|
|
||
| - bash: | | ||
| set -e | ||
|
|
||
| if git diff --quiet; then | ||
| echo "No changes detected. Skipping commit and PR." | ||
| echo "##vso[task.setvariable variable=HasChanges]false" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "##vso[task.setvariable variable=HasChanges]true" | ||
|
|
||
| git checkout -B $(BranchName) | ||
| git add --all | ||
| git commit -m "$(CommitMessage)" | ||
| displayName: 'Commit dependency changes' | ||
|
|
||
| - bash: | | ||
| set -e | ||
|
|
||
| node common/scripts/install-run-rush.js change \ | ||
| --bulk \ | ||
| --bump-type none \ | ||
| --commit-message "chore: generate change files for decoupled dependency bump" | ||
| displayName: 'Generate change files' | ||
| condition: and(succeeded(), eq(variables.HasChanges, 'true')) | ||
|
|
||
| - template: /common/config/azure-pipelines/templates/push-and-create-github-pr.yaml@self | ||
| parameters: | ||
| BranchName: $(BranchName) | ||
| PrTitle: $(CommitMessage) | ||
| PrDescription: 'Automated PR to bump decoupled local dependencies to the latest published versions.' | ||
|
|
||
| # ────────────────────────────────────────────────────────────────────────── | ||
| # Stage 2: Update API documentation on rushstack-websites | ||
| # ────────────────────────────────────────────────────────────────────────── | ||
| - stage: UpdateApiDocs | ||
| displayName: 'Update API documentation' | ||
| dependsOn: WaitForPropagation | ||
| variables: | ||
| BranchName: 'automated/update-api-docs' | ||
| CommitMessage: 'docs: update API documentation' | ||
| jobs: | ||
| - job: | ||
| displayName: 'Update API docs and create PR' | ||
| pool: | ||
| name: publish-rushstack | ||
| os: linux | ||
| steps: | ||
| - checkout: rushstackWebsites | ||
| persistCredentials: true | ||
|
|
||
| - template: /common/config/azure-pipelines/templates/install-node.yaml@self | ||
| parameters: | ||
| NodeMajorVersion: 24 | ||
|
|
||
| # Build the custom Docusaurus plugin for api-documenter in the | ||
| # rushstack-websites repo. | ||
| - script: 'node common/scripts/install-run-rush.js install' | ||
| displayName: 'Rush Install (rushstack-websites)' | ||
|
|
||
| - script: 'node common/scripts/install-run-rush.js build --to-except api.rushstack.io --verbose' | ||
| displayName: 'Rush Build to-except api.rushstack.io (rushstack-websites)' | ||
|
|
||
| # Download the api artifact from the triggering publish pipeline. | ||
| # AzDO automatically resolves which pipeline resource triggered this run. | ||
| - task: DownloadPipelineArtifact@2 | ||
| displayName: 'Download API review files' | ||
| inputs: | ||
| source: specific | ||
| project: GitHubProjects | ||
| pipeline: 'rushstack NPM Publish' | ||
| runVersion: latest | ||
| artifact: api | ||
| path: $(Pipeline.Workspace)/api | ||
|
|
||
| # Run api-documenter with the Docusaurus plugin from the | ||
| # api.rushstack.io project directory so it picks up | ||
| # config/api-documenter.json. | ||
| - script: 'npx @microsoft/api-documenter@latest generate --input-folder $(Pipeline.Workspace)/api --output-folder ./docs/pages' | ||
| displayName: 'Generate API documentation' | ||
| workingDirectory: websites/api.rushstack.io | ||
|
|
||
| # Update the API docs folder in rushstack-websites and commit. | ||
| - bash: | | ||
| set -e | ||
|
|
||
| git config --local user.email rushbot@users.noreply.github.com | ||
| git config --local user.name Rushbot | ||
|
|
||
| # Move the generated nav data file to the expected location. | ||
| mv websites/api.rushstack.io/docs/api_nav.json websites/api.rushstack.io/data/api_nav.json | ||
|
|
||
| # Check for changes (tracked and untracked) | ||
| if git diff --quiet && [ -z "$(git ls-files --others --exclude-standard)" ]; then | ||
| echo "No API documentation changes detected." | ||
| echo "##vso[task.setvariable variable=HasChanges]false" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "##vso[task.setvariable variable=HasChanges]true" | ||
|
|
||
| git checkout -B $(BranchName) | ||
| git add --all | ||
| git commit -m "$(CommitMessage)" | ||
| displayName: 'Update API docs and commit' | ||
|
|
||
| - template: /common/config/azure-pipelines/templates/push-and-create-github-pr.yaml@self | ||
| parameters: | ||
| BranchName: $(BranchName) | ||
| PrTitle: $(CommitMessage) | ||
| PrDescription: 'Automated PR to update API reference documentation from the latest published packages.' |
119 changes: 119 additions & 0 deletions
119
common/config/azure-pipelines/templates/push-and-create-github-pr.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| parameters: | ||
| - name: BranchName | ||
| type: string | ||
| - name: PrTitle | ||
| type: string | ||
| - name: PrDescription | ||
| type: string | ||
| default: '' | ||
| - name: TargetBranch | ||
| type: string | ||
| default: 'main' | ||
| - name: HasChangesVariableName | ||
| type: string | ||
| default: 'HasChanges' | ||
| - name: WorkingDirectory | ||
| type: string | ||
| default: '$(Build.SourcesDirectory)' | ||
|
|
||
| steps: | ||
| # Force-push the branch. This is safe because the branch (e.g. "automated/bump-decoupled-deps") | ||
| # is exclusively owned by this pipeline and is never manually committed to. | ||
| - bash: | | ||
| set -e | ||
| git push origin ${{ parameters.BranchName }} --force | ||
| displayName: 'Push branch' | ||
| condition: and(succeeded(), eq(variables['${{ parameters.HasChangesVariableName }}'], 'true')) | ||
| workingDirectory: ${{ parameters.WorkingDirectory }} | ||
|
|
||
| - bash: | | ||
iclanton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| set -e | ||
|
|
||
| # ── Resolve the GitHub owner/repo from the git remote URL ── | ||
| # Handles both HTTPS (https://github.com/owner/repo.git) and SSH (git@github.com:owner/repo.git) URLs. | ||
| REPO_SLUG=$(git remote get-url origin | sed -E 's#.*github\.com[:/](.+/[^.]+)(\.git)?$#\1#') | ||
| echo "Repository: ${REPO_SLUG}" | ||
| OWNER=$(echo "${REPO_SLUG}" | cut -d/ -f1) | ||
|
|
||
| # ── Extract credentials from the AzDO-managed git config ── | ||
| # When "persistCredentials: true" is set on the checkout step, AzDO injects an | ||
| # "http.<url>.extraheader" git config entry containing an "AUTHORIZATION: basic <token>" | ||
| # header for the GitHub service connection. We reuse this for GitHub API calls so that | ||
| # no additional secrets or PATs need to be configured. | ||
| AUTH_HEADER=$(git config --get-regexp 'http\..*\.extraheader' | head -1 | sed 's/^[^ ]* //') | ||
| if [ -z "$AUTH_HEADER" ]; then | ||
| echo "##[error]Could not extract authorization header from git config. Ensure persistCredentials is enabled on the checkout step." | ||
| exit 1 | ||
| fi | ||
|
|
||
| # ── Write credentials to a temporary curl config file ── | ||
| # This avoids passing the auth token as a command-line argument, which would be | ||
| # visible in process listings (e.g. "ps aux") and could leak into logs. | ||
| CURL_CONFIG=$(mktemp) | ||
| trap 'rm -f "$CURL_CONFIG"' EXIT | ||
| echo "-H \"${AUTH_HEADER}\"" > "$CURL_CONFIG" | ||
| echo '-H "Accept: application/vnd.github+json"' >> "$CURL_CONFIG" | ||
|
|
||
| API_BASE="https://api.github.com/repos/${REPO_SLUG}" | ||
|
|
||
| # ── GitHub API helper ── | ||
| # Calls the GitHub API using the temporary curl config file for auth headers. | ||
| # On success (2xx), prints the response body to stdout. | ||
| # On failure, prints the HTTP status and error body to stderr and returns non-zero. | ||
| github_api() { | ||
| local RESPONSE HTTP_CODE BODY | ||
| RESPONSE=$(curl -s -w "\n%{http_code}" -K "$CURL_CONFIG" "$@") | ||
| HTTP_CODE=$(echo "$RESPONSE" | tail -n1) | ||
| BODY=$(echo "$RESPONSE" | sed '$d') | ||
|
|
||
| if [[ "$HTTP_CODE" -ge 200 && "$HTTP_CODE" -lt 300 ]]; then | ||
| echo "$BODY" | ||
| else | ||
| echo "##[error]GitHub API returned HTTP ${HTTP_CODE}:" >&2 | ||
| echo "$BODY" >&2 | ||
| return 1 | ||
| fi | ||
| } | ||
|
|
||
| # ── Check for an existing open PR from this branch ── | ||
| # The GitHub "List pull requests" API filters by "head=OWNER:BRANCH" to find any | ||
| # open PR already targeting this branch. If one exists, we update it instead of | ||
| # creating a duplicate. | ||
| EXISTING_PR=$(github_api \ | ||
| "${API_BASE}/pulls?head=${OWNER}:${{ parameters.BranchName }}&state=open" \ | ||
| | jq '.[0].number // empty') | ||
iclanton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| if [ -n "$EXISTING_PR" ]; then | ||
| # ── Update existing PR ── | ||
| # Only the description is updated; the title is left as-is since the branch was | ||
| # already force-pushed with the new commits above. | ||
| echo "Updating existing PR #${EXISTING_PR}" | ||
| github_api -X PATCH \ | ||
| "${API_BASE}/pulls/${EXISTING_PR}" \ | ||
| -d "$(jq -n --arg body "$PR_BODY" '{body: $body}')" > /dev/null | ||
| else | ||
| # ── Create new PR ── | ||
| # jq --arg safely handles JSON escaping of the title and body, so special | ||
| # characters (quotes, newlines, etc.) in the parameter values are safe. | ||
| echo "Creating new PR" | ||
| github_api -X POST \ | ||
| "${API_BASE}/pulls" \ | ||
| -d "$(jq -n \ | ||
| --arg title "$PR_TITLE" \ | ||
iclanton marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| --arg body "$PR_BODY" \ | ||
| --arg head "${{ parameters.BranchName }}" \ | ||
| --arg base "${{ parameters.TargetBranch }}" \ | ||
| '{title: $title, body: $body, head: $head, base: $base}')" > /dev/null | ||
| fi | ||
| displayName: 'Create or update GitHub PR' | ||
| condition: and(succeeded(), eq(variables['${{ parameters.HasChangesVariableName }}'], 'true')) | ||
| # Pass PR title and description as environment variables rather than using | ||
| # ${{ }} template expansion inside the script. Template expansion would | ||
| # substitute the raw string into the Bash source code, which breaks if the | ||
| # value contains quotes or other shell metacharacters. Environment variables | ||
| # are set by the AzDO agent outside of the shell, so they are safe regardless | ||
| # of content. | ||
| workingDirectory: ${{ parameters.WorkingDirectory }} | ||
| env: | ||
| PR_TITLE: ${{ parameters.PrTitle }} | ||
| PR_BODY: ${{ parameters.PrDescription }} | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.