Skip to content

gitops_bootstrap: userinfo ConfigMap discovery + cascade cleanup on destroy#62

Open
prakhar1985 wants to merge 3 commits intomainfrom
gitops-bootstrap-userinfo
Open

gitops_bootstrap: userinfo ConfigMap discovery + cascade cleanup on destroy#62
prakhar1985 wants to merge 3 commits intomainfrom
gitops-bootstrap-userinfo

Conversation

@prakhar1985
Copy link
Contributor

@prakhar1985 prakhar1985 commented Feb 7, 2026

What's in this PR

Two independent improvements to ocp4_workload_gitops_bootstrap, both needed for multi-tenant shared cluster labs.


1. Userinfo ConfigMap discovery

Enables the bootstrap role to read a demo.redhat.com/userinfo-labeled ConfigMap created by GitOps and report its data back to Babylon via agnosticd_user_info. This allows GitOps deployments to surface lab URLs, credentials, and other user-facing data without requiring a separate Ansible step.

How it works:

  • After ArgoCD syncs, the role searches for ConfigMaps with label demo.redhat.com/userinfo anywhere on the cluster
  • Values from data.users_json are imported as per-user data
  • Global data fields are imported as shared data
  • This was already written but commented out — this PR uncomments and wires it up

2. Cascade cleanup on destroy (resources-finalizer + remove_workload.yml)

Problem: remove_workload.yml was not implemented (msg: "Not implemented"), leaving bootstrap Applications and all their child Applications/resources on the cluster after destroy.

Fix:

  • Added resources-finalizer.argocd.argoproj.io to the bootstrap Application template
  • Implemented remove_workload.yml to delete the bootstrap Application and wait for ArgoCD to cascade

How cascade works:

remove_workload deletes bootstrap-tenant-{guid}
  → ArgoCD sees resources-finalizer
  → ArgoCD deletes child Applications (agent, librechat, mcp-gitea, etc.)
    → each child has resources-finalizer
    → ArgoCD deletes their K8s resources from namespaces
      → namespaces become empty → safe to delete

No custom ArgoCD cleanup code needed anywhere else. ArgoCD handles it all via the finalizer chain.

Destroy order in AgnosticV:

remove_workloads:
- rhpds.litellm_virtual_keys.ocp4_workload_litellm_virtual_keys  # revoke external keys first
- agnosticd.core_workloads.ocp4_workload_gitops_bootstrap          # cascade ArgoCD cleanup
- agnosticd.namespaced_workloads.ocp4_workload_tenant_gitea        # delete Gitea CR + user
- agnosticd.namespaced_workloads.ocp4_workload_tenant_namespace    # delete namespaces (now clean)
- agnosticd.namespaced_workloads.ocp4_workload_tenant_keycloak_user # delete RHBK user

Test plan

  • Deploy a lab using ocp4_workload_gitops_bootstrap with a tenant GitOps repo that creates a demo.redhat.com/userinfo ConfigMap — verify user data flows to Babylon
  • Run destroy — verify bootstrap-tenant-{guid} and all child Applications/resources are cleaned up automatically
  • Verify namespaces are empty before deletion (no ArgoCD sync conflicts)

Uncomment the demo.redhat.com/userinfo ConfigMap discovery block so
GitOps deployments can report user-facing data (URLs, credentials)
back to Babylon via agnosticd_user_info. Supports both global data
and per-user data via users_json.
@stencell
Copy link
Contributor

stencell commented Feb 9, 2026

@prakhar1985 Why do you want to re-enable this now? Has it been tested against CIs that deploy workloads where we need this data? The last time I spoke to Judd about this, he said it was disabled because it wasn't being used and wasn't well tested. If that has changed, please confirm that this has been validated and is functional.

@prakhar1985
Copy link
Contributor Author

prakhar1985 commented Feb 10, 2026

Tested via CI: https://catalog.demo.redhat.com/catalog?search=showroom+embed&item=babylon-catalog-dev%2Ftests.showroom-embed-test.dev

The showroom-embed-test catalog uses the gitops-bootstrap-userinfo branch and the userinfo ConfigMap discovery works -- Showroom URLs are reported back to the RHDP portal correctly.

Without this change, any catalog that uses ocp4_workload_gitops_bootstrap to deploy Showroom via ArgoCD can't surface lab URLs in the RHDP portal. The userinfo ConfigMap discovery code exists but is commented out on main, so the workload never reports URLs back to Babylon. Users have to manually find the Showroom route instead of clicking the link in the portal. We're currently pointing our catalog to this branch as a workaround.

@prakhar1985 prakhar1985 changed the title Enable userinfo ConfigMap discovery in gitops_bootstrap gitops_bootstrap: userinfo ConfigMap discovery + cascade cleanup on destroy Mar 3, 2026
Copy link
Contributor

@wkulhanek wkulhanek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment on config map.
Also needs merge conflicts removed, please.

# ansible.builtin.debug:
# msg: "{{ lookup('agnosticd_user_data', '*') }}"

- name: Retrieve ConfigMaps with the demo.redhat.com/userinfo label
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need some logic to wait until all the Application(Set)s are in Synced and Ready state? Wouldn't this otherwise run and never find the ConfigMap because it would be way too quick to check?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants