[Feature Request] Compare diff against live environments

[jgautheron]

Problem Statement

When ArgoCD manages critical system Helm charts (Istio, AWS Load Balancer Controller, etc.), enabling auto-sync means a merge deploys to production immediately. If that upgrade contains a bug, production is impacted right away.

At the scale of dozens of system services, issues are inevitable. Helm charts introduce bugs all the time—that's why there are constant patch releases. Auto-syncing production on every merge is simply not safe for infrastructure-critical components.

Some upgrades also require manual intervention. Keeping production sync manual ensures an infra engineer is actively watching the rollout and can intervene while the upgrade is ongoing.

To mitigate this risk, I follow a staged rollout process:

1. Merge deploys to non-production environments first
2. Evaluate stability of the upgrade in staging
3. Manually sync to production only after validation

The problem: The current argocd-diff-preview tool compares the PR branch against the base branch (e.g., main). This shows what would change if merged, but doesn't show what's actually deployed in a given environment. Since I don't auto-sync production, there can be different diff between main and what's live in production—making the base-vs-target diff misleading for production impact assessment.

Proposed Solution

Add a live comparison mode that compares the PR's rendered manifests against the actual live state fetched from a remote ArgoCD instance via its API.

New CLI flags:

• --compare-live - Enable live comparison mode (compare PR against remote ArgoCD instead of base branch)
• --live-argocd-url - URL of the remote ArgoCD instance (e.g., https://argocd-staging.example.com)
• --live-argocd-token - API token for authentication
• --live-argocd-insecure - Skip TLS verification (for self-signed certs)

Workflow:

1. Tool renders the PR branch manifests locally (as it does today)
2. Tool queries the remote ArgoCD API to fetch currently deployed manifests
3. Tool generates a diff between PR manifests and live state

Benefits:

• See exactly what will change relative to what's currently running in production
• Validate upgrades against actual deployed state, not just the Git base branch
• Support staged rollout workflows where production lags behind main

[madestreel]

This is a nice feature that we would definitely use :D

[dag-andersen]

Hi @madestreel,
Could you add a few words about why you'd like this feature and what value it brings? Are the benefits the same as in the feature request, or is there something specific to your use case? An example would be really helpful!

[dag-andersen]

@jgautheron

To mitigate this risk, I follow a staged rollout process:

1. Merge deploys to non-production environments first
2. Evaluate stability of the upgrade in staging
3. Manually sync to production only after validation

Just to make sure I understand correctly: Do both your non-prod and prod environments track the same files/applications, but with auto-sync disabled for prod? So when you upgrade a component, you merge to main, non-prod syncs automatically, and then you manually sync prod after validating in staging?

[jgautheron]

@dag-andersen Yes that's the use case.

[madestreel]

Hi @dag-andersen, yes of course.
We have the same pattern as @jgautheron (only a few more envs) and we never really sync from main, always from a tag (at least for the upper environments like qa/inhouse/staging/production). It is a big plus for operation to actually see what a new release is going to change on the cluster, especially when not all clusters are configured the same way (different cloud providers, timezones, different load that requires some tuning of the apps, etc).

But if it was only that I think we could manage with non live diff and comparing two tags. What makes it difficult for us is that we are using rancher to manage our clusters. On top of that we have ArgoCD that deploys a bunch of "infrastructure-apps" on new clusters. So we have an ApplicationSet with a matrix generator that generates for every cluster all required apps.

When doing the diff in a new ArgoCD cluster we do not have all those clusters, so no apps generated. Furthermore the cluster resources (secrets for argocd) have some labels that define the version of the infrastructure to deploy.

I hope this is more or less clear ;)