cicd: backend migration actually requires external-secrets created in the step after

[davidcheung]

Note: On brand new build this doesnt happen, but can be a problem if people deploy one(staging) environment, do some development then go ahead and deploy their 2nd(prod) environment at a later date.

The existing flow for CI/CD pipeline deployment goes something like this

1. build image
2. check migration files
   2.1 if no migration files skip to deploy
   2.2 if there are migration files create migration job and waits for job to succeed
3. deploy (kustomize apply overlay/)

On a brand new repo during ci/cd, when there's no migration file it works, But if the first time running one already has migration files, the k8s environment does not have secret for migration job to mount, which would be created from external-secrets during deploy step.

[davidcheung]

basically the same reason as commitdev/zero-backend-go#72

[deven96]

How would you suggest going about it @davidcheung ?
If there are migration files but no k8s secret, then we deploy first and migrate after?

[davidcheung]

@deven96 Thanks so much!
I think the solution could be applying the external secret manifest before the deployments,
so it can deploy pre-requisites(kustomize) -> run migration -> deploy application(kustomize) then I think it should work in all scenarios
but separating the kustomize in 2 steps would increase some complexity, not sure if there's a better way.