What is this?

The gitops stack is the deployment infrastructure of the FAForever project. Currently it only contains a subset of service, but in the long term, all services shall be migrate from faf-stack to here.

Branching

• main branch = Production state
• develop branch = Testing state

Until things are settled the current workflow is:

• Test changes on develop
• Cherry pick working stuff on to main
• Rebase and force push develop afterwards

Maintenance / Release management

This repository and the cluster are setup to do the following:

• Keel is used to update mutable auto-update images (e.g. nodebb:4.4 updating to 4.4.x releases)
• Renovate is used to create PRs when image tags are outdated (for pinned versions as well as mutable versions)

Especially for 3rd party apps we try to use release-line tags. For our own apps we use pinned versions for apps that require combined deployments, for other apps we might use a release branch like main. In some cases only the test cluster points to rolling-release tags, while prod uses pinned release.

Attention: Renovate does not detect version changes if prod/dev use helm value based release tags.

Bootstrapping

First, we need to make the cluster gitops-ready by installing ArgoCD:

kubectl create namespace argocd
helm repo add argo https://argoproj.github.io/argo-helm
helm install argocd argo/argo-cd -n argocd

Then you can connect to argocd using the credentials generated:

PW=$(kubectl get secret -n argocd argocd-initial-admin-secret -o "jsonpath={.data.password}" | base64 -d)

echo "Access argocd via http://localhost:8080"
echo "User: admin, password: $PW"

kubectl port-forward -n argocd svc/argocd-server 8080:80

Now, we need to deploy the appsets via

kubectl apply -f app-set-test.yaml

Many apps will fail to start, because the lack the secrets that will be generated by infisical. But infiscal needs be setup too. We use the cloud edition, but there is also a self hosted one we do not cover here. For our stack you need to create a service token in the web ui and add this as a secret in all affected namespaces:

for namespace in "faf-apps faf-ops argocd"; do
kubectl create secret generic "infisical-service-token" \
    -n "$namespace" \
    --from-literal=infisicalToken=<your-token-here>
done

Local Development Using Tilt

This repository aims to provide a ready-to-go Tilt setup for ephemeral FAF infrastructure local development on kubernetes.

Prerequisites

• Tilt
• Kubernetes (Docker Desktop or Minikube are recommended for those new to kubernetes) Tilt cluster setup
• For Windows users a bash program. By default git bash is used with an assumed installation directory of C:/Program Files/Git
• helm must be installed to generate the k8s from the helm charts in the gitops repo

Startup Services

In the root directory of the repository run tilt up. This will start all the faf services in the correct order. The status of each service can be viewed in the tilt UI by visiting http://localhost:10350. This is the control server for tilt where you can restart services or disable them for substitution by services you would like to run from source code as you actively develop them.

Development

To develop against the FAF infrastructure you should disable the service in tilt that you are actively developing. Once disabled you can start up your developed version. Some tweaks may need to be made to the default configuration parameters in the source code. The proper values can be found in the configMaps in each of the services kubernetes deploy yaml files.

Test Data

The default test data that is loaded can be found in /sql/test-data.sql. This can be overridden by providing a new path with the tilt configuration key test-data-path when running tilt up or in the tilt_config.json file in the repository root directory.