Compatibility with EKS 1.21 and token service account expiry

[tomsucho]

After our EKS was upgraded to 1.21, we saw annotations like the following appear in api server audit logs in AWS, for service accounts that kiam-server pods are using:

subject: system:serviceaccount::, seconds after warning threshold: 3989
This is due to changes in token expiry in K8s 1.21 as described here:
https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html#identify-pods-using-stale-tokens

It would appear that there is 90d grace period, after which tokens will be rejected.
It looks like the kiam server needs to use a later client SDK version, or is there a workaround?

[cloudwitch]

Talked to AWS support about this. They confirmed KIAM 4.2 has high enough Kubernetes Client SDK (v0.20.0) and is good to go from that perspective. This was a worry for us as we're on 3.6.

@tomsucho What KIAM version are you on?

[tomsucho]

@cloudwitch Thanks a lot for checking this! I was actually testing this based on the latest Helm chart which was installing v4.0 I think. And it was still showing up, the annotation. I think it was only reported for kiam-server and not the kiam-agent.
I used the Helm chart repo as shown on github:

NAME        	CHART VERSION	APP VERSION	DESCRIPTION                      
uswitch/kiam	6.1.2        	4          	Integrate AWS IAM with Kubernetes

[h2hoe]

@cloudwitch @tomsucho Is there a new version of the chart that needs to be released with the updated 4.2 version or will the 4.0 version suffice?

[tomsucho]

@h2hoe in my testing I could see on v4.0 annotations still showing up, so if that is really fixed in 4.2 would be good to get updated chart :)