Refreshing Kubernetes Docker Secrets via CronJobs

In order to pull private Docker images from ECR into Kubernetes, you need to authenticate with the Docker registry. The credentials can then be stored in a Docker config secret. However, the credentials in that secret will eventually expire, leading to failures when pulling. One solution to this problem is to periodically refresh credentials via a CronJob.

First, create a secret containing everything you need to authenticate with the Docker registry. The following example upserts a secret with AWS access key information, a default AWS region, and the location of the ECR registry. This example assumes that these values come from environment variables.

$ kubectl create secret generic ecr-secret-helper \
  --save-config \
  --dry-run=client \
  --from-literal=AWS_ACCESS_KEY_ID=$(AWS_ACCESS_KEY_ID) \
  --from-literal=AWS_SECRET_ACCESS_KEY=$(AWS_SECRET_ACCESS_KEY) \
  --from-literal=AWS_DEFAULT_REGION=$(AWS_DEFAULT_REGION) \
  --from-literal=ECR_REGISTRY=$(ECR_REGISTRY) \
  -o yaml | kubectl apply -f -

Next, create a CronJob that runs periodically. The following example runs once per hour. Depending on the configuration of your cluster, you may need to configure additional permissions via Roles, RoleBindings, etc.

apiVersion: batch/v1
kind: CronJob
metadata:
  name: ecr-secret-cron
spec:
  schedule: "0 * * * *"
  successfulJobsHistoryLimit: 1
  suspend: false
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: ecr-secret-cron-container
            image: alpine/k8s:1.27.4
            imagePullPolicy: IfNotPresent
            envFrom:
              - secretRef:
                  name: ecr-secret-helper
            command:
              - /bin/sh
              - -c
              - |-
                kubectl create secret docker-registry regcred \
                  --save-config \
                  --dry-run=client \
                  --docker-server=${ECR_REGISTRY} \
                  --docker-username=AWS \
                  --docker-password=`aws ecr get-login-password` \
                  -o yaml | kubectl apply -f -
          restartPolicy: Never

The CronJob shown above creates a pod and uses the previously created ecr-secret-helper secret to populate environment variables. The names of the environment variables were chosen to coordinate with those supported by the AWS CLI. If everything is configured properly, the regcred secret will be created/refreshed each time the job runs.

One final note - you can manually trigger the CronJob at any time using the following command:

kubectl create job --from=cronjob/ecr-secret-cron name-of-job