Issue
There are many operations in the CI cluster that pull images from
Docker Hub.
After Docker enabled a download rate limit,
I am facing issues downloading the images with errors like:
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Environment
- CloudBees CI (CloudBees Core)
- CloudBees CI (CloudBees Core) on modern cloud platforms - Managed Master
- CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
Resolution
There are 2 simple ways to increase the rate limits.
-
Free Account. If you use a free account of docker hub, the limit will be increased from 100 to 200.
-
Premium Account. You will have unlimited downloads
Independently on the kind of account you use (even if you decide to create your own registry), you will need to provide the OC and Master with the credentials.
Add your credentials to the cluster.
This can be done cluster wise using:
kubectl create secret generic regcred \
--from-file=.dockerconfigjson=<path/to/.docker/config.json> \
--type=kubernetes.io/dockerconfigjson
If you need to create a config.json you can run:
docker login
and a file ~/.docker/config.json
will be created once that you finish the login.
How to use the credentials
To be able to use the credentials we added to the cluster, we need to tell the pods to use them.
How to apply it to OC and MM:
You can update the values in the yaml with:
OperationsCenter:
ImagePullSecrets: regcred
Where the regcred
is the secret created before as example.
This setting will update the pods and the secret will be added.
How to apply it to the Agents:
Manually
You can update manually the pod templates adding the same secret, something like:
apiVersion: v1
kind: Pod
metadata:
name: private-reg
spec:
containers:
- name: private-reg-container
image: <your-private-image>
imagePullSecrets:
- name: regcred
As is explained in the k8s docs:
https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
Using the Service Account
There is an alternative way where you can define which credentials will be used by the Service Account that is going to pull the image, you can get more context here:
In a default installation the Service Account used is jenkins
. This might change if you have customized it, by default, the command:
kubectl patch serviceaccount jenkins -p '{"imagePullSecrets": [{"name": "regcred"}]}'
In a default installation using RBAC, you hace a chance to setup a Service Account that the agents will use in the cluster, if that is the scenario you have a chance to share the secret with the Service Account used by the agents jenkins-agents
.
kubectl patch serviceaccount jenkins-agents -p '{"imagePullSecrets": [{"name": "regcred"}]}'
By the time the article was wrote (14 of November of 2020) there was no solution for agents defined in Helm when RBAC is disabled.
0 Comments