Cannot build images with Kaniko due to the sidecar injector

Issue

  • I am building an image using kaniko but it fails with an issue similar to:
Updating certificates in /etc/ssl/certs...
rm: cannot remove 'ca-certificates.crt': Device or resource busy

Environment

Explanation

Kaniko uses a very specific approach to build docker images that relies on the root file system. A particularity of the kaniko processor is that it automatically exclude directories that are mounted as volumes to the kaniko container environment. When using the sidecar injector, many volumes are automatically injected to the container pods. And this may impact kaniko’s build when images are doing operations under the mounted file systems, for example /etc/ssl/certs.

Resolution

The solution is to disable the sidecar injection for the pod by adding the annotation com.cloudbees.sidecar-injector/inject:no so that the certificates are not automatically injected

kind: Pod
metadata:
  name: kaniko
  annotations:
    com.cloudbees.sidecar-injector/inject: no
spec:
  containers:
  - name: jnlp
    workingDir: /home/jenkins
  - name: kaniko
    workingDir: /home/jenkins
    image: gcr.io/kaniko-project/executor:debug
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true

Disable injection for the kaniko container only

If the certificates are required for the other containers in the pod, the volumes may be injected manually to those containers. For example:

kind: Pod
metadata:
  annotations:
    com.cloudbees.sidecar-injector/inject: no
spec:
  containers:
  - name: jnlp
    volumeMounts:
      - name: ca-bundles
        mountPath: /etc/ssl/certs/ca-certificates.crt
        subPath: ca-certificates.crt
      - name: ca-bundles
        mountPath: /etc/ssl/certs/java/cacerts
        subPath: cacerts
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true
volumes:
  - name: ca-bundles
    configMap:
      defaultMode: 420
      name: ca-bundles
      
kind: Pod
metadata:
  name: kaniko
  annotations:
    com.cloudbees.sidecar-injector/inject: no
spec:
  containers:
  - name: jnlp
    workingDir: /home/jenkins
    volumeMounts:
      - name: ca-bundles
        mountPath: /etc/ssl/certs/ca-certificates.crt
        subPath: ca-certificates.crt
      - name: ca-bundles
        mountPath: /etc/ssl/certs/java/cacerts
        subPath: cacerts
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true
volumes:
  - name: ca-bundles
    configMap:
      defaultMode: 420
      name: ca-bundles
Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.