Updating Kubernetes plugin can break containerized builds which use custom Working Directory setting

Issue

After upgrading to CloudBees Core version 2.176.4.3 or newer, or Kubernetes plugin version 1.18.0 or later, your builds which use their own Docker containers start to fail with “permission denied” or similar errors accessing files within /home/jenkins.

Environment

Resolution

Kubernetes plugin version 1.18.0 changed the default Working Directory for agents from /home/jenkins to /home/jenkins/agent. This change was made because the Kubernetes plugin uses an empty directory to mount the working directory volume. Users who built their own containers and put data in the /home/jenkins directory couldn’t access that data because it was masked when the empty working dir was mounted on top of it. See [JENKINS-58705] for more on this issue.

If you are going to run builds in your own customized containers on top of these agents, you need to make sure your containers use the same working dir setting if it is set explicitly. Or, if those containers do not set a working dir for themselves, they should inherit the same setting from the agent container, and continue to work normally. Otherwise, these builds will likely be unable to run successfully.

Improvements to the Kubernetes plugin in version 1.22.1 may alleviate this issue. Refer to [PR-2187] for more on this.

Workaround

As a temporary solution, you can go to Manage Jenkins -> Configure System, look at the Container Template configuration, and set the Working Directory back to /home/jenkins. But this could expose you to the same data masking issues previously mentioned, which were the motivation for changing this default. For a long term solution, we recommend taking the steps described in the above Resolution section.

References

Have more questions?

0 Comments

Please sign in to leave a comment.