Change the default JNLP container image for kubernetes agents provisioning

Issue

  • I would like to modify the default jnlp container image used by pod templates

Environment

Explanation

By default, the kubernetes plugin uses the jenkins/inbound-agent public image.

In CloudBees CI:

  • Prior to version 2.204.1.3, the Kubernetes plugin default is used across all Masters.
  • Since version 2.204.1.3, the default image is set to cloudbees/cloudbees-core-agent by default across all masters.

However, enterprises might need to:

  • Use a Private Docker Registry
  • Use a different default JNLP image

In CloudBees Core - Modern cloud platforms, there are different approaches for changing the default jnlp container image for an instance:

  • (Recommended) Specify the default image via a system property
  • Define a default parent template (also known as “Default Provider Template”) to the Kubernetes Cloud(s)

Resolution

System Property Approach

The default jnlp container image can be set by adding the system property org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution.defaultImage=<fullImageNameAndTag> on startup. In fact, since version 2.204.1.3 CloudBees CI adds the system property org.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution.defaultImage=cloudbees/cloudbees-core-agent:<version> as a Master Global Java Options to enforce cloudbees/cloudbees-core-agent:<version> as the default jnlp container image to all Masters. And also provides a solution to change it.

With Helm

The default agent jnlp image can be set via the .Agents.Image.dockerImage attribute:

Agents:
  Image:
    # Change the following with full image name and tag such as `my.registry.com/cloudbees/cloudbees-core-agent:2.163.1.2`
    dockerImage: "<fullImageNameAndTag>"

Upon Operations Center restart, redeploy existing Managed Masters to apply the change.

With Helm prior to 2.204.1.3

For versions prior to 2.204.1.3, add -Dorg.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution.defaultImage=<fullImageNameAndTag> to the value of .Master.JavaOpts. Upon Operations Center restart, redeploy existing Managed Masters to apply the change.

With .yaml modifications

Add -Dorg.csanchez.jenkins.plugins.kubernetes.pipeline.PodTemplateStepExecution.defaultImage=<fullImageNameAndTag> to the MASTER_GLOBAL_JAVA_OPTIONS environment variable. Upon Operations Center restart, redeploy existing Managed Masters to apply the change.

Default Provider Template Approach

In a Kubernetes Cloud, there is an option called Defaults Provider Template Name, which provides a way of using this template as a parent to all other pod templates for that cloud. This can be used to customize the default jnlp container (not only the image). Here is an example:

modern-cloud-default-jnlp-image-default-provider.png

When defining a Defaults Provider Template Name in the kubernetes shared cloud in Operations Center, most templates defined in the Managed Masters inherits this parent template.

There are considerations to this approach:

  • the Defaults Provider Template Name in the kubernetes shared cloud applies to all pod templates defined under Manage Jenkins > Kubernetes Pod Templates or inside pipelines for the cloud named “kubernetes”. It does not apply to templates defined in a local Master Kubernetes cloud - i.e. a Kubernetes Cloud that is configured in the Managed Master global configuration. Each cloud has its own “Default Provider Template”.
  • this approach relies on Pod Template inheritance that must be understood. Especially around how YAML merge works

Due to those considerations, this is NOT the preferred approach. It may be a viable option in cases where a Master has different Kubernetes Clouds configured, and those clouds need different defaults for the jnlp container.

Further Notes

To disable the enforcement of the default jnlp container image in CloudBees CI Modern, the .Agents.Image.dockerImage attribute may be left empty. With this configuration, the kubernetes plugin defaults will be used:

Agents:
  Image:
    dockerImage: 

Upon Operations Center restart, redeploy existing Managed Masters to apply the change.

Tested products/plugins version

The latest update of this article was tested with:

References

Have more questions?

1 Comments

  • 0
    Avatar
    Dumitru Gutu

    This is very helpful in general but not helpful at all when you are using regular opensource Jenkins :(

    When you run on enterprise, you are not allowed to pull anything from outside, so how do I configure multiple registries based on the namespace in which the jenkins agent will run?

    Edited by Dumitru Gutu
Please sign in to leave a comment.