Issue

• You need access to services using non-trusted CA (this can apply either to the CJE cluster itself, or accessing other services in the company)
• We want to customize docker images for our agents that uses the certificates added to the cluster

Environment

• CloudBees Jenkins Enterprise (CJE)
• CloudBees Jenkins Enterprise - Managed Master (CJE-MM)
• CloudBees Jenkins Enterprise - Operations Center (CJE-OC)

Resolution

When using customer docker agent templates in a Cluster that is set up with HTTPS / SSL, you need to provide the certificates added to your CJE cluster to your agents. This can be done by using the certs data container in the docker agent template configuration:

See also the article Customize a Docker Image to use it on Palace

The certs container

The certs container is a data-only container contains all the certificates that you added to your cluster with either the domain-name-change operation or the dna run certificates-update command. It is meant to have its volumes mounted to any container that requires the cluster certificates (agents, masters, …). The documentation is in cloudbees/docker-certificates.

The certificates added to the cluster are copied in /etc/certs/ssl in the certs container. The certificates used by java are located in /etc/certs/ssl/java/cacerts. This follows the structure used by ca-certificates, a tool used to manage certificates.

Therefore any container that is run from an image with ca-certificates installed and that has the certs volumes mounted is automatically set up with SSL in your cluster. For others, the image needs to be adapted.

Note: We recommend using certificates in the PEM format.

CloudBees Images

This works well with images based on the openjdk image - and that is the case for images that are used by CloudBees Jenkins Enterprise default installation - because this image has ca-certificates installed. Therefore the JDK is setup to look for certificates under /etc/ssl/certs/java/cacerts.

Custom Images

For images that have a custom JDK installation or are based on an image with a custom JDK installation, this may not work. A typical error would be:

[...]
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    at sun.security.validator.Validator.validate(Validator.java:260)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1351)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:156)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:925)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:860)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
    at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:728)
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
    at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:138)
    at SSLPoke.main(SSLPoke.java:31)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[...]

That’s because the default location that the JDK looks for to load certificates is $JAVA_HOME/jre/lib/security/cacerts.

In such cases, there are few solutions:

1) One solution is to install ca-certificates in the docker image
2) Another solution is to import the certificates in the JDK cacerts inside the docker image.