Issue

The issue can manifest when we have two Operation Center instances running in High Availability. The issue avoids the secondary node connects to the Cluster, showing the following error message in the Jenkins.log file of the secondary node during the connection:

SEVERE c.c.j.h.AbstractJenkinsSingleton#licenseCheck: /var/lib/jenkins-oc/license.xml not found or invalid. Bailing out from a standby role
java.lang.ClassNotFoundException: com.cloudbees.bouncycastle.v160.util.io.pem.PemReader
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
Caused: java.lang.NoClassDefFoundError: com/cloudbees/bouncycastle/v160/util/io/pem/PemReader
    at com.cloudbees.jenkins.ha.AbstractJenkinsSingleton.licenseCheck(AbstractJenkinsSingleton.java:418)
    at com.cloudbees.jenkins.ha.AbstractJenkinsSingleton.demote(AbstractJenkinsSingleton.java:380)
    at com.cloudbees.jenkins.ha.singleton.HASingleton.reactToPrimarySwitch(HASingleton.java:225)
    at com.cloudbees.jenkins.ha.singleton.HASingleton$3$1.run(HASingleton.java:135)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

The error is located in the cloudbees-ha plugin version 4.20 when the CloudBees Jenkins Operation Center version is 2.222.2.1 either 2.222.4.3.

Therefore, we will need to check if we are running any of those CloudBees Jenkins Operation Center versions and we have installed the cloudbees-ha plugin version 4.20.

Environment

• CloudBees Core on traditional platforms - Client controller
• CloudBees Core on traditional platforms - Operations Center
• CloudBees Jenkins Platform - Client controller
• CloudBees Jenkins Platform - Operations Center
• cloudbees-ha plugin.
• HA Architecture setup.

Explanation

The root cause of the error is the lcrypto-jdk15on-1.60-1.jar library is requested by any cloudbees-ha plugin version less or equal than 4.20 and in the CloudBees Jenkins Operation Center version 2.222.2.1 this library was replaced by the newer lcrypto-jdk15on:jar:1.64-1.jar library. Therefore, the cloudbees-ha plugin isn’t able to find the correct version of the lcrypto library.

The CloudBees Jenkins Operation Center version 2.235.1.2 fixes the issue. We can see the issue resolved in the Resolved issues page for CloudBees Jenkins Operation Center version 2.235.1.2.

Resolution

To fix this issue, upgrade to version 2.235.1.2 or newer, which will include the cloudbees-ha plugin version 4.21 (or newer) that has the fix to this issue.