HA Failing Due To NoClassDefFoundError

Issue

HA can fail with many java.lang.NoClassDefFoundError related exceptions:

 Exception in thread "ConnectionMap.Acceptor [10.0.0.1:56736],null,null" java.lang.NoClassDefFoundError: org/jgroups/blocks/TCPConnectionMap$TCPConnection$Receiver
 at org.jgroups.blocks.TCPConnectionMap$TCPConnection.start(TCPConnectionMap.java:424)
 at org.jgroups.blocks.TCPConnectionMap$Acceptor.handleAccept(TCPConnectionMap.java:310)
 at org.jgroups.blocks.TCPConnectionMap$Acceptor.run(TCPConnectionMap.java:283)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.ClassNotFoundException: org.jgroups.blocks.TCPConnectionMap$TCPConnection$Receiver
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at winstone.classLoader.WebappClassLoader.loadClass(WebappClassLoader.java:83)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 4 more
 Apr 01, 2016 3:13:35 PM org.jgroups.util.TimeScheduler2$RecurringTask run
 SEVERE: failed running task STABLE: StableTask
 java.lang.NoClassDefFoundError: org/jgroups/protocols/pbcast/STABLE$1
 at org.jgroups.protocols.pbcast.STABLE.sendStableMessage(STABLE.java:687)
 at org.jgroups.protocols.pbcast.STABLE$StableTask.run(STABLE.java:829)
 at org.jgroups.util.TimeScheduler2$RecurringTask.run(TimeScheduler2.java:561)
 at org.jgroups.util.TimeScheduler2$MyTask.run(TimeScheduler2.java:493)
 at org.jgroups.util.TimeScheduler2$Entry.execute(TimeScheduler2.java:398)
 at org.jgroups.util.TimeScheduler2$1.run(TimeScheduler2.java:258)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.ClassNotFoundException: org.jgroups.protocols.pbcast.STABLE$1
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at winstone.classLoader.WebappClassLoader.loadClass(WebappClassLoader.java:83)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 9 more

The problem impacts older versions of CJE/CJOC and customers who use custom init scripts that probably don’t take the --webroot switch into account. This problem is happening because the default webroot data resides in the ${JENKINS_HOME}/war directory which is a default setting for single node installations of Jenkins.

Newer versions of CJE/CJOC include a --webroot switch, but older customers may need to make this adjustment manually if they want to use the HA features properly.

Environment

  • CloudBees Jenkins Enterprise
  • CloudBees Operations Center
  • Windows, Linux, Mac, BSD

Resolution

Adjust or add the webroot switch and point it ouside of ${JENKINS_HOME}

To resolve the problem, we advise adding --webroot to your init scripts and point it to a folder that exists outside of your shared ${JENKINS_HOME} data directory. Note that the webroot data needs to be stored and accessible on each individual node on it’s local filesystem, so avoid sharing it over NFS.

Quick example:

java -server -jar jenkins.war --webroot=/var/cache/jenkins/war

Make sure that the new folder you create for the webroot data has read/write permissions for your jenkins service account on each HA node.

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.