How to troubleshoot HA issues on CloudBees Jenkins

Issue

  • How to troubleshoot HA issues on CloudBees Jenkins

Environment

  • CloudBees Jenkins Enterprise
  • CloudBees Jenkins Operations Center
  • High Availability Plugin

Resolution

This article gives simple steps for troubleshooting a High Availability installations. It also gives steps to gather the most relevant information so that the Support Team to resolve your issue as quickly as possible.

Try to bind jgroups with the corresponded interface

Firstly, add the following Java arguments in BOTH instances (where <IP_ADDRESS> is the IP address of the network interface on the instance itself which should be able to reach out the other instance):

-Djgroups.bind_addr=<IP_ADDRESS>
-Djava.net.preferIPv4Stack=true

Then, restart and give a new try. If it didn’t work, collect some information with the following steps.

Collect some information about your instance

Run troubleshooting application to get more details

Download the latest version of our troubleshooter. You need to go to both instances and run the following command (Please, change $JENKINS_HOME for the corresponding value):

java -DJENKINS_HOME=$JENKINS_HOME -DHA_JGROUPS_DIR=$JENKINS_HOME/jgroups/ -jar troubleshooter-4.7-SNAPSHOT-jar-with-dependencies.jar

Check if one instance start as primary node and the other one as backup. If it doesn’t happen - it is expected since ha doesn’t work on the real instances - then perform the command below on both instances and attach the input in two different .txt files.

java -DJENKINS_HOME=$JENKINS_HOME -DHA_JGROUPS_DIR=$JENKINS_HOME/jgroups/ -Dlogging.org.jgroups=ALL -Dlogging.com.cloudbees.jenkins.ha=ALL -jar troubleshooter-4.7-SNAPSHOT-jar-with-dependencies.jar

What should I attach in a CloudBees support ticket?

Gather the following information and attach it to a the support ticket:

  1. Is there any firewall in the middle of both nodes?
  2. Do you have several network interfaces on those instances?
  3. Support bundles from both CJOC and CJE
  4. The content inside of $JENKINS_HOME/jgroups/
  5. The .txt files produced in the above step

Recommendations

Ensure that:

  • All instances should run the same Jenkins version
  • All instances should run using the same JDK version

More Information

Please have a look at the following requirements and troubleshooting pages:

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.