- How to set up a Load Balancer for CJOC and CM HA
- CloudBees Jenkins Operations Center (CJOC)
- Client Master (CM)
- Load Balancer
- The load balancer needs to be configured to load balance HTTP(S) traffic to the CloudBees Jenkins Instances which form the HA singleton.
- The Load Balancer needs to perform a health check on the Jenkins Instances on
/ha/health-check. Configure the load balancer to perform a health check by issuing a HEAD HTTP request on the nodes on
/ha/health-check. The primary node is the only one giving you
HTTP 200response code.
- The communication through JNLP port should not go through the load Balancer unless your company policies obligates you to configure JNLP through the load balancer.We should bypass the load balancer by adding the Java Argument in each OC instance
-Dhudson.TcpSlaveAgentListener.hostName=<OC_MACHINE_HOSTNAME>. If you don’t want to perform a restart, after adding the Java Argument you can execute
TcpSlaveAgentListener.CLI_HOST_NAME="CJOC_HOSTNAME"in Manage Jenkins -> Script Console in the Operations Center machine.
To check if the set-up was correctly performed you can run the command below and check that the header
X-Jenkins-CLI-Host is present with the value we added.
$curl -I http://cje-1.jenkins.example.com:8080/ HTTP/1.1 200 OK Date: Thu, 13 Jul 2017 11:49:29 GMT X-Content-Type-Options: nosniff Set-Cookie: JSESSIONID.7b40bd3d=hbc0fd3lhh0w1bg9cusqtea8r;Path=/;HttpOnly Expires: 0 Cache-Control: no-cache,no-store,must-revalidate X-Hudson-Theme: default Content-Type: text/html;charset=UTF-8 X-Hudson: 1.395 X-Jenkins: 22.214.171.124 X-Jenkins-Session: 1af0d556 X-Hudson-CLI-Port: 54789 X-Jenkins-CLI-Port: 54789 X-Jenkins-CLI2-Port: 54789 X-Jenkins-CLI-Host: cje-1.jenkins.example.com X-Frame-Options: sameorigin X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAlx9/b60USmo2Eexm8TBtkcG881tkAP77iBbct0gDlSLaDdhbP7thD86Mu3WQVxmq23Rzqd/WR2VpN+JaQCQYttQ/KdxqnkTPZFO4vYy7n9E0pWgw8VKVZ9WEPfwfDEf4tHm/NmT4i43YrG4oeJvrVrGpwDp7vBviZNzQqOlcxLsr6njOX3+aZMYGkSdkduKgtKB5tV5qJSQxNFw/3I9HuxSZlaWMyni9NhAs1jGJ7EaxKT3Jtix56EY+vIRY43YhLU6YvnU/O6R2H9KdmxyFjthr1H3x8Wjfo1qYqePIL73bjA5guRyZZirbKL+1ep9LjTqJZYpH3EUjGoiBtYeyiwIDAQAB Content-Length: 17904 Server: Jetty(9.2.z-SNAPSHOT)
Configure the “frontend” of the Load Balancer to route incoming HTTP(S) requests to the two CloudBees Jenkins Instances “alpha” and “beta” in the “backend”.
The combination of Hostname / VIP / Port must be unique for each of the cluster: that is, if the port is fixed (443 usually if using HTTPS) the VIP can be shared in the different clusters if the hostnames are different. The VIP is used to select the correct “backend” based on the Host header.
You can run the following script from Manage Jenkins->Script Console to check if headers are set properly:
println Jenkins.instance.rootUrlFromRequest println org.kohsuke.stapler.Stapler.currentRequest.getHeader('X-Forwarded-For') println org.kohsuke.stapler.Stapler.currentRequest.getHeader('X-Forwarded-Host') println org.kohsuke.stapler.Stapler.currentRequest.getHeader('X-Forwarded-Proto') println org.kohsuke.stapler.Stapler.currentRequest.getHeader('X-Forwarded-Port') println org.kohsuke.stapler.Stapler.currentRequest.serverPort println org.kohsuke.stapler.Stapler.currentRequest.serverName
As it was previously specified, CloudBees does not recommend to route the JNLP traffic unless it is a must coming from your company policies - or because OC and CMs are in different networks only reachable between them when there is a load balancer in the middle.
In case this is necessary, you need to ensure that the idle timeout is bigger than the PingThread timeout. By default, the PingThread timeout is 4 minutes, so a minimum timeout of 300 seconds is recommended. If the default PingThread was modified, then you need to modify the timeout in the load balancer accordingly.
In AWS there in a idle timeout that you can configure under Description -> Attributes -> Idle timeout. Ensure that this is configured at very least with 300 seconds.
Finally, ensure that under the Listeners section the JNLP port is configured to forward the packages using the protocol TCP.
If you are using a feature called OneConnection or something similar to improve the networking performance, please deactivate it for this environment as we have the experience it makes the connection to break.