Troubleshooting CloudBees Core for Modern Cloud Platforms Part 1 - Operations Center issues

Issue

Installed CloudBees Core on Kubernetes but I’m having issues accessing the UI.

Environment

Resolution

For your browser to access the CloudBees Core UI (i.e., Jenkins) requires an HTTP/S request to go through several layers to reach the pod hosting your new Operations Center. To troubleshoot those layers I would start with the inner layer and work our way outward. First start with the pod, then with the service and finally ingress.

  1. Start with the pod and perform a HTTP request locally,

kubectl exec -it cjoc-0 -- curl -D- http://localhost:8080/cjoc

Example output,

HTTP/1.1 302 Found
Date: Tue, 03 Dec 2019 16:48:24 GMT
Location: http://localhost:8080/cjoc/
Content-Length: 0
Server: Jetty(9.4.z-SNAPSHOT)
  1. Retrieve the pod IP address,

kubectl exec -it cjoc-0 -- cat /etc/hosts | grep cjoc

Example output,

10.244.1.112 cjoc-0.cjoc.cloudbees-core.svc.cluster.local cjoc-0

  1. And, perform the same HTTP request against the IP,

kubectl exec -it cjoc-0 -- curl -D- http://10.244.1.112:8080/cjoc

Which results in the same output,

HTTP/1.1 302 Found
Date: Tue, 03 Dec 2019 16:52:45 GMT
Location: http://10.244.1.112:8080/cjoc/
Content-Length: 0
Server: Jetty(9.4.z-SNAPSHOT)
  1. Find the cjoc service IP, i.e. ClusterIP of cjoc, with the command,

kubectl get service

Example output,

NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)            AGE
cjoc         ClusterIP   10.100.224.84    <none>        80/TCP,50000/TCP   4d2h
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP            62d
  1. Use curl again, this time against the cjoc service (Notice: cjoc service is listening on port 80)

kubectl exec -it cjoc-0 -- curl -D- http://10.100.224.84/cjoc

Example output,

HTTP/1.1 302 Found
Date: Tue, 03 Dec 2019 16:57:42 GMT
Location: http://10.100.224.84/cjoc/
Content-Length: 0
Server: Jetty(9.4.z-SNAPSHOT)
  1. After validating the pod and the service, you can now test ingress

kubectl get ingress

Example output,

NAME    HOSTS                        ADDRESS       PORTS   AGE
cjoc    cloudbees-core.example.com   192.168.1.3   80      4d3h

Note: In this example, ingress for cjoc has address 192.168.1.3, this is the eth0 physical interface of this node. On a cloud provider the address will be different.

  1. From your workstation, curl the hostname and not the IP, e.g.,

curl -D- http://cloudbees-core.example.com/cjoc

Successful output,

HTTP/1.1 302 Found
Server: nginx/1.15.9
Date: Tue, 03 Dec 2019 17:04:09 GMT
Content-Length: 0
Connection: keep-alive
Location: http://cloudbees-core.example.com/cjoc/

with IP,

curl -D- http://192.168.1.3/cjoc

Unsuccessful output,

HTTP/1.1 404 Not Found
Server: nginx/1.15.9
Date: Tue, 03 Dec 2019 17:07:20 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 21
Connection: keep-alive

Troubleshooting

If you installed CloudBees Core using the CloudBees Helm chart, e.g.,

helm install cloudbees-core cloudbees/cloudbees-core --set OperationsCenter.HostName=cloudbees-core.example.com

you have seen the name cloudbees-core.example.com. In our example, this is the hostname which ingress expects in HTTP requests in order to forward your request. Therefore you must use the hostname in the request. Below are two methods of providing a Host header, e.g.,

curl -D- --resolve cloudbees-core.example.com:80:[IP of ingress address] http://cloudbees-core.example.com/cjoc

OR

curl -D- -H "Host: cloudbees-core.example.com" http://[ingress address]

In both examples above, a host header is included in the request and ingress will pass the request through to the pod. You can run this command both inside or outside of the cluster.

Reference

Have more questions?

0 Comments

Please sign in to leave a comment.