- Jenkins hangs (may be anytime, including startup)
- Jenkins is not responsive or coming up, the process is running and Jenkins logs are stale
This article describes how to collect the minimum required information for deadlock issues on a Jenkins environment so that it can be efficiently troubleshooted.
If the required data is bigger than 20 MB you will not be able to use ZenDesk to upload all the information. On this case we would like to encourage you to use our upload service in order to attach all the required information.
- Operating System: Linux
- CloudBees Core on modern cloud platforms - Managed Master
- CloudBees Core on modern cloud platforms - Operations Center
- CloudBees Core on traditional platforms - Client Master
- CloudBees Core on traditional platforms - Operations Center
- CloudBees Jenkins Enterprise - Managed Master
- CloudBees Jenkins Enterprise - Operations Center
- CloudBees Jenkins Platform - Client Master
- CloudBees Jenkins Platform - Operations Center
- CloudBees Jenkins Distribution
- Jenkins LTS
Ideally, your environment is configured as documented in [Prepare Jenkins For Support].
- Output of the script jenkinshangWithJstack.sh
- Support bundle
- List of File Descriptors
- Optional (if an I/O issue is suspected): output of the script listDProcessesNativeStacks.sh
Thread dumps are the best way to pinpoint the root cause of this issue.
A support bundle from the problematic instance ideally when the issue is happening or in the worst case right after the issue is exposed. Please, follow the KB below in case you don’t know how to generate a support bundle.
Please, enable at minimum the following checks:
Master Log Recorders,
Garbage Collection Logs,
Slow Request Records,
Out Of Memory Errors,
Master Heap Histogram,
Thread dumps of running Pipeline builds and
- What is a support-bundle and how to create one
- How to get a support bundle when Jenkins will not start? (if Jenkins is stuck while starting)
The list of file descriptor hold by the Jenkins process.
Find the PID of the Jenkins process. For example the following output of
ps aux | grep jenkins show that Jenkins PID is
[ec2-user@ip-10-240-0-112 ~]$ ps -ef | grep jenkins ec2-user 2487 2435 1 May07 ? 00:26:30 java -Duser.home=/var/jenkins_home -Xmx2150m -Xms2150m -jar /usr/share/jenkins/jenkins.war --webroot=/tmp/jenkins/war --pluginroot=/tmp/jenkins/plugins ec2-user 14540 13480 0 03:20 pts/0 00:00:00 grep --color=auto jenkins
Collect the list of file descriptor of that process with
sudo lsof if sudo privileges are needed):
lsof -a -p $JENKINS_PID > /tmp/file-descriptors.txt
lsof is not available on the host, you may use the following command instead:
ls -l /proc/$JENKINS_PID/fd/ > /tmp/file-descriptors.txt
Collect the file produced