Issue
- How do I generate a heap dump.
- Diagnosing memory issues: OutOfMemoryError (OOM). Analyzing possible Memory Leaks.
Resolution
A heap dump is a snapshot of the memory of a Java process at a certain point in time. There are different formats for persisting this data, and depending on the format it may contain different pieces of information, but in general the snapshot contains information about the Java objects and classes in the heap at the moment the snapshot was triggered. Usually a full GC is triggered before the heap dump is written so it contains information about the remaining objects.
1. Get the PID of the Java process where Jenkins is running
To generate a head dump the first part is to obtain the process id (or PID). There are multiple methods to obtain the PID:
- jps - From Unix Terminal or Windows Cmd with the JDK installed into the OS. Lists the instrumented Java Virtual Machines (JVMs) on the target system.
- ps -ef | grep java - From Unix Terminal only. It is used to obtain the process id of all java processes.
- Process explorer - There are task manager UI applications (System Tools) which can be used to obtain the process id. Find the process and view its PID in the corresponding column.
- On Windows and Linux (Ubuntu) named
Task manager - On Mac OS named
Activity Monitor
- On Windows and Linux (Ubuntu) named
2. Get the Heap Dump
NOTE: You must run the following commands as the user that Jenkins is running as. Usually this user is ‘jenkins.’ No other user, including root, will be able to run the command successfully.
After obtaining the process id ($PID), the next step is to generate the heap dump using jcmd:
- jcmd $PID GC.heap_dump filename=$FILENAME - From Unix Terminal or Windows Powershell with the JDK running Jenkins. It prints shared object memory maps or heap memory details for a process, core file, or remote debug server. Please, change the $FILENAME for any of the heap dumps you would like to send to us via our Upload Site.
jcmd is the preferred method for generating a heap dump per Oracle Documentation
If you are experiencing a problem using jcmd please use the alternative jmap method: jmap -dump:format=b,file=$FILENAME.bin $PID - From Unix Terminal or Windows Powershell
NOTE: The JVM will need to perform a full garbage collection cycle before it can generate a heap dump, in order to free as many unused objects as possible first. Jenkins will be paused/unavailable during this time. For very large heap sizes, the full GC could be slow, so it’s important to realize that getting a heap dump may impact users of the application.
Generating heap dumps periodically during an OOM issue is beneficial, particularly after a service restart to baseline the instance, and subsequently later to show the memory growth over a period of time. Attach the output generated from these commands to a CloudBees support ticket, so we can help diagnose further.
Heap Dumps can be quite large in size, therefore the split utility comes in handy here, as our upload site supports 10GB uploads, but can be difficult depending on your internet connection. Thus, use the split linux utility to split the generated heapdump file into 2GB segments for ease of upload to your support ticket and our team will cat them together once received.
or Windows Cmd
- split -b 2gb $FILENAME - From Unix Terminal or Windows Powershell
To concatenate files after splitting:
- cat ${file1} {file2} {file3} > {output_filename} -From Unix Terminal or Windows Powershell
- copy /B {file1} + {file2} + {file3} {output_filename} -From Windows CMD
Notes
- Some of these commands might require admin permissions.
- A JDK is required, not JRE. To run the proposed commands you need to run them from
$JDK_PATH/binor from anywhere by including$JDK_PATH/bininto your System$PATH. - For Windows >
Task Managerif PID is not displayed > click View - Select Columns > Select the PID (Process Identifier). - Unix Terminal includes Linux and MacOS OS.
- As an option, for running Unix commands into Windows Cygwin can be installed.
- Please check the KB CloudBees Jenkins Platform supported Java versions.
0 Comments