Metrics plugin leaks 'QueueSubTaskMetrics' threads when monitoring the Queue

Issue

  • The Jenkins master thread dump show lot of threads like the following:
"QueueSubTaskMetrics [#XXX]" #XXXXX daemon prio=5 os_prio=0 tid=0x00001a1234567890 nid=0xa46e waiting on condition [0x00001a0987654321]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000q2w3e4r5> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
	at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Environment

Related Issues

Explanation

The Metrics plugin monitors the Queue asynchronously. It is leaking QueueSubTaskMetrics threads when a SubTask from the build queue fails badly - in such a way that the task is never updated. In such cases the QueueSubTaskMetrics thread wait forever for a event that will never occur.

The master needs to be restarted to clean those threads.

The root cause of this issue is actually a bug in Jenkins core that is not updating the SubTask consistently in case of premature failure captured by JENKINS-59793.

Resolution

Upgrade CloudBees Core to version 2.204.1.3 or later.

Have more questions?

0 Comments

Please sign in to leave a comment.