Issue

• I am using the Amazon EC2 plugin to provision ephemeral agents and Jenkins thread count is progressively growing. Threads like the following are being leaked:

  "Thread-10000" daemon prio=5 RUNNABLE
      java.net.SocketInputStream.socketRead0(Native Method)
      java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
      java.net.SocketInputStream.read(SocketInputStream.java:171)
      java.net.SocketInputStream.read(SocketInputStream.java:141)
      com.trilead.ssh2.crypto.cipher.CipherInputStream.fill_buffer(CipherInputStream.java:41)
      com.trilead.ssh2.crypto.cipher.CipherInputStream.internal_read(CipherInputStream.java:52)
      com.trilead.ssh2.crypto.cipher.CipherInputStream.getBlock(CipherInputStream.java:79)
      com.trilead.ssh2.crypto.cipher.CipherInputStream.read(CipherInputStream.java:108)
      com.trilead.ssh2.transport.TransportConnection.receiveMessage(TransportConnection.java:232)
      com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:706)
      com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
      java.lang.Thread.run(Thread.java:748)
  

Environment

• CloudBees CI (CloudBees Core)
• CloudBees CI (CloudBees Core) on modern cloud platforms - Managed controller
• CloudBees CI (CloudBees Core) on modern cloud platforms - Operations Center
• CloudBees CI (CloudBees Core) on traditional platforms - Client controller
• CloudBees CI (CloudBees Core) on traditional platforms - Operations Center
• CloudBees Jenkins Platform - Client controller
• CloudBees Jenkins Platform - Operations Center
• CloudBees Jenkins Distribution
• Jenkins LTS
• Amazon EC2 plugin < 1.66

Explanation

When using connectBySSHProcess - labelled as “Connect by SSH Process” in the EC2 Cloud Templates configuration - the EC2 Cloud actually still uses trilead SSH during the early setup of agent and is leaking threads after each provisioning.

Jenkins eventually needs to be restarted to clean up those threads.

Related Issue(s)

• JENKINS-60292: EC2 Plugin Thread Leak

Workaround

Set connectBySSHProcess to false in the EC2 Cloud Templates configuration. In the UI, this is equivalent to disabling the “Connect by SSH Process” checkbox.

Solution

Upgrade the Amazon EC2 Plugin to version 1.66 or later.