KBEC-00374 - Error "i/O reactor has been shut down"

Problem

You meet the error “i/O reactor has been shut down” when using CloudBees CD (CloudBees Flow). This error can occur in different places. For example, it may occur when running an step and the following error message displays in the UI:

Step error [InternalError]: i/O reactor has been shut down

An unexpected error occurred while invoking the command

Solution

The cause of this error is probably that the process (which reports this error. It can be the commanderServer or commanderAgent process) reaches the maximum limit of open files. And the reason why it reaches the maximum limit of open files could be:

1. The open file limit is too low. In many cases, the limit is 1024 and it is not enough for commanderServer. It is recommended to change it to 8192 (or higher based on the situation).
2. The process load is high and it does a lot of tasks. For example, the commanderServer is running many jobs or the commanderAgent is running too many job steps.
3. Any other possible causes. A recent case is that two NFS were mounted in an agent machine and resulted in this error.