You are running into issues where your Commander server is suddenly slowing down or some API requests taking way too long to complete. You need some help narrowing down what could be stalling this operation.
Thread dumps: These are sometimes printed into Commander server logs, but during an issue, its best to get a few of these thread dumps every 30 seconds to 1 minute.
These results will get printed into the commander-service.log and will help narrow down what threads are unavailable or stalled.
How to get Thread dumps
To get the thread dump as explained in http://wrapper.tanukisoftware.com/doc/english/integrate-start-stop-nix.html do steps below:
1. The ProcessID of the Commander wrapper process is stored in /opt/electriccloud/electriccommander/server/commander.pid file In this file if you see the content is "1052" that means the Commander wrapper process ID is 1052.
2. Run the command from the shell prompt:kill -3
3. Wait for 30 seconds.
Note: 30 seconds is just an example. Depending on how long the slowness is or if the slowness is due to a burst of requests received by Flow server, this may have to be altered to 5 or 10 second intervals. Please follow the interval given by agent.
4. Repeat step (2)
5. Repeat step (3) + (4) , 10 times
All the thread dumps will be logged to the /opt/electriccloud/electriccommander/logs/commander-service.log
Please send us:
- Zip and send the commander-service.log file ( and if it rolled over then the recent server*.log.zip file which got rolled over).
- Commander log files.
- All Product versions
- Linux OS versions