KBEC-00311 - Get Thread dumps in Linux

Summary

You are running into issues where your Commander server is suddenly slowing down or some API requests taking way too long to complete. You need some help narrowing down what could be stalling this operation.

Solution

Thread dumps: These are sometimes printed into Commander server logs, but during an issue, its best to get a few of these thread dumps every 15 seconds to 1 minute.

These results will get printed into the commander-service.log and will help narrow down what threads are unavailable or stalled.

How to get Thread dumps

To get the thread dump as explained in http://wrapper.tanukisoftware.com/doc/english/integrate-start-stop-nix.html do steps below:

1- The ProcessID of the Commander wrapper process is stored in /opt/electriccloud/electriccommander/server/commander.pid file In this file if you see the content is “1052” that means the Commander wrapper process ID is 1052.
2- To get the thread dumps every 15 seconds you can run:

$ watch -n 15 "kill -3 < ProcessID of the Commander wrapper process>"

For example:

$ watch -n 15 "kill -3 1052"
Or
$ watch -n 15 "kill -3 `ps -ef | grep -ie "COMMANDER_HTTPS_PORT=8443" | grep -v grep | awk '{print  $3}'`"

3- Once you get 5 minutes of thread dumps you can stop the watch command with a ctrl+c

4- Repeat these steps in all the Flow server Nodes in an HA environment.

***Note:***15 seconds is just an example. Depending on how long the slowness is or if the slowness is due to a burst of requests received by Flow server, this may have to be altered to 5 or 10 second intervals. Please follow the interval given by agent.

All the thread dumps will be logged to the /opt/electriccloud/electriccommander/logs/commander-service.log

Please send us:

  • Zip and send the commander-service.log file ( and if it rolled over then the recent server*.log.zip file which got rolled over).

  • Commander log files.

See Also

KBEC-00312 - Get Thread dumps in Windows

Applies to

  • All Product versions
  • Linux OS versions

Have more questions?

0 Comments

Please sign in to leave a comment.