KBEC-00458 - Required data: Performance issue with CD

Issue

Your CloudBees CD (Flow) server’s web UI is responding slowly, or you have other reasons to believe your CD server is encountering a performance issue, and you’d like to open a support ticket to get help.

Environment

Resolution

The required data is:

  1. Symptoms - Screen shots of what you see in the UI, or a description of what you see in the UI that makes you suspect a performance issue
  2. Time range - Approximate time range when this issue was observed, including the time zone
  3. Logs - the commander.log from that timeframe
  4. If you have Garbage Collection logs enabled, then upload the gc logs from the path -Xloggc:$path/gc.log
  5. The output of running collectPerformanceData.sh with the PID of your CD server process:

a. Check you have the required tools to run the script included in the $PATH

The collectPerformanceData.sh script collects thread dumps using the jstack command. It also needs to be able to run top, vmstat, netstat, nfsiostat, nfsstat and iostat. Please make sure that the user running this script can execute all of these commands.

b. Download collectPerformanceData.sh

c. Make the script executable by running: chmod +x collectPerformanceData.sh.

d. Determine the user and PID running CloudBees CD: by running ps -ef | grep commander. You will see output similar to:

rkivisto  1681  1499 99 15:47 ?        00:00:51 /opt/electriccloud/electriccommander/jre/bin/java -server -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -Dec.logRoot="/opt/electriccloud/electriccommander/logs/commander.log" -Dcommander.logSetup=true -Dcommander.logEvents=true -Dcommander.appName=ElectricFlowServer -Djava.net.preferIPv4Stack=true -Dcommander.localLogConfig=conf/logback-local.groovy -Dcommander.tmpLogConfigDir=conf/tmp -Dcommander.removeOldTmpLogConfigs=true -Djavax.net.ssl.sessionCacheSize=1000 -Djgroups.logging.log_factory_class=com.electriccloud.cluster.JGroupsLogFactory -DCOMMANDER_PORT=8000 -DCOMMANDER_HTTPS_PORT=8443 -DCOMMANDER_KEYSTORE_PASSWORD=abcdef -DCOMMANDER_KEYSTORE=conf/keystore -DCOMMANDER_CRL_FILE=conf/security/crl.pem -DWEBAPPS_DIR=/opt/electriccloud/electriccommander/server/wars -Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.StdErrLog -Dorg.eclipse.jetty.LEVEL=INFO -Djdk.tls.client.protocols="TLSv1.2" -Djdk.tls.ephemeralDHKeySize=2048 -Djdk.tls.rejectClientInitiatedRenegotiation=true -Dcommander.tls.enabledProtocols="TLSv1.2" -Dcom.sun.jndi.ldap.object.disableEndpointIdentification=true -Dorg.tanukisoftware.wrapper.WrapperManager.mbean=false -DREST_DOC_TRANSFER_PROTOCOL=https -DREST_DOC_PORT=8443 -DCHANGE_TRACKING_HARD_MAX_RECORDS=1000 -Dliquibase.ignoreRecycleBinWarning=true -Xms2621m -Xmx5242m -Djava.library.path=/opt/electriccloud/electriccommander/server/lib -Dwrapper.key=0huB0v6SnVVnShbaawu4bbNfFnnlQ3gt -Dwrapper.port=32001 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.disable_console_input=TRUE -Dwrapper.pid=1499 -Dwrapper.version=3.5.27-st -Dwrapper.native_library=wrapper -Dwrapper.arch=x86 -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 -Dwrapper.lang.domain=wrapper -Dwrapper.lang.folder=../lang com.electriccloud.util.AppStart -jaxpmodule javax.xml.jaxp-provider

The first two columns show the user and process ID.

e. Check that the user running the process can write to the same folder where collectPerformanceData.sh will be run. The script will generate data in that directory.

e. Execute the script while the issue is occurring. Run the collectPerformanceData.sh script as the same user you found in step d by running:

sudo -u $USER sh collectPerformanceData.sh $PID 300 5

Tested product/plugin versions

CloudBees CD V8.0.5, V10.0

Have more questions?

0 Comments

Please sign in to leave a comment.