KBEA-00088 - Builds are suddenly slower than expected

Summary

Build performance has suddenly and unexpectedly degraded.

Solution

Sudden changes in build performance can be caused by a variety of factors. Here are some things to check:

• Is your build using a good history file?
• Are you running with the same number of agents in the "fast" and "slow" cases?
• Are the agents used in the "fast" and "slow" builds comparable in terms of hardware configuration?
• Are the eMake hosts used in the "fast" and "slow" builds comparable in terms of hardware configuration?
  • Is the license manager accessible from the agents?
  • Is the license manager configuration on the agents correct? For example, on Windows the vxWorks compilers may store configuration information in HKEY_LOCAL_MACHINE\SOFTWARE\FLEXlm License Manager\WRSD_LICENSE_FILE. If this value exists on your agent hosts, verify that it has the correct value for your site.
  • Is the license manager configuration in your environment correct? For example, the vxWorks compilers may use the LM_LICENSE_FILE environment variable to determine how to contact the license manager.
• Are the "fast" and "slow" builds actually comparable builds? Are they building the same sources? Do they run the same number of jobs?
• Are you using third-party tools that are explicitly licensed, such as vxWorks compilers?
• Is disk performance slow on the eMake machine?
  • Use ecdiskperf to determine the speed of the disk subsystem. You should aim for speeds around 100 MB/s for reads and 80 MB/s for writes.
• Is network performance slow?
  • Are the agents on the same switch?
  • It should be at least a 100 Mbit network (1 Gbit is recommended)
• Is the NFS/Samba server slow?
  • Try running from a local disk.
• High levels of logging can affect performance. Try running eMake without high levels of debugging and annotation output.
• Does the annotation show large times of serial work (use ElectricInsight for this analysis).
• Is the effective agent allocation 100? 100 means eMake always got all the agents it wanted. If not, was the percentage different when builds were faster?

Related knowledge base articles:

• KBEA-00028 - How the Windows Recycle Bin affects build speed
• KBEA-00031 - Configuring the Windows stalled job killer
• KBEA-00067 - Diagnosing a slow ClearCase server
• KBEA-00087 - VS2010 or msbuild v4.0 builds are excessively slow

Applies to

• Product versions: All
• OS versions: All