KBEA-00144 - Managing emake temporary files

Summary

During the build process, temporary files are created, then deleted automatically when a build completes. To maintain optimum efficiency, it is important to identify where these files are created in relation to the final output files.

Solution

Configuring the Electric Make Temporary Directory

Electric Make runs commands and collects output in parallel, but the results are written in order serially. This feature ensures a build behaves exactly as if it were run locally.

The Electric Make temporary directory is used to store output files from commands (for example, object files) that finished running but are waiting their turn to be relocated to a final destination on disk. By default, Electric Make creates a temporary directory in the current working directory. The directory name will have the form:

• where pid is the eMake process identifier.

• where n is a counter assigned by eMake to differentiate multiple temporary directories created during a single build if you specify multiple directories.

Electric Make removes the temporary directory on exit (including Ctrl-C user interrupt). If Electric Make is terminated unexpectedly (for example, by the operating system), the temporary directory may persist and need to be removed manually.

Important temporary directory requirements include:

• Must be writable by the eMake process
• Must not be on an NFS share
• Must have enough space to contain the build output

Ideally, the temporary directory should have enough space to hold the complete build output; in practice however, it may need only enough space for output from the largest commands. The exact temporary directory space requirement varies greatly with the build size, the number of Agents used, and build system speed.

You can use the --emake-tmpdir command-line option or the EMAKE_TMPDIR environment variable to change the temporary directory default location:

Files are relocated from the temporary directory to their final location. Keeping the temporary directory on the same file system as the EMAKE_ROOT helps performance because files can simply be renamed in place, instead of copied. Electric Make and agent machines reset the values for TEMP, TMP, and TMPDIR. This is necessary to avoid possible conflicts with multiple build agents/jobs running on the same machine. If you have more than one EMAKE_ROOT that spans multiple file systems, you can specify more than one temporary directory to ensure Electric Make can always rename files in place, instead of copying them.

For example, if your EMAKE_ROOT contains three directories:

• that reside on two different file systems
• to ensure better performance, specify two temporary directories.
• write permission at both locations is required

When specifying multiple temporary directories, note the following:

• If a temporary directory was specified for a particular file system, Electric Make automatically uses that directory for any files destined to reside on that file system.
• If a temporary directory was not specified for a particular file system, Electric Make uses the first directory specified for files destined to reside on that file system.

Deleting Temporary Files

During a build, Electric Make creates a temporary directory inside the directory specified by the EMAKE_TMPDIR environment variable or in the directory specified by the --emake-tmpdir command-line option (or the current working directory if no Electric Make temporary directory is specified). If a specified directory does not exist, eMake creates it. All temporary directories created by eMake are automatically deleted when a build completes.

For example, if you set your temporary directory to /foo/bar/baz and only /foo/bar exists, eMake creates /foo/bar/baz and /foo/bar/baz/ecloud_tmp_, and then deletes both directories and all their contents when it exits. If /foo/bar/baz exists at the start of the build, only /foo/bar/baz/ecloud_tmp _ is created and deleted.

However, if the build is aborted in-progress, eMake will not have the opportunity to remove the temporary directory.

Subsequent Electric Make invocations automatically delete temporary directories if they are more than 24 hours old. You can reclaim disk space more quickly by deleting temporary directories and their contents by hand.

However, do not delete the temporary directory while a build is in-progress, this action causes the build to fail.

Temporary directory names are in this form:

• where pid is the eMake process identifier.

• where n is a counter assigned by eMake to differentiate multiple temporary directories created during a single build if you specify multiple directories.

Applies to

• Product versions: All
• OS versions: All