scons-time - generate and display SCons timing information

Authors

       Steven Knight <knight at baldmt dot com>

SCons __VERSION__                                  10/08/2023                                      SCONS-TIME(1)

Configuration File

Various scons-time subcommands can read information from a specified configuration file when passed the
-f or --file options. The configuration file is actually executed as a Python script. Setting Python
variables in the configuration file controls the behavior of the scons-time script more conveniently than
having to specify command-line options or arguments for every run, and provides a handy way to
"shrink-wrap" the necessary information for producing (and reporting) consistent timing runs for a given
configuration.

archive_list
A list of archives (files or directories) that will be copied to the temporary directory in which
SCons will be invoked. .tar, .tar.gz, .tgz and .zip files will have their contents unpacked in the
temporary directory. Directory trees and files will be copied as-is.

initial_commands
A list of commands that will be executed before the actual timed scons runs. This can be used for
commands that are necessary to prepare the source tree-for example, creating a configuration file
that should not be part of the timed run.

key_location
The location of the key on Gnuplot graphing information generated with the --format=gnuplot option.
The default is bottomleft.

prefix
The file name prefix to be used when running or extracting timing for this configuration.

python
The path name of the Python executable to be used when running or extracting information for this
configuration. The default is the same version of Python used to run the SCons

scons
The path name of the SCons script to be used when running or extracting information for this
configuration. The default is simply scons.

scons_flags
The scons flags used when running SCons to collect timing information. The default value is
--debug=count--debug=memory--debug=time--debug=memoizer.

scons_lib_dir, scons_wrapper, startup_targets, subdir
The subdirectory of the project into which the scons-time script should change before executing the
SCons commands to time.

subversion_url
The Subversion URL from

svn
The subversion executable used to check out revisions of SCons to be timed. The default is simple
svn.

svn_co_flag, tar, targets
A string containing the targets that should be added to the command line of every timed scons run.
This can be used to restrict what's being timed to a subset of the full build for the configuration.

targets0, targets1, targets2, title, unzip, verbose, vertical_barsExample
Here is an example scons-time configuration file for a hypothetical sample project:

# The project doesn't use SCons natively (yet), so we're
# timing a separate set of SConscript files that we lay
# on top of the vanilla unpacked project tarball.
arguments = ['project-1.2.tgz', 'project-SConscripts.tar']

# The subdirectory name contains the project version number,
# so tell scons-time to chdir there before building.
subdir = 'project-1.2'

# Set the prefix so output log files and profiles are named:
# project-000-[012].{log,prof}
# project-001-[012].{log,prof}
# etc.
prefix = 'project'

# The SConscript files being tested don't do any SConf
# configuration, so run their normal ./configure script
# before we invoke SCons.
initial_commands = [
'./configure',
]

# Only time building the bin/project executable.
targets = 'bin/project'

# Time against SCons revisions of the branches/core branch
subversion_url = 'http://scons.tigris.org/svn/scons/branches/core'

Description

       The scons-time command runs an SCons configuration through a standard set of profiled timings and can
       extract and graph information from the resulting profiles and log files of those timings. The action to
       be performed by the scons-time script is specified by a subcommand, the first argument on the command
       line. See the SUBCOMMANDS section below for information about the operation of specific subcommands.

       The basic way to use scons-time is to run the scons-timerun subcommand (possibly multiple times) to
       generate profile and log file output, and then use one of the other subcommands to display the results
       captured in the profiles and log files for a particular kind of information: function timings (the
       scons-timefunc subcommand), total memory used (the scons-timemem subcommand), object counts (the
       scons-timeobj subcommand) and overall execution time (the scons-timetime subcommand). Options exist to
       place and find the profiles and log files in separate directories, to generate the output in a format
       suitable for graphing with the gnuplot(1) program, and so on.

       There are two basic ways the scons-timerun subcommand is intended to be used to gather timing statistics
       for a configuration. One is to use the --svn= option to test a configuration against a list of revisions
       from the SCons Subversion repository. This will generate a profile and timing log file for every revision
       listed with the --number= option, and can be used to look at the impact of committed changes to the SCons
       code base on a particular configuration over time.

       The other way is to profile incremental changes to a local SCons code base during a development
       cycle--that is, to look at the performance impact of changes you're making in the local tree. In this
       mode, you run the scons-timerun subcommand without the --svn= option, in which case it simply looks in
       the profile/log file output directory (the current directory by default) and automatically figures out
       the next run number for the output profile and log file. Used in this way, the development cycle goes
       something like: make a change to SCons; run scons-timerun to profile it against a specific
       configuration; make another change to SCons; run scons-timerun again to profile it; etc.

Environment

       The scons-time script uses the following environment variables:

       PRESERVE
           If this value is set, the scons-time script will not remove the temporary directory or directories in
           which it builds the specified configuration or downloads a specific version of SCons.

Generating Timing Information

scons-timerun [-hnqv] [-fFILE] [--number=NUMBER] [--outdir=OUTDIR] [-pSTRING] [--python=PYTHON] [-sDIR] [--scons=SCONS] [--svn=URL] [ARGUMENTS]

   ExtractingFunctionTimingsscons-timefunc [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [--func=NAME] [-pSTRING] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

   ExtractingMemoryStatisticsscons-timemem [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [--stage=STAGE] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

   ExtractingObjectCountsscons-timeobj [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [--stage=STAGE] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

   ExtractingExecutionTimesscons-timetime [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [-tNUMBER] [--title=TITLE]
       [--which=WHICH] [ARGUMENTS]

   HelpTextscons-timehelpSUBCOMMAND [...]

Name

       scons-time - generate and display SCons timing information

Options

       The scons-time command only supports a few global options:

       -h, --help
           Displays the global help text and exits, identical to the scons-timehelp subcommand.

       -V, --version
           Displays the scons-time version and exits.

       Most functionality is controlled by options to the individual subcommands. See the next section for
       information about individual subcommand options.

Subcommands

       The scons-time command supports the following individual subcommands.

   ThefuncSubcommandscons-timefunc [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [--func=NAME] [-pSTRING] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

       The scons-timefunc subcommand displays timing information for a specific Python function within SCons.
       By default, it extracts information about the _main() function, which includes the Python profiler timing
       for all of SCons.

       The scons-timefunc subcommand extracts function timing information from all the specified file
       arguments, which should be Python profiler output files. (Normally, these would be *.prof files generated
       by the scons-timerun subcommand, but they can actually be generated by any Python profiler invocation.)
       All file name arguments will be globbed for on-disk files.

       If no arguments are specified, then function timing information will be extracted from all *.prof files,
       or the subset of them with a prefix specified by the -p option.

       Options include:

       -C DIRECTORY, --chdir=DIRECTORY
           Changes to the specified DIRECTORY before looking for the specified files (or files that match the
           specified patterns).

       -f FILE, --file=FILE
           Reads configuration information from the specified FILE.

       -fmt=FORMAT, --format=FORMAT
           Reports the output in the specified FORMAT. The formats currently supported are ascii (the default)
           and gnuplot.

       --func=NAME
           Extracts timings for the specified function NAME. The default is to report cumulative timings for the
           _main() function, which contains the entire SCons run.

       -h, --help
           Displays help text for the scons-timefunc subcommand.

       -p STRING, --prefix=STRING
           Specifies the prefix string for profiles from which to extract function timing information. This will
           be used to search for profiles if no arguments are specified on the command line.

       -t NUMBER, --tail=NUMBER
           Only extracts function timings from the last NUMBER files.

   ThehelpSubcommandscons-timehelpSUBCOMMAND [...] The help subcommand prints help text for any other subcommands listed as
       later arguments on the command line.

   ThememSubcommandscons-timemem [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [--stage=STAGE] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

       The scons-timemem subcommand displays how much memory SCons uses.

       The scons-timemem subcommand extracts memory use information from all the specified file arguments,
       which should be files containing output from running SCons with the --debug=memory option. (Normally,
       these would be *.log files generated by the scons-timerun subcommand.) All file name arguments will be
       globbed for on-disk files.

       If no arguments are specified, then memory information will be extracted from all *.log files, or the
       subset of them with a prefix specified by the -p option.

       -C DIR, --chdir=DIR
           Changes to the specified DIRECTORY before looking for the specified files (or files that match the
           specified patterns).

       -f FILE, --file=FILE
           Reads configuration information from the specified FILE.

       -fmt=FORMAT, --format=FORMAT
           Reports the output in the specified FORMAT. The formats currently supported are ascii (the default)
           and gnuplot.

       -h, --help
           Displays help text for the scons-timemem subcommand.

       -p STRING, --prefix=STRING
           Specifies the prefix string for log files from which to extract memory usage information. This will
           be used to search for log files if no arguments are specified on the command line.

       --stage=STAGE
           Prints the memory used at the end of the specified STAGE: pre-read (before the SConscript files are
           read), post-read, (after the SConscript files are read), pre-build (before any targets are built) or
           post-build (after any targets are built). If no --stage option is specified, the default behavior is
           post-build, which reports the final amount of memory used by SCons during each run.

       -t NUMBER, --tail=NUMBER
           Only reports memory statistics from the last NUMBER files.

   TheobjSubcommandscons-timeobj [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [--stage=STAGE] [-tNUMBER]
       [--title=TITLE] [ARGUMENTS]

       The scons-timeobj subcommand displays how many objects of a specific named type are created by SCons.

       The scons-timeobj subcommand extracts object counts from all the specified file arguments, which should
       be files containing output from running SCons with the --debug=count option. (Normally, these would be
       *.log files generated by the scons-timerun subcommand.) All file name arguments will be globbed for
       on-disk files.

       If no arguments are specified, then object counts will be extracted from all *.log files, or the subset
       of them with a prefix specified by the -p option.

       -C DIR, --chdir=DIR
           Changes to the specified DIRECTORY before looking for the specified files (or files that match the
           specified patterns).

       -f FILE, --file=FILE
           Reads configuration information from the specified FILE.

       -fmt=FORMAT, --format=FORMAT
           Reports the output in the specified FORMAT. The formats currently supported are ascii (the default)
           and gnuplot.

       -h, --help
           Displays help text for the scons-timeobj subcommand.

       -p STRING, --prefix=STRING
           Specifies the prefix string for log files from which to extract object counts. This will be used to
           search for log files if no arguments are specified on the command line.

       --stage=STAGE
           Prints the object count at the end of the specified STAGE: pre-read (before the SConscript files are
           read), post-read, (after the SConscript files are read), pre-build (before any targets are built) or
           post-build (after any targets are built). If no --stage option is specified, the default behavior is
           post-build, which reports the final object count during each run.

       -t NUMBER, --tail=NUMBER
           Only reports object counts from the last NUMBER files.

   TherunSubcommandscons-timerun [-hnqv] [-fFILE] [--number=NUMBER] [--outdir=OUTDIR] [-pSTRING] [--python=PYTHON] [-sDIR] [--scons=SCONS] [--svn=URL] [ARGUMENTS] The scons-timerun subcommand is the basic subcommand for
       profiling a specific configuration against a version of SCons.

       The configuration to be tested is specified as a list of files or directories that will be unpacked or
       copied into a temporary directory in which SCons will be invoked. The scons-timerun subcommand
       understands file suffixes like .tar, .tar.gz, .tgz and .zip and will unpack their contents into a
       temporary directory. If more than one argument is specified, each one will be unpacked or copied into the
       temporary directory "on top of" the previous archives or directories, so the expectation is that multiple
       specified archives share the same directory layout.

       Once the file or directory arguments are unpacked or copied to the temporary directory, the scons-timerun subcommand runs the requested version of SCons against the configuration three times:

       Startup
           SCons is run with the --help option so that just the SConscript files are read, and then the default
           help text is printed. This profiles just the perceived "overhead" of starting up SCons and processing
           the SConscript files.

       Full build
           SCons is run to build everything specified in the configuration. Specific targets to be passed in on
           the command l ine may be specified by the targets keyword in a configuration file; see below for
           details.

       Rebuild
           SCons is run again on the same just-built directory. If the dependencies in the SCons configuration
           are correct, this should be an up-to-date, "do nothing" rebuild.

       Each invocation captures the output log file and a profile.

       The scons-timerun subcommand supports the following options:

       -f FILE, --file=FILE
           Reads configuration information from the specified FILE. This often provides a more convenient way to
           specify and collect parameters associated with a specific timing configuration than specifying them
           on the command line. See the CONFIGURATION FILE section below for information about the configuration
           file parameters.

       -h, --help
           Displays help text for the scons-timerun subcommand.

       -n, --no-exec
           Do not execute commands, just printing the command-line equivalents of what would be executed. Note
           that the scons-time script actually executes its actions in Python, where possible, for portability.
           The commands displayed are UNIX equivalents of what it's doing.

       --number=NUMBER
           Specifies the run number to be used in the names of the log files and profile outputs generated by
           this run.

       When used in conjunction with the --svn=URL option, NUMBER specifies one or more comma-separated
       Subversion revision numbers that will be retrieved automatically from the Subversion repository at the
       specified URL. Ranges of delta or revision numbers may be specified be separating two numbers with a
       hyphen (-).

       Example:

           % scons-time run --svn=http://scons.tigris.org/svn/trunk --num=1247,1249-1252 .

       -p STRING, --prefix=STRING
           Specifies the prefix string to be used for all of the log files and profiles generated by this run.
           The default is derived from the first specified argument: if the first argument is a directory, the
           default prefix is the name of the directory; if the first argument is an archive (tar or zip file),
           the default prefix is the the base name of the archive, that is, what remains after stripping the
           archive suffix (.tgz, .tar.gz or .zip).

       --python=PYTHON
           Specifies a path to the Python executable to be used for the timing runs. The default is to use the
           same Python executable that is running the scons-time command itself.

       -q, --quiet
           Suppresses display of the command lines being executed.

       -s DIR, --subdir=DIR
           Specifies the name of directory or subdirectory from which the commands should be executed. The
           default is XXX

       --scons=SCONS
           Specifies a path to the SCons script to be used for the timing runs. The default is XXX

       --svn=URL, --subversion=URL
           Specifies the URL of the Subversion repository from which the version(s) of scons being timed will be
           extracted. When --svn is specified, the --number=NUMBER option specifies revision numbers that will
           be tested. Output from each invocation run will be placed in file names that match the Subversion
           revision numbers. If the --number= option is not specified, then the default behavior is to time the
           HEAD of the specified URL.

       -v, --verbose
           Displays the output from individual commands to the screen (in addition to capturing the output in
           log files).

   ThetimeSubcommandscons-timetime [-h] [--chdir=DIR] [-fFILE] [--fmt=FORMAT] [-pSTRING] [-tNUMBER] [--title=TITLE]
       [--which=WHICH] [ARGUMENTS]

       The scons-timetime subcommand displays SCons execution times as reported by the scons--debug=time
       option.

       The scons-timetime subcommand extracts SCons timing from all the specified file arguments, which should
       be files containing output from running SCons with the --debug=time option. (Normally, these would be
       *.log files generated by the scons-timerun subcommand.) All file name arguments will be globbed for
       on-disk files.

       If no arguments are specified, then execution timings will be extracted from all *.log files, or the
       subset of them with a prefix specified by the -p option.

       -C DIR, --chdir=DIR
           Changes to the specified DIRECTORY before looking for the specified files (or files that match the
           specified patterns).

       -f FILE, --file=FILE
           Reads configuration information from the specified FILE.

       -fmt=FORMAT, --format=FORMAT
           Reports the output in the specified FORMAT. The formats currently supported are ascii (the default)
           and gnuplot.

       -h, --help
           Displays help text for the scons-timetime subcommand.

       -p STRING, --prefix=STRING
           Specifies the prefix string for log files from which to extract execution timings. This will be used
           to search for log files if no arguments are specified on the command line.

       -t NUMBER, --tail=NUMBER
           Only reports object counts from the last NUMBER files.

       --which=WHICH
           Prints the execution time for the specified WHICH value: total (the total execution time),
           SConscripts (total execution time for the SConscript files themselves), SCons (exectuion time in
           SCons code itself) or commands (execution time of the commands and other actions used to build
           targets). If no --which option is specified, the default behavior is total, which reports the total
           execution time for each run.

Synopsis

scons-timesubcommand [options...] [arguments...]