run — find, execute, and log the results of tests

run  command  has  three  basic  modes of operation.  With neither -c nor -w, run processes the arguments
       provided on the command line, adding them to the list for this  run.   If  a  specified  pathname  is  an
       executable  file,  it  is  added as a test.  If a specified pathname is a directory, the behavior depends
       upon the presence of -g.  If -g is specified, the directory is treated as a test group.  See the  section
       on  TestGroups below.  Without -g, run simply descends into the directory looking for executable files.
       The tests are then executed, and the results are logged.

       With -w, run finds tests in the manner described above.  Rather than executing the tests and logging  the
       results,  the  test  configuration  is  stored  in a runfile, which can be used in future invocations, or
       edited to modify which tests are executed and which options are applied.  Options included on the command
       line with -w become defaults in the runfile.

       With -c, run parses a runfile, which can specify a series of tests and test groups to be  executed.   The
       tests are then executed, and the results are logged.

   TestGroups
       A test group is comprised of a set of executable files, all of which exist in one directory.  The options
       specified  on  the command line or in a runfile apply to individual tests in the group.  The exception is
       options pertaining to pre and post scripts, which act on all tests  as  a  group.   Rather  than  running
       before and after each test, these scripts are run only once each at the start and end of the test group.

   TestExecution
       The  specified  tests  run  serially, and are typically assigned results according to exit values.  Tests
       that exit zero and non-zero are marked PASS and FAIL, respectively.  When a pre script fails for  a  test
       group,  only  the  post  script  is  executed, and the remaining tests are marked SKIPPED.  Any test that
       exceeds its timeout is terminated, and marked KILLED.

       By default, tests are executed with the credentials of  the  run  script.   Executing  tests  with  other
       credentials  is  done  via  sudo(1m), which must be configured to allow execution without prompting for a
       password.  Environment variables from the calling shell are available to individual tests.   During  test
       execution, the working directory is changed to outputdir.

   OutputLogging
       By default, run will print one line on standard output at the conclusion of each test indicating the test
       name,  result  and  elapsed time.  Additionally, for each invocation of run, a directory is created using
       the ISO 8601 date format.  Within this directory is a file named log containing all the test output  with
       timestamps,  and  a  directory  for  each  test.  Within the test directories, there is one file each for
       standard output,  standard  error  and  merged  output.   The  default  location  for  the  outputdir  is
       /var/tmp/test_results.

   Runfiles
       The runfile is an INI-style configuration file that describes a test run.  The file has one section named
       DEFAULT,  which  contains configuration option names and their values in name = value format.  The values
       in this section apply to all the subsequent sections, unless they are also specified there, in which case
       the default is overridden.  The  remaining  section  names  are  the  absolute  pathnames  of  files  and
       directories, describing tests and test groups respectively.  The legal option names are:

       outputdir = pathname     The name of the directory that holds test logs.

       pre = script             Run script prior to the test or test group.

       pre_user = username      Execute the pre script as username.

       post = script            Run script after the test or test group.

       post_user = username     Execute the post script as username.

       quiet = True|False       If True, only the results summary is printed to standard out.

       tests = ['filename', …]  Specify  a  list  of  filenames  for  this test group.  Only the basename of the
                                absolute path is required.  This option is only valid for test groups, and  each
                                filename must be single quoted.

       timeout = n              A timeout value of n seconds.

       user = username          Execute the test or test group as username.

Example1: Running ad-hoc tests.
           This example demonstrates the simplest invocation of run.

           % runmy-tests
           Test: /home/jkennedy/my-tests/test-01                    [00:02] [PASS]
           Test: /home/jkennedy/my-tests/test-02                    [00:04] [PASS]
           Test: /home/jkennedy/my-tests/test-03                    [00:01] [PASS]

           Results Summary
           PASS       3

           Running Time:   00:00:07
           Percent passed: 100.0%
           Log directory:  /var/tmp/test_results/20120923T180654

       Example2: Creating a runfile for future use.
           This example demonstrates creating a runfile with non-default options.

           % run-psetup-xroot-g-wnew-tests.runnew-tests
           % catnew-tests.run
           [DEFAULT]
           pre = setup
           post_user =
           quiet = False
           user =
           timeout = 60
           post =
           pre_user = root
           outputdir = /var/tmp/test_results

           [/home/jkennedy/new-tests]
           tests = ['test-01', 'test-02', 'test-03']

       run — find, execute, and log the results of tests

-crunfile    Specify a runfile to be consumed by the run command.

       -d            Dry  run mode.  Execute no tests, but print a description of each test that would have been
                     run.

       -m            Enable kmemleak reporting (Linux only)

       -g            Create test groups from any directories found while searching for tests.

       -ooutputdir  Specify the directory in which to write test results.

       -pscript     Run script prior to any test or test group.

       -Pscript     Run script after any test or test group.

       -q            Print only the results summary to the standard output.

       -sscript     Run script as a failsafe after any test is killed.

       -Susername   Execute the failsafe script as username.

       -tn          Specify a timeout value of n seconds per test.

       -uusername   Execute tests or test groups as username.

       -wrunfile    Specify the name of the runfile to create.

       -xusername   Execute the pre script as username.

       -Xusername   Execute the post script as username.

sudo(1m)

OpenZFS                                           May 26, 2021                                            RUN(1)

run [-dgq] [-ooutputdir] [-pPscript] [-t-seconds] [-uxXusername] pathname…

       run-wrunfile [-gq] [-ooutputdir] [-pPscript] [-t-seconds] [-uxXusername] pathname…

       run-crunfile [-dq]

       run [-h]