Charliecloud comes with a comprehensive test suite that exercises the container workflow itself as well
as a few example applications. ch-test coordinates running the test suite.
While the CLI has lots of options, the defaults are reasonable, and bare ch-test will give useful results
in a few minutes on single-node, internet-connected systems with a few GB available in /var/tmp.
The test suite requires a few GB (standard scope) or tens of GB (full scope) of storage for test
fixtures:
• Builderstorage (e.g., layer cache). This goes wherever the builder puts it.
• Packedimagesdirectory: image tarballs or SquashFS files.
• Unpackedimagesdirectory. Images are unpacked into and then run from here.
• Filesystempermissions directories. These are used to test that the kernel is enforcing permissions
correctly. Note that this exercises the kernel, not Charliecloud, and can be omitted from routine
Charliecloud testing.
The first three are created when needed if they don’t exist, while the filesystem permissions fixtures
must be created manually, in order to accommodate configurations where sudo is not available via the same
login path used for running tests.
The packed and unpacked image directories specified for testing are volatile. The contents of these
directories are deleted before the build and run phases, respectively.
In all four cases, when creating directories, only the final path component is created. Parent
directories must already exist, i.e., ch-test uses the behavior of mkdir rather than mkdir-p.
Some of the tests exercise parallel functionality. If ch-test is run on a single node, multiple cores
will be used; if in a Slurm allocation, multiple nodes too.
The subset of tests to run mostly splits along two key dimensions. The phase is which parts of the
workflow to run. Different parts of the workflow can be tested on different systems by copying the
necessary artifacts between them, e.g. by building images on one system and running them on another. The
scope allows trading off thoroughness versus time.
PHASE must be one of the following:
build Image building and associated functionality, with the selected builder.
run Running containers and associated functionality. This requires a packed images directory
produced by a successful build phase, which can be copied from the build system if it’s not
also the run system.
rootemu
Test the root emulation modes (seccomp, fakeroot, and none) on various linux distributions.
examples
Example applications. Requires an unpacked images directory produced by a successful run phase.
all Execute phases build, rootemu, run, and examples, in that order.
mk-perm-dirs
Create the filesystem permissions directories. Requires --perm-dirs.
build-images
Build images from build phase, without running the associated tests.
clean Delete automatically-generated test files, and packed and unpacked image directories.
rm-perm-dirs
Remove the filesystem permissions directories. Requires --perm-dirs.
-f,--fileFILE[:TEST]
Run the tests in the given file only, which can be an arbitrary .bats file, except for
test.bats under examples, where you must specify the corresponding Dockerfile or Build file
instead. This is somewhat brittle and typically used for development or debugging. For example,
it does not check whether the pre-requisites of whatever is in the file are satisfied. Often
running build and run first is sufficient, but this varies.
If TEST is also given, then run only tests with name containing that string, skipping the
others. The separator is a literal colon. If the string contains shell metacharacters such as
space, you’ll need to quote the argument to protect it from the shell.
Scope is specified with:
-s,--scopeSCOPESCOPE must be one of the following:
• quick: Most important subset of workflow. Handy for development.
• standard: All tested workflow functionality and a selection of more important examples.
(Default.)
• full: All available tests, including all examples.
Image format is specified with:
--pack-fmtFMTFMT must be one of the following:
• squash-mount or 🐘: SquashFS archive, run directly from the archive using ch-run’s internal
SquashFUSE functionality. In this mode, tests that require writing to the image are skipped.
• tar-unpack or 📠: Tarball, and the images are unpacked before running.
• squash-unpack or 🎃: SquashFS, and the images are unpacked before running.
Default: $CH_TEST_PACK_FMT if set. Otherwise, if mksquashfs(1) is available and ch-run was
built with libsquashfuse support, then squash-mount, else tar-unpack.
Additional arguments:
-b,--builderBUILDER
Image builder to use. Default: $CH_TEST_BUILDER if set, otherwise ch-image.
--dry-run
Print summary of what would be tested and then exit.
-h,--help
Print usage and then exit.
--img-dirDIR
Set unpacked images directory to DIR. In a multi-node allocation, this directory may not be
shared between nodes. Default: $CH_TEST_IMGDIR if set; otherwise /var/tmp/${USER}.img.
--lustreDIR
Use DIR for run-phase Lustre tests. Default: CH_TEST_LUSTREDIR if set; otherwise skip them.
The tests will create, populate, and delete a new subdirectory under DIR, leaving everything
else in DIR untouched.
--pack-dirDIR
Set packed images directory to DIR. Default: $CH_TEST_TARDIR if set; otherwise
/var/tmp/${USER}.pack.
--pedantic(yes|no)
Some tests require configurations that are very specific (e.g., being a member of at least two
groups) or unusual (e.g., sudo to a non-root group). If yes, then fail if the requirement is
not met; if no, then skip. The default is yes for CI environments or people listed in
README.md, no otherwise.
If yes and sudo seems to be available, implies --sudo.
--perm-dirDIR
Add DIR to filesystem permission fixture directories; can be specified multiple times. We
recommend one such directory per mounted filesystem type whose kernel module you do not trust;
e.g., you probably don’t need to test your tmpfses, but out-of-tree filesystems very likely
need this.
Implies --sudo. Default: CH_TEST_PERMDIRS if set; otherwise skip the filesystem permissions
tests.
--sudo Enable things that require sudo, such as certain privilege escalation tests and
creating/removing the filesystem permissions fixtures. Requires generic sudo capabilities. Note
that the Docker builder uses sudodocker even without this option.
Install Now
