The testing system for Boost.Build is derived from the TestCmd Python module used for testing the Scons build tools. The changes are very few, and I hope that some of them may be adopted by TestCmd in future.
Unfortunately, there are not documentation for TestCmd, apart from the comments in code, so the most essential base methods will be described here.
The basic steps of testing a build system behaviour are:
Prior to using the test system, make sure you have checked out the Boost CVS and have successfully build Jam -- which means that bjam executable is available.
For example, suppose that you've placed under tools/build/test a file test1.py and a subdirectory test1 with files foo.cpp, Jamfile and Jamrules. A definition of function main is the only content of foo.cpp, Jamfile builds an executable foo from foo.cpp and Jamrules is empty. Finally, test1.py contains:
from BoostBuild import Tester, List
# Create a temporary working directory
t = Tester()
# Make content of the working directory equal to the content of the 'test1' directory.
t.set_tree('test1')
# Invoke 'bjam -sTOOLS=borland' and record which changes were made
t.run_build_system("-sTOOLS=borland")
# First, create a list of three pathnames
file_list = List("bin/foo/borland/debug/runtime-link-dynamic/foo") * List(".exe .obj")
# Second, assert that those files were added as result of the last build system invocation.
t.expect_addition(file_list)
# Invoke the build system once again
t.run_build_system("clean")
# Check if the files added previously were removed.
t.expect_removal(t.mul("bin/foo/borland/debug/runtime-link-dynamic/foo", [".exe", ".obj", ".tds"]))
This is all needed for a minimal test (and to find a bug in the borland toolset!). Running the test1.py gives the following output:
File bin/foo/borland/debug/runtime-link-dynamic/foo.tds not removed as expected
FAILED test of D:\MyDocu~1\Work\build\boost-build\boost-build -d0
at line 144 of TestBoostBuild.py (expect_removal)
from line 12 of test1.py
Overview of the most important methods of class TestBoostBuild follows.
The class TestBoostBuild creates a temporary directory in its constructor and changes to that directory. It can be modified by calling these methods:
The method read, inherited from the TestCmd class, can be used to read any file in the working directory and check its content. TestBoostBuild adds another method for tracking changes. Whenever build system is run (via run_build_system), the state of working dir before and after running is recorded. In addition, difference between the two states -- i.e. lists of files that were added, removed, modified or touched -- is stored in two member variables, tree_difference and unexpected_difference.
After than, the test author may specify that some change is expected, for example, by calling expect_addition("foo"). This call will check if the file was indeed added, and if so, will remove its name from the list of added files in unexpected_difference. Likewise, it's possible to specify that some changes are not interesting, for example a call ignore("*.obj") will just remove every files with ".obj" extension from unexpected_difference.
When test has finished with expectations and ignoring, the member unexpected_difference will contain the list of all changes not yet accounted for. It is possible to assure that this list is empty by calling expect_nothing_more member function.
Any of the expect* methods below will fail the test if the expectation is not met. It is also possible to perform manually arbitrary test and explicitly cause the test to either pass or fail. Ordinary filesystem functions can be used to work with the directory tree. Methods pass_test and fail_test are used to explicitly give the test outcome.
Typically, after test termination, the working directory is erased. If is possible to prevent it by setting environmental variables PRESERVE, PRESERVE_PASS and PRESERVE_FAIL. Non-zero value of the first makes the working directory preserved in all cases, while the other variables apply only to specific outcomes. When a directory is preserved, its name will be printed to the standard output.
The test system is composed of class Tester, derived form TestCmd.TestCmd, and helper class List. The methods of Tester, and the class List are described below.
The documentation frequently refer to filename. In all cases, files are specified in unix style: a sequence of components, separated by "/". This is true on all platforms. In some contexts, a list of files is allowed. In that case any object with sequence interface is allowed.
Effects:
bjam by
default) and build system files, assuming that the current directory is
tools/build/test. Formulates jam invocation command, which
will include explicit setting for BOOST_BUILD_PATH variable
and arguments passed to this methods, if any. This command will be used
by subsequent invocation of run_build_system. Finally,
initializes the base class.Effects:
Replaces the content of the current working directory with the content of directory at tree_location. If tree_location is not absolute pathname, it will be treated as relative to self.original_workdir. This methods also explicitly makes the copied files writeable.
Effects:
Writes the specified content to the file given by name under the temporary working directory. If the file already exists, it is overwritten. Any required directories are automatically created.
Effects:
Equvivalent to self.write(self.read(src), dst).
Effects:
Sets the access and modification times for all files in names to the current time. All the elements in names should be relative paths.
Effects:
Stores the new state of the working directory in self.tree. Computes the difference between previous and current trees and store them in variables self.tree_difference and self.unexpected_difference.
Both variables are instances of class tree.Trees_different, which have four attributes: added_files, removed_files, modified_files and touched_files. Each is a list of strings.
Effects:
Read the specified file and returns it content. Raises an exception is the file is absent.
Effects:
Read the specified file and returns it content, after removing trailing whitespace from every line. Raises an exception is the file is absent.
Rationale:
Althought this method is questionable, there are a lot of cases when jam or shells it uses insert spaces. It seems that introducing this method is much simpler than dealing with all those cases.
Accordingly to the number of changes kinds that are detected, there are four methods that specify that test author expects a specific change to occur. They check self.unexpected_difference, and if the change is present there, it is removed. Otherwise, test fails.
Each method accepts a list of names. Those names use / path separator on all systems. Additionaly, the test system translates suffixes appropriately. For the test to be portable, suffixes should use Windows convention: exe for executables, dll for dynamic libraries and lib for static libraries.
Note: The List helper class might be useful to create lists of names.
Note: The file content can be examined using TestCmd.read function.
The members are:
There's also a member expect_nothing_more, which checks that all the changes are either expected or ignored, in other words that unexpected_difference is empty by now.
Lastly, there's a method to compare file content with expected content:
expect_content(self, name, content, exact=0)The method fails the test if the content of file identified by 'name' is different from 'content'. If 'exact' is true, the file content is used as-is, otherwise, two transformations are applied:
There are five methods which ignore changes made to the working tree. They silently remove elements from self.unexpected_difference, and don't generate error if element is not found. They accept shell style wildcard.
The following methods correspond to four kinds of changes:
The method ignore(self, wildcard) ignores all the changes made to files that match a wildcard.
Effects: Cause the test to fail if condition is true.
Effects: Splits the string on unescaped spaces and tabs. The split components can further be retrieved using standard sequence access.
Effects: Returns an List instance, which elements are all possible concatenations of two string, first of which is from self, and second of which is from other.
The class also defines __str__ and __repr__ methods. Finally, there's __coerce__ method which allows to convert strings to instances of List.
Example:
l = "a b" * List("c d")
for e in l:
print e
will output
ac
ad
bc
bd
Last modified: May 5, 2002
© Copyright Vladimir Prus 2002. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.