Boost.Build v2 user manual


This preliminary reference is intended to document everything currently implemeneted, but is not yet ready for any practical use.



Tutorial
Hello, world
Properties
Project hierarchy
Using libraries
Static and shared libraries
Prebuilt targets
Reference documentation
Overview
Features and properties
Defintions
Property Validity
Feature Attributes
Feature Declaration
Build Variants
Subfeatures
Initialization
Command line
Projects
Targets
Build process

Installation

When using package management or installers, Boost.Build is ready to work instantly. In other case, two steps are required:
  1. Place the Boost.Jam binary, called "bjam", somewhere in your PATH.
  2. Specify the location of Boost.Build files. You can either set environmental variable BOOST_BUILD_PATH, or create, at the top of your project, a file called boost-build.jam, with a single line:
    boost-build /path/to/boost.build ;
    
    The directory "examples" in the distribution already has this file, so you can build projects there and add new without doing anything.

To verify your installation, you can use bjam --version at the command line.

Tutorial

Hello, world

The simplest project that Boost.Build can construct is stored in examples-v2/hello directory. The targets are declared in a file called Jamfile, which contains the following:
   exe hello : hello.cpp ;
Even with this simple setup, you can do some interesting things. First of all, running "bjam" would build binary "hello" from hello.cpp, in debug version. After that, you can run
    bjam release 
which would create release version of the 'hello' binary. Note that debug and release version would be created in different directories, so if you want to switch from debug to release version and back, no recompilation is needed. Let's extend the example by adding another line to Jamfile:
   exe hello2 : hello.cpp ;
You can now rebuild both debug and release versions:
    bjam debug release
You'll see that two versions of "hello2" binary are linked. Of course, hello.cpp won't be recompiled. Now you decide to remove all build products. You do that with the following command
   bjam --clean debug release
It's also possible to create or clean only specific targets. Both following commands are legal and create or clean only files that belonging the the named binary:
   bjam hello2
   bjam --clean hello2

Properties

Boost.Build attempts to allow building different variants of projects, e.g. for debugging and release, or in single and multithreaded mode. In order to stay portable, it uses the concept of features, which is abstract aspect of build configuration. Property is just a (feature,value) pair. For example, there's a feature "debug-symbols", which can have a value of "on" or "off". When users asks to build project is a particual value, Boost.Build will automatically find the appropriate flags to the used compiler.

The "release" and "debug" in bjam invocation that we've seen are just are short form of specifying values of feature "variant". There is a lot of builtin features, and it's possible to write something like:

   bjam release inlining=off debug-symbols=on
The first command line element specified the value of feature "variant". The feature is very common and is therefore special — it's possible to specify only value. Another feature, "inlining" is not special, and you should use
   feature-name=feature-value
syntax for it. Complete description of features can be found here. The set of properties specified in the command line constitute build request — the desired properties for requested targets, or for the project in the current directory. The actual set of properties used for building is often different. For example, when compiling a program you need some include paths. It's not reasonable to ask the user to specify those paths with each bjam invocation, so must be specified in Jamfile and added to the build request. For another example, certain application can only be linked in multithreaded mode. To support such situations, every target is allowed to specify requirements -- properties that are required to its building. Consider this example:
   exe hello 
       : hello.cpp
       : <include>/home/ghost/Work/boost <threading>multi
In this case, when hello is build, the two specified properties will always be present. This leads to a question: what if user explictly requested single-threading. The answer is that requirement can affect build properties only to a certain degree: the requested and actual properties must be link-compatible. See link compatibility below. If they are not link compatible, the bulding of the target is skipped. Previously, we've added "hello2" target. Seems like we have to specify the same requirements for it, which results in duplication. But there's a better way. Each project (i.e. each Jamfile), can specify a set of attributes, including requirements:
   project 
       : requirements <include>/home/ghost/Work/boost <threading>multi 
       ;

   exe hello : hello.cpp ;
   exe hello2 : hello.cpp ;
The effect would be as if we specified this requirement for both "hello" and "hello2".

Project hierarchy

So far we only considered examples with one project (i.e. with one Jamfile). Typically, you'd have a lot of projects organized into a tree. At the top of the tree there's project root. This is a directory which contains, besides Jamfile, a file called "project-root.jam". Each other Jamfile has a single parent, which is the Jamfile in the nearest parent directory. For example, in the following directory layout:

    [top] 
      |
      |-- Jamfile
      |-- project-root.jam
      |
      |-- src
      |    |
      |    |-- Jamfile
      |    \-- app.cpp
      | 
      \-- lib
           |
           |-- lib1
           |    |
           |    |-- Jamfile
                |-- lib1.cpp
project root is at top. Both src/Jamfile and lib/lib1/Jamfile have [top]/Jamfile as parent project. Projects inherit all attributes (such as requirements) from their parents. When the same attributes are specified in the project, they are combined with inherited ones. For example, if [top]/Jamfile has
     <include>/home/ghost/local
in requirements, then all other projects will have that in their requirements too. Of course, any project can add additional includes. More details can be found in the section on projects. Projects are not automatically built when their parents are built. You should specify this explicitly. In our example, [top]/Jamfile might contain:
    build-project src ;
It will cause project in src to be built whenever project in [top] is built. However, targets in lib/lib1 will be built only if required. For example, there may be 10 targets, and two of them are used by targets in src/Jamfile. Then, only those two targets will be built.

Using libraries

Let's continue the above example and see how src/Jamfile can use libraries from lib/lib1. (TODO: need to make this section consistent with "examples-v2/libraries". Assume lib/lib1/Jamfile contains:
     lib lib1 : lib1.cpp ;
Then, to use this library in src/Jamfile, we can write:
     exe app : app.cpp ../lib/lib1/lib1 ;
When linking the "app" binary, the needed library will be used. But what is meant by "needed"? For example, we can request to build "app" with properties
    <optimization>full <cxxflags>-w-8080
Which properties must be used for "lib1"? The answer is that some properties are propagated — Boost.Build attemps to use dependencies with the same value of propagated features. The <optimization> feature is propagated, so both "app" and "lib1" will be compiled with full optimization. But <cxxflags> feature is not propagated: its value will be added as-is to compiler flags for "a.cpp", but won't affect "lib1". There is still a couple of problems. First, the library probably has some headers which must be used when compiling "app.cpp". We could use requirements on "app" to add those includes, but then this work will be repeated for all programs which use "lib1". A better solution is to modify lib/lib1/Jamfilie in this way:
    project 
        : usage-requirements <include>.
        ;

     lib lib1 : lib1.cpp ;
Usage requirements are requirements which are applied to dependents. In this case, <include> will be applied to all targets which use "lib1". You'd need to specify usage requirements only once, and programs which use "lib1" don't have to care about include paths any longer. Or course, the path will be interpreted relatively to "lib/lib1" and will be adjusted according to the bjams invocation directory. For example, if building from project root, the final compiler's command line will contain -Ilib/lib1. The second problem is that we hardcode the path to library's Jamfile. Imagine it's hardcoded in 20 different places and we change the directory layout. The solution is to use project ids. First, we assign a project id to Jamfile in lib/lib1:
    project lib1
        : usage-requirements <include>.
        ;
Second, we use the project id to refer to the library in src/Jamfile:
    exe app : app.cpp @/lib1/lib1 ;
This way, users of "lib1" do not depend on its location, only on id, which is supposedly stable. The only thing left, it to make sure that src/Jamfile knows the project id that it uses. We add to [top]/Jamfile the following line:
    use-project /lib1 : lib/lib1 ;
Now, all projects can refer to "lib1" using the symbolic name. If the library is moved somewhere, only a single line in the top-level Jamfile should be changed.

Static and shared libaries

While the previous section explained how to create and use libraries, it omitted one important detail. Libraries can be either static, which means they are included in executable files which use them, or shared (a.k.a. dynamic), which are only referred to from executables, and must be available at run time. Boost.Build can work with both types. By default, all libraries are shared. This is much more efficient in build time and space. But the need to install all libraries to some location is not always convenient, especially for debug builds. Also, if the installed shared library changes, all application which use it might start to behave differently.

Static libraries do not suffer from these problems, but considerably increase the size of application. Before describing static libraries, it's reasonable to give another, quite simple approach. If your project is built with <hardcode-dll-paths>true property, then the application will include the full paths for all shared libraries, eliminating the above problems. Unfortunately, you no longer can move shared library to a different location, which makes this option suitable only for debug builds. Further, only gcc compiler supports this option.

Building a library statically is easy. You'd need to change the value of <link> feature from it's deafault value shared, to static. So, to build everything as static libraries, you'd say

    bjam link=static
on the command line. The linking mode can be fine-tuned on per-target basis.
  1. Suppose your library can be only build statically. This is easily achieved using requirements:
        lib l : l.cpp : <link>static ;
       
    
  2. What if library can be both static and shared, but when using it in specific executable, you want it static? Target references are here to help:
        exe important : main.cpp helpers/<link>static ;
       
    
  3. What if the library is defined in some other project, which you cannot change. But still, you want static linking to that library in all cases. You can use target references everywhere:
        exe e1 : e1.cpp @/other_project/lib1/<link>static ;
        exe e10 : e10.cpp @/other_project/lib1/<link>static ;
       
    
    but that's far from being convenient. Another way is to introduce a level of indirection: create a local target, which will refer to static version of lib1. Here's the solution:
        alias lib1 : @/other_project/lib1/<link>static ;
        exe e1 : e1.cpp lib1 ;
        exe e10 : e10.cpp lib1 ;
       
    
    (Note, that the "alias" target type is not yet implemented, but it's quite simple to do. I bet it's waiting for you to do it ;-))

Prebuilt targets

We've just learned how to use libraries which are created by Boost.Build. But some libraries are not. At the same time, those libraries can have different versions (release and debug, for example), that we should select depending on build properties. Prebuilt targets provide a mechanism for that. Jamfile in lib/lib2 can contain:
    lib lib2
       : 
       : <file>lib2_release.a <variant>release
       ;

    lib lib2
       : 
       : <file>lib2_debug.a <variant>debug
       ;
This defines two alternatives for target "lib2", and for each one names a prebuilt file. Naturally, there are no sources. Instead, the <file> feature is used to specify the file name. Which alternative is selected depends on properties of dependents. If "app" binary should use "lib2", we can write:
    exe app : app.cpp @/lib/lib1/lib2 ../lib/lib2/lib2 ;
If we build release version of "app", then it will be linked with "lib2_release.a", and debug version will use "lib2_debug.a". Another important kind of prebuilt targets are system libraries — more specifically, libraries which are automatically found by the compiler. E.g. gcc uses "-l" switch for that. Such libraries should be declared almost like regular ones:
    lib zlib : : <name>z ;
We again don't specify any sources, but give a name which should be passed to the compiler. In this example, and for gcc compiler, the "-lz" option will be added. Paths where library should be searched can also be specified:
    lib zlib : : <name>z <search>/opt/lib ;
And, of course, two variants can be used:
    lib zlib : : <name>z <variant>release ;
    lib zlib : : <name>z_d <variant>debug ;
Of course, you'll probably never in your life need debug version of zlib, but for other libraries this is quite reasonable.

Reference

This section will document mostly high-level view of Boost.Build, mentioning appropriate modules and rules. The on-line help system must be used to obtain low-level documentation (see the help option).

Overview

The most fundemental entity in Boost.Build is main target. This is object that user want to construct from sources and keep up to date with regard to those sources. Typical examples of main targets are executable files and libraries.

Main targets are grouped in projects. Their main purpose is organization: related targets placed in one project, can then be built together, or share some definitions.

Main targets and projects are described mostly in declarative fashion. (TODO: what "declarative fashion" means to a user?) To make some use of them, user issues build request, which specifies what targets user wants to build, and what properties are desirable. Build request is not necessary explicit. Invoking the build system without parameters will build the project in current directory with default properties.

The properties describe various aspects of constructed objects. For portability, they are specified in a normalized form, for example

<optimization>full <inlining>off
Depending on the compiler used, this will be translated into appropriate flags.

Construction of each main target begins with finding properties for this main target. They are found by processing both build request, and target requirements, which give properties needed for the target to build. For example, a given main target might require certian defines, or will not work unless compiled in multithreaded mode. The process of finding properties for main target is described in property refinement.

After that, dependencies (i.e. other main targets) are build recursively. Build request for dependencies is not always equal to those of dependent — certain properties are dropped and user can explicitly specify desired properties for dependencies. See propagated features and target reference for details.

When dependencies are constructed, the dependency graph for this main target and for this property set is created, which describes which files need to be created, on which other files they depend and what actions are needed to construct those files. There's more that one method, and user can define new ones, but usually, this involves generators and target types.

Target type is just a way to classify targets. For example, there are builtin types EXE, OBJ and CPP. Generators are objects that know how to convert between different target type. When a target of a given type must be created, all generators for that type, which can handle needed properties, are found. Each is passed the list of sources, and either fails, or returns a dependency graph. If a generator cannot produce desired type from given sources, it may try to recursively construct types that it can handle from the types is was passed. This allows to try all possible transformations. When all generators are tried, a dependency graph is selected.

Finally, the dependency graph is passed to underlying Boost.Jam program, which runs all actions needed to bring all main targets up-to date. At this step, implicit dependencies are also scanned and accounted for, as described here.

Features and properties

Definitions

A feature is a normalized (toolset-independent) aspect of a build configuration, such as whether inlining is enabled. Feature names may not contain the '>' character.

The last two prohibited characters might well be present in defines. And what about dash?

Each feature in a build configuration has one or more associated values. Feature values may not contain the '<', ':', or '=' characters.

A property is a (feature,value) pair, expressed as <feature>value.

A subfeature is a feature which only exists in the presence of its parent feature, and whose identity can be derived (in the context of its parent) from its value. A subfeature's parent can never be another subfeature. Thus, features and their subfeatures form a two-level hierarchy.

A value-string for a feature F is a string of the form value-subvalue1-subvalue2...-subvalueN, where value is a legal value for F and subvalue1...subvalueN are legal values of some of F's subfeatures. For example, the properties <toolset>gcc <toolset-version>3.0.1 can be expressed more conscisely using a value-string, as <toolset>gcc-3.0.1.

A property set is a set of properties (i.e. a collection without dublicates), for instance: <toolset>gcc <runtime-link>static.

A property path is a property set whose elements have been joined into a single string separated by slashes. A property path representation of the previous example would be <toolset>gcc/<runtime-link>static.

A build specification is a property set which fully describes the set of features used to build a target.

Property Validity

For free features, all values are valid. For all other features, the valid values are explicitly specified, and the build system will report an error for the use of an invalid feature-value. Subproperty validity may be restricted so that certain values are valid only in the presence of certain other subproperties. For example, it is possible to specify that the <gcc-target>mingw property is only valid in the presence of <gcc-version>2.95.2.

Feature Attributes

Each feature has a collection of zero or more of the following attributes. Feature attributes are low-level descriptions of how the build system should interpret a feature's values when they appear in a build request. We also refer to the attributes of properties, so that a incidental property, for example, is one whose feature is has the incidental attribute.

TODO: document active features..

Feature Declaration

The low-level feature declaration interface is the feature rule from the feature module:
rule feature ( name : allowed-values * : attributes * )
A feature's allowed-values may be extended wit The build system will provide high-level rules which define features in terms of valid and useful combinations of attributes.

Build Variants

A build variant, or (simply variant) is a special kind of composite feature which automatically incorporates the default values of features that . Typically you'll want at least two separate variants: one for debugging, and one for your release code. [ Volodya says: "Yea, we'd need to mention that it's a composite feature and describe how they are declared, in pacticular that default values of non-optional features are incorporated into build variant automagically. Also, do we wan't some variant inheritance/extension/templates. I don't remember how it works in V1, so can't document this for V2.". Will clean up soon -DWA ]

When the build system tries to generate a target (such as library dependency) matching a given build request, it may find that an exact match isn't possible — for example, the target may impose additonal build requirements. We need to determine whether a buildable version of that target can actually be used.

The build request can originate in many ways: It may come directly from the user's command-line, from a dependency of a main target upon a library, or from a dependency of a target upon an executable used to build that target, for example. For each way, there are different rules whether we can use a given subvariant or not. However we currently only assume linking and therefore use a simple approach described in the following paragraph.

In general, there are many possible situations: a libary which is dependency of a main target and should be linked into it, target which is directly requested on the command line, or build executable which is used in the build process itself. At this moment we use a simple approach.

Two property sets are called link-compatible when targets with those property sets can be used interchangably. In turn, two property sets are link compatible when there's no link-incompatible feature which has different values in those property sets. Whenever requested and actual properties are link-compatible, it's OK. Otherwise, it's an error.

Definition of property refinement

When a target with certain properties is requested, and that target requires some set of properties, it is needed to find the set of properties to use for building. This process is called property refinement and is performed by these rules

  1. If original properties and required properties are not link-compatible, refinement fails.
  2. Each property in the required set is added to the original property set
  3. If the original property set includes property with a different value of non free feature, that property is removed.

Conditional properties

Sometime it's desirable to apply certain requirements only for specific combination of other properties. For example, one of compilers that you use issues a poinless warning that you want to suppress by passing a command line option to it. You would not want to pass that option to other compilers. Condititional properties allow to do that. Their systax is:

  property ( "," property ) * ":" property
For example, the problem above would be solved by:
exe hello : hello.cpp : <toolset>yfc:<cxxflags>-disable-pointless-warning ;

Initialization

bjam's first job upon startup is to load the Jam code which implements the build system. To do this, it searches for a file called "boost-build.jam", first in the invocation directory, then in its parent and so forth up to the filesystem root, and finally in the directories specified by the environment variable BOOST_BUILD_PATH. When found, the file is interpreted, and should specify the build system location by calling the boost-build rule:

      rule boost-build ( location ? )
If location is a relative path, it is treated as relative to the directory of boost-build.jam. The directory specified by location and directories in BOOST_BUILD_PATH are then searched for a file called bootstrap.jam which is interpreted and is expected to bootstrap the build system. This arrangement allows the build system to work without any command-line or environment variable settings. For example, if the build system files were located in a directory "build-system/" at your project root, you might place a boost-build.jam at the project root containing:
     boost-build build-system ;
In this case, running bjam anywhere in the project tree will automatically find the build system.

The default "bootstrap.jam", after loading some standard definitions, loads two files, which can be provided/customised by user: "site-config.jam" and "user-config.jam".

Locations where those files a search are summarized below:

search paths for configuration files
site-config.jam user-config.jam
Linux /etc
$HOME
$BOOST_BUILD_PATH
$HOME
$BOOST_BUILD_PATH
Windows $SystemRoot
$HOME
$BOOST_BUILD_PATH
$HOME
$BOOST_BUILD_PATH
Boost.Build comes with default versions of those files, which can serve as templates for customized versions.

Command line

The comamnd line may contain:

Command line arguments

Command line arguments specify targets and build request using the following rules. For example, the command line
    target1 debug gcc/runtime-link=dynamic,static
   
would cause target called target1 to be rebuild in debug mode, except that for gcc, both dynamically and statically linked binaries would be created.

Command line options

All of the Boost.Build options start with the "--" prefix. They are described in the following table.

Command line options
Option Description
--version Prints information on Boost.Build and Boost.Jam versions.
--help Access to the online help system. This prints general information on how to use the help system with additional --help* options.
--clean Removes everything instead of building. Unlike clean target in make, it is possible to clean only some targets.
--debug Enables internal checks.
--dump-projects Cause the project structure to be output.
--no-error-backtrace Don't print backtrace on errors. Primary usefull for testing.
--ignore-config Do not load site-config.jam and user-config.jam

Projects

Boost.Build considers every software it build as organized into projects, each of which corresponds to a single Jamfile. Projects are organized in a hierarchical structure, so each project may have a single parent project and a number of subprojects.

Each project is also associated with project root. That's a root for a tree of projects, which specifies some global properties.

Project root

Project root for a projects is the nearest parent directory which contains a file called project-root.jam. That file defines certain properties which apply to all projects under project root. It can:

Project attributes

For each project, there are several attributes.

Project id is a short way to denote a project, as opposed to the Jamfile's pathname. It is a hierarchical path, unrelated to filesystem, such as "boost/thread". Target references make use of project ids to specify a target.

Source location specifies the directory where sources for the project are located.

Project requirements are requirements that apply to all the targets in the projects as well as all subprojects.

Default build is the build request that should be used when no build request is specified explicitly.

The default values for those attributes are given in the table below. In order to affect them, Jamfile may call the project rule. The rule has this syntax:

    project id : <attributes> ;
   
Here, attributes is a sequence of (attribute-name, attribute-value) pairs. The list of attribute names along with its handling is also shown in the table below. For example, it it possible to write:
    project tennis 
        : requirements <threading>multi 
        : default-build release
        ;
   
Attribute Name for the 'project' rule Default value Handling by the 'project' rule
Project id none none Assigned from the first parameter of the 'project' rule. It is assumed to denote absolute project id.
Source location source-location The location of jamfile for the project Sets to the passed value
Requirements requirements The parent's requirements The parent's requirements are refined with the passed requirement and the result is used as the project requirements.
Default build default-build none Sets to the passed value
Build directory build-dir If parent has a build dir set, the value of it, joined with the relative path from parent to the current project. Otherwise, empty Sets to the passed value, interpreted as relative to the project's location.

Project relationship

There are three kinds of project relationships.

First is parent-child. This relationship is established implicitly: parent directories of a project are searched, and the first found Jamfile is assumed to define the parent project. The parent-child relationship affects only attribute values for the child project.

Second is build relationship. Some project may request to recursively build other projects. Those project need not be child projects. The build-project rule is used for that:

    build-project src ;   

The third kind is the 'use' relationship. In means that one project uses targets from another. It is possible to just refer to target in other projects using target id. However, if target id uses project id, it is required that the project id is known. The use-project rule is employed to guarantee that.

    use-project ( id : location )
   
It loads the project at the specified location, which makes its project id available in the project which invokes the rule. It is required that the id parameter passed to the use-project rule be equal to the id that the loaded project declared. At this moment, the id paremeter should be absolute project id.

Targets

There are two user-visible kinds of targets in Boost.Build. First are "abstract" — they correspond to things declared by user, for example, projects and executable files. The primary thing about abstract target is that it's possible to request them to be build with a particular values of some properties. Each combination of properties may possible yield different set of real file, so abstract target do not have a direct correspondence with files.

File targets, on the contary, are associated with concrete files. Dependency graphs for abstract targets with specific properties are constructed from file targets. User has no was to create file targets, however it can specify rules that detect file type for sources, and also rules for transforming between file targets of different types. That information is used in constructing dependency graph, as desribed in the next section. Note:File targets are not the same as targets in Jam sense; the latter are created from file targets at the latest possible moment. Note:"File target" is a proposed name for what we call virtual targets. It it more understandable by users, but has one problem: virtual targets can potentially be "phony", and not correspond to any file.

Main targets and main target alternatives

Main target is a named entity which can be build, for example a named executable file. To declare a main target, user invokes some of the main target rules, passing it things like list of sources and requirements.

It is possible to have different list of sources for different toolsets, therefore it is possible to invoke main target rules several times for a single main target. For example:

    exe a : a_gcc.cpp : <toolset> ;
    exe a : a.cpp ;
Each call to the 'exe' rule defines a new main target alternative for the main target a.exe. In this case, the first alternative will be used for the gcc toolset, while the second alternative will be used in other cases. TODO: document the exact selection method under "Build process" below.

Target identifiers and references

Target identifier is used to denote a target. It is described by the following grammar:

    target-id -> project-reference local-target-name
    project-reference -> [jamfile-location] [ "@" [project-id] ]
    jamfile-location -> pathname 
    project-id -> pathname
    local-target-name -> identifier
For example, valid target ids might be:
    a
    lib/b
    @/boost/thread
    /home/ghost/build/lr_library@parser/lalr1
To map the target id into target, the project where that target is contained is first found:
  1. If project-reference is empty, then the current project is used — i.e. the project there the target id occurs.
  2. If the project id is absolute, the project with that id is used.
  3. If the project id is relative, it is treated relatively to project-id of the project at jamfile-location. If that project does not declare project id, it is an error.
After that, the target given by local-target-name in the found project is used.

Target reference is used to specify a source target, and may additionally specify desired properties for that target. It has this syntax:

    target-reference -> target-id [ "/" requested-properties ]
    requested-properties -> property-path
For example,
    exe compiler : compiler.cpp libs/cmdline/<optimization>space ;
would cause the version of cmdline library, optimized for space, to be linked in even if the compiler executable is build with optimization for speed.
Ambiguity resolution

Target reference may have the same form as a pathname, for example lib/a. In order to determine if this is target reference or pathname, it is checked if there's a jamfile in the specified path. If there is one, it is loaded and if the specified target is declared by that project it is used. Otherwise, we just treat the target reference as a file name.

File targets

As described above, file targets corresponds to files that Boost.Build manages. User's may be concerned about file targets in three ways: when declaring file target types, when declaring transformations between types, and when determining where file target will be placed. File targets can also be connected with actions, that determine how the target is created. Both file targets and actions are implemented in the virtual-target module.
Types
A file target can be given a file, which determines what transformations can be applied to the file. The type.register rule declares new types. File type can also be assigned a scanner, which is used to find implicit dependencies. See dependency scanning below.

Target paths

To distinguish targets build with different properties, they are put in different directories. Rules for determining target paths are given below:

  1. All targets are placed under directory corresponding to the project where they are defined.
  2. Each non free, non incidental property cause an additional element to be added to the target path. That element has the form <feature-name>-<feature-value> for ordinary features and <feature-value> for implicit ones. [Note about composite features].
  3. If the set of free, non incidental properties is different from the set of free, non incidental properties for the project in which the main target that uses the target is defined, a part of the form main_target-<name> is added to the target path. Note:It would be nice to completely track free features also, but this appears to be complex and not extremely needed.

For example, we might have these paths:

    debug/optimization-off
    debug/main-target-a
   

Build request

Build process

Given a list of targets ids and a build request, building goes this way. First, for each id we obtain the abstract targets corresponding to it. This also loads all necessary projects. If no target id is given, project in the current directory is used. Build request is expanded, and for each resulting property set, the generate method of all targets is called, which yields a list of virtual targets. After that all virtual targets are actualized, and target "all" is set to depend on all created actual targets. Lastly, depending on whether --clean option was given, either target "all" or target "clean" is updated. Generation of virtual target from abstract one is performed as follows:

Generators

To construct a main target with given properties from sources, it is required to create a dependency graph for that main target, which will also include actions to be run. The algorithm for creating the dependency graph is described here.

The fundamental concept is generator. If encapsulates the notion of build tool and is capable to converting a set of input targets into a set of output targets, with some properties. Generator matches a build tool as closely as possible: it works only when the tool can work with requested properties (for example, msvc compiler can't work when requested toolset is gcc), and should produce exactly the same targets as the tool (for example, if Borland's linker produces additional files with debug information, generator should also).

Given a set of generators, the fundamental operation is to construct a target of a given type, with given properties, from a set of targets. That operation is performed by rule generators.construct and the used algorithm is described below.

Selecting and ranking viable generators

Each generator, in addition to target types that it can produce, have attribute that affects its applicability in particular sitiation. Those attributes are:

  1. Required properties, which are properties absolutely necessary for the generator to work. For example, generator encapsulating the gcc compiler would have <toolset>gcc as required property.
  2. Optional properties, which increase the generators suitability for a particual build.
Generator's required and optional properties may not include either free or incidental properties. (Allowing this would greatly complicate caching targets).

When trying to construct a target, the first step is to select all possible generators for the requested target type, which required properties are a subset of requested properties. Generators which were already selected up the call stack are excluded. In addition, if any composing generators were selected up the call stack, all other composing generators are ignored (TODO: define composing generators). The found generators assigned a rank, which is the number of optional properties present in requested properties. Finally, generators with highest rank are selected for futher processing.

Running generators

When generators are selected, each is run to produce a list of created targets. This list might include targets which are not of requested types, because generators create the same targets as some tool, and tool's behaviour is fixed. (Note: should specify that in some cases we actually want extra targets). If generator fails, it returns an empty list. Generator is free to call 'construct' again, to convert sources to the types it can handle. It also can pass modified properties to 'constuct'. However, a generator is not allowed to modify any propagated properties, otherwise when actually consuming properties we might discover that the set of propagated properties is different from what was used for building sources.

For all targets which are not of requested types, we try to convert them to requested type, using a second call to construct. This is done in order to support transformation sequences where single source file expands to several later. See this message for details.

Selecting dependency graph

After all generators are run, it is necessary to decide which of successfull invocation will be taken as final result. At the moment, this is not done. Instead, it is checked whether all successfull generator invocation returned the same target list. Error is issued otherwise.

Property adjustment

Because target location is determined by the build system, it is sometimes necessary to adjust properties, in order to not break actions. For example, if there's an action which generates a header, say "a_parser.h", and a source file "a.cpp" which includes that file, we must make everything work as if a_parser.h is generated in the same directory where it would be generated without any subvariants.

Correct property adjustment can be done only after all targets are created, so the approach taken is:

  1. When dependency graph is constructed, each action can be assigned a rule for property adjustment.
  2. When virtual target is actualized, that rule is run and return the final set of properties. At this stage it can use information of all created virtual targets.

In case of quoted includes, no adjustment can give 100% correct results. If target dirs are not changed by build system, quoted includes are searched in "." and then in include path, while angle includes are searched only in include path. When target dirs are changed, we'd want to make quoted includes to be search in "." then in additional dirs and then in the include path and make angle includes be searched in include path, probably with additional paths added at some position. Unless, include path already has "." as the first element, this is not possible. So, either generated headers should not be included with quotes, or first element of include path should be ".", which essentially erases the difference between quoted and angle includes. Note: there only way to get "." as include path into compiler command line is via verbatim compiler option. In all other case, Boost.Build will convert "." into directory where it occurs.

Transformations cache

Under certain conditions, an attempt is made to cache results of transformation search. First, the sources are replaced with targets with special name and the found target list is stored. Later, when properties, requested type, and source type are the same, the store target list is retrieved and cloned, with appropriate change in names.

Dependency scanning

Dependency scanning is the process of finding implicit dependencies due to "include" statements and similar things. It has to take into account two things:

Dependency scanning is implemented by objects called scanners. See documentation for the "scanner" module to detail.

Regarding the first problem, we really have no choice. We can't treat the same actual target differently depending on from where it is used. Therefore, when handling of includes differers depending on actions, we have to duplicate targets and assign different properties to it.

For the reason, when actualizing a virtual target we optionally pass the needed scanner to the "virtual-target.actualize" method. When no scanner is passed, a new actual target is created, with it's dependencies and updating actions set accordingly. When a particular scanner is specified, a new actual target is created. That target will depend on target created without scanner. In effect, this will allow to use different scanners for the same file.

Generated headers
Let me explain what I find the right semantic, first without any subvariants. We have a target "a.cpp" which includes "a_parser.h", we have to search through all include directories, checking:
  1. If there's such file there, or
  2. If there's a target of the same name, bound to that dir via LOCATE_TARGET.
Jam allows to do 1 via SEARCH variable, but that's not enough. Why can't we do simpler: first check if there's target of the same name? I.e. including of "a_parser.h" will already pick generated "a_parser.h", regardless of search paths? Hmm... just because there's no reason to assume that. For example, one can have an action which generated some "dummy" header, for system which don't have the native one. Naturally, we don't want to depend on that generated headers. To implement proposed semantic we'd need a new builtin. We can do this in Jam code, but really, this belongs to core. Using GLOB and jam code would just duplicate existing binding functionality and be inefficient. New builtin will accept a name of new target and a list of directories. It will perform the search as explained above and return either the name of exising target that it found, or create a new target with that name that it was passed. So, we'd write something like
    INCLUDES $(<) : [ SEARCH_FOR_TARGET $(>) : $(SEARCH_PATH) ] ;
What shall we do when using subvariants. For user, subvariants must be more or less transparent. If without subvariant a header was generated to a certain directory, everything must work. Suppose that file a.cpp belongs to a dependency graph of main target a. Include paths are
     "/usr/include" "/home/t" "."
We start by finding all places where headers that are part of a's dependency graph are generated. We insert those places to the include paths, immediately after ".". For example, we might end with:
     "/usr/include" "/home/t" "." "build"
As a result:
  1. File "a.cpp" will be correctly compiled. Note that it's already necessary to adjust paths to ensure this. We'll have to add target paths for all generated headers, because determining the exact set of additional include path for each source -- i.e the set of headers that it uses --- will be hard.
  2. With the proposed SEARCH_FOR_TARGET rule, dependency on generated header will work magically --- it would find the "a_parser.h" target bound via LOCATE_TARGET to "build" and we'll call INCLUDE on that found target, instread of creating a completely unrelated one.

Last modified: Oct 10, 2002

© Copyright Vladimir Prus 2002. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.