The *-config Scripts and the *.m4 Macro Files

The various *-config scripts (vpr-config, tweek-config, vrjuggler-config, etc.) play a vital role in the design of the global build. Unfortunately, this is also where the global build gets complex. Here, code that is intended for use by users of VR Juggler and associated modules is put to use by the code that compiles everything. It makes use of a strict set of behaviors wherein various environment settings and command-line options form a hierarchy of preferences and fallbacks. If anything goes wrong with a user's configuration process, it is almost always related to a .m4 file or a -config script giving unexpected results because of a misused command-line option or a “dirty” environment.

A Typical -config Script

We will now examine a typical -config script. We will not focus on any script in particular, but instead, we will describe the fundamental concepts and requirements shared by all implementations. Readers interested in implementations to use as references should consider the following:

  • vpr-config: This is the most basic -config script. It has no external dependencies, and it only deals with one library.

  • tweek-config: This script depends on vpr-config for proper execution, and it deals with some interesting special cases. Namely, it must be able to inform callers about information relating to C++, Java, and IDL. This script deals with one C++ library and multiple Java libraries distributed as JAR files.

  • vrjuggler-config: This script is interesting because it has the most dependencies (it depends on vpr-config, tweek-config, jccl-config, gadgeteer-config, and sonix-config) and because it deals with multiple libraries (libJuggler, libJuggler_ogl, and libJuggler_pf).

To proceed with the abstract discussion, we will now describe the job that must be performed by all -config scripts. We then explain how external dependencies are managed. We conclude with a discussion of how a -config script is generated as part of the configuration process.

The Job

The job of any -config script is simple: provide the information needed to compile against the associated library. This information can work in the context of building a higher level library or an application. The basic information that must be provided is as follows:

  • Module version number

  • C++ compiler flags including header paths and compiler-specific options such as -fexceptions or -LANG:std

  • C++ linker flags separated into two categories:

    1. The basic list of libraries that are distributed with the module in question

    2. The complete list of external dependencies needed to link an application

  • Static linking options

  • Profiled library linking options (if profiled libraries are available)

Other information may be provided as necessary (see tweek-config and jccl-config, for example).

External Dependencies

In the Juggler Project, we use the -config scripts in an interesting manner. A given script, say gadgeteer-config, will call all the -config scripts of the modules on which it depends. The collected output is compressed and returned to the user. In this way, we avoid trying to manage all the dependency information in every module. Instead, we rely on each module to report its information correctly. Then, the highest level module only has to collect it and print it out.

The key to this functionality is that all the -config scripts can be found in the user's path. Furthermore, because our -config scripts are written in Perl, we have easy access to the path used to invoke each script. (This is actually a side effect of the scripts being in the user's path, and the path information would be available regardless of the scripting language. Perl just makes it easy to extract and operate on the given information.)

The dependencies only come into play for certain information requests. Those requests are the following:

  • C++ flags (--cxxflags)

  • Include paths (--include)

  • Basic libraries (--libs)

  • External dependency libraries (--extra-libs)

Each of the above iteratively calls the -config script(s) from the dependency module(s) using at least a subset of the arguments specified by the user on the command line. One additional argument is given that reduces the amount of output: --min. This causes each script to print out only the minimal information needed for compiling. The motivation for doing this is to keep the compile lines short whenever possible.

Script Generation

Finally, we explain the last detail relating to all -config scripts used in the Juggler Project: script generation. As part of the global build, all these scripts must be generated at configuration time. This is necessary for each subsequent module to be configured correctly. That means that each module's configure script is making use of “external-config scripts to get command-line arguments and version information. This allows the use of pre-existing installations of dependencies. For example, Gadgeteer depends on JCCL and VPR. Normally, it would satisfy those dependencies using source from the same tree, but a user may already have JCCL and VPR built and installed. By setting up his or her path correctly, vpr-config and jccl-config can be found, and Gadgeteer can be built using the existing installations.

As of this writing, there are no hard and fast rules regarding script generation other than the fact that they must be generated as part of module configuration. The prevailing convention within a given configure.in is to make separate substitution variables used only in the -config.in template file. That is to say that these variables are separate from those used in template makefiles and other .in templates. The separation is done through syntax alone. Those variables that are substituted in the generation of a -config script are spelled using lowercase letters exclusively. Other substitution variables are spelled using all uppercase letters.

Again, this is not a hard and fast rule. Indeed, some variables are shared between all files (for example, $USE_GCC or $MAJOR_VERSION). In general, such exceptions are allowed because there is no difference in usage between a -config script and a generated makefile. The important distinctions arise with compiler and linker flags. In particular, there are some flags that should only be used in the process of building a given module but should not be exposed to users. An example is the MIPSpro -Woff flag that is used to disable compile-time warnings. Exporting this option would force users to disable the same warnings whether they want to or not. Essentially, it is up to the module build system author to use good judgement when deciding what to export and what to use internally.

To summarize, the separation of the substitution variables is done for two reasons:

  1. To manage potential differences in semantics between module compilation and module use.

  2. To make it clear to readers of the relevant files which variables are used for which purpose.

To date, this mechanism has worked well (at least for those few who know about it). In one case, failure to follow this convention caused compilation of a module to break because a linker variable was serving double duty. This variable was setting internal linker options and forcing the use of those same options for external code.

A Typical .m4 File

With an understanding of what information a -config file provides, we can move on to the m4 macros that make use of those files at configuration time. Each module in the Juggler Project has a corresponding m4 macro suitable for use within an Autoconf configure.in file. For example, Gadgeteer has a macro called GADGETEER_PATH. This macro is used in VR Juggler's configure.in to find a usable Gadgeteer installation.

The basic concept behind all these macros is the same: provide a way for a configure script to detect a usable installation. The way this is done is fairly straightforward. The basic step-by-step process is as follows:

  1. Find the module's -config script. If the script cannot be found, execute the user-specified failure steps and “return[4].

  2. Using the -config script, get the version of the installation and compare it against the user-specified minimum required version.

  3. If the version is sufficient, execute the user-specified success steps and set variables for compiler and linker arguments using the -config script. If the version comparison fails, execute the user-specified failure steps and “return”.

We now proceed into the details of a typical .m4 file. First, we will cover the mechanism used to deal with paths to installations. Then, we explain what variables must be set by a module's m4 macro.

Path Setting Preferences

The most complex part about understanding the Juggler Project .m4 files is the management of path setting preferences. The path in question is the path to the installed module. The installation may be on the local file system, or it may exist as a developer pseudo-installation (refer to the section called “The Developer Installation” for more information on that topic). The specification of that path by the user is where we now direct our interest. The following list gives the path setting preferences in order of decreasing preference (i.e., the first has highest precedence, and the last has lowest precedence):

  1. The _CONFIG environment variable which gives the full path to the module's -config script. The name of the environment depends on the specific .m4 file. For example, vpr.m4 checks for $VPR_CONFIG. In general, the environment variable name should match the name of the -config script except in capitalization and the use of an underscore (_) instead of a hyphen (-).

  2. The command-line argument --with-<module>-exec-prefix which specifies the directory containing the -config script. Here, the string <module> depends on the way the .m4 file is written to declare accepted command-line arguments.

  3. The command-line argument --with-<module>-prefix which specifies the directory containing the full module installation. The named directory must contain a bin subdirectory, and this subdirectory must contain the module's -config script.

  4. The module's _BASE_DIR environment variable. The directory named by the environment variable must contain a bin subdirectory that in turn contains the module's -config script.

  5. The user's path which must include the directory that contains the module's -config script.

The magic that happens in configure.pl is a result of setting the appropriate _CONFIG environment variable and extending the path to include the various module directories in the build tree. Through the combination of these steps, a given -config script is found using the _CONFIG environment variable, and any other -config scripts it needs are found using the path. For each module that is built, the environment in which configure.pl runs is extended. Refer back to the section called “Build Configuration File” for information about how the necessary file and environment variable names are provided to configure.pl.

Variables Defined

Upon successful completion of step 3 (see above), there are several shell variables defined for the calling code to use. Typically, these variables are concatenated with other variables to form the full set of options passed to the compiler and the linker. In order to maintain consistency across all the .m4 files, it is important to know what variables must be defined and why.

First, the variables can be separated into two broad categories: minimal and maximal. The basic idea with minimal versus maximal flags is to allow the user some flexibility in composing the full command-line options. The minimal variables provide only the minimum amount of information needed for compiling. No dependency information is included. For example, such settings usually include one or two header path extensions (-I options) and mandatory compiler flags such as -LANG:std. The maximal variables, on the other hand, include all the information needed for compiling including dependency data. More concretely, the minimal C++ compiler flags for Gadgeteer would give the header path for the Gadgeteer headers and any mandatory compiler flags. The maximal C++ compiler flags would include the minimal information as well as flags relating to JCCL, Tweek, and VPR (in that order). The same would be true for linker flags. To summarize, the minimal variables must be mixed with other minimal variables if the module has dependencies; the maximal variables can stand on their own.

Within the minimal and maximal categories, there are two more categories: compiler flags and linker flags. Based on the discussion above, this distinction should be fairly obvious. This distinction is made to deal with platforms where the compiler is not used to perform the link stage. Since some compilers can call the linker as necessary, linking flags[5] suitable for use with the compiler are provided. The distinction is made by the variable name. Linking flags that can be passed to the compiler have _CC_ in their name; linking flags for the linker have _LD_ in their name.

Continuing with the subdivision, the linking flags are divided into two categories: static and dynamic. This is done to allow static linking of one or more libraries instead of dynamic linking. The usual default would be dynamic linking, and the variable names reflect that. Those variables that contain static linking flags have _STATIC_ in their names.



[4] Actually, these macros do not return because the code is inlined. Moreover, the user-specified failure steps may include halting the configure script and exiting with failure status.

[5] Note the distinction between linker flags (flags for the linker program) and linking flags (flags used to link object files).