The Glue Makefile

Each module in the Juggler Project has a glue makefile in its top-level directory that directs compilation of the module's main code base[7]. Every module's top-level makefile employs recursive calls to GNU make to build its code. The recursion occurs based on the source code directory structure. An iterative loop over each subdirectory is used make the recursive call to GNU make. Behind the scenes, all of this occurs within Doozer++ makefile stubs, but it is directed by variable settings in the glue makefile.

In this section, we will discuss how the glue makefiles work so that they may be extended correctly. We begin by explaining the basic structure employed by most of the glue makefiles found in the Juggler Project source tree. Learning how to extend those makefiles follows from that.

Basic Structure

In all but one module in the Juggler Project, the basic structure of the glue makefile is the same: one makefile that defines high-level targets, and one included makefile that does all the work. In this subsection, we explain the way these two files work to make up the glue that holds a module's build system together. We begin with the simpler of the two: Makefile.in.

Makefile.in

All modules have to have a Makefile.in in their top-level source directory. The real makefile is generated as part of executing the configure script. In most cases, the contents of this file is roughly the same. The file is divided into sections, and we will examine each section in turn.

Define Default Target

As with most makefiles, the first thing done is the definition of a default target, and the glue makefile is no exception. In most cases, the default target is one that is useful to developers. In Doozer++ terms, that target is 'debug'.

Definition of a default target unfortunately has special significance if dpp.subdir.mk is used. In order for a recursive build to execute based on its definition in Doozer++, the file dpp.subdir.mk must have the special target called 'default'. Because it is not possible with GNU make to test if a target has already been defined, a workaround must be used. Namely, the variable $(DEFAULT_SET) must be defined if a makefile that includes dpp.subdir.mk defines its own 'default' target. Thus, most top-level Makefile.in files have the following as the first three lines:

default: debug

DEFAULT_SET=	1

It is possible that some refactoring of dpp.subdir.mk might eliminate the need for this. As of this writing, it is not considered a critical change.

Include Project-Wide Variable Settings

Since VR Juggler 1.0, the build system has collected project-wide variable settings into a single file so that it may be included by all makefiles in the module's core build system. In VR Juggler 2.0, the (Autoconf-generated) file is called make.defs.mk[8]. It is always included using the following line:

include @topdir@/make.defs.mk
Set Local Variables

Once make.defs.mk has been included, local variable settings are performed. These can provide further customizations on variables set in make.defs.mk, but usually, the variables are used only within the scope of the glue makefile. That is, the variables are needed exclusively for directing the process of building the module source code.

For example, a decision to build a certain subset of the code base may be made at this level. Alternatively, a decision about how to build the code may be made. In most cases, such a decision is whether to build profiled versions of the library (or libraries).

The most important local variable setting is that of the list of subdirectories in the module. This list is provided through the $(SUBDIR) variable. Due to the structure of the various modules in the Juggler Project, there would be nothing to do without providing Doozer++ with the list of subdirectories. The way $(SUBDIR) is assigned its value may vary depending on the specific structure of a given module.

Define High-Level Targets

The main purpose of a project's Makefile.in is the definition of high-level build targets. These are the targets that users and developers will run to compile the module. Such targets include 'world', 'release', 'debug', and 'clobber'. The first two must be defined manually; the third and fourth are inherited through the use of Doozer++.

The high-level targets make use of low-level targets that exist in Doozer++ and, usually, in Makefile.inc.in. The idea is that a casual (or experienced) user can look at Makefile.in alone and understand how to perform common builds of a given module. Makefile.inc.in, with all its ugly details, is hidden—more or less.

Include External Makefiles

Finally, the top-level makefile must include the external that usually have all the hard bits. In most cases, this means having the following two lines as the last of Makefile.in:

include $(MKPATH)/dpp.subdir.mk
include Makefile.inc

Makefile.inc.in

In the context of the glue makefile, this particular file is often quite complex. If there is any one place in a module's build system that is hard to understand, it is within this file. However, the whole purpose of having this file is to collect all the complex build details into one place that is “out of the way” in some sense.

Due to the complexity and the wide variation in requirements between modules, we cannot examine a specific Makefile.inc.in and still provide a general explanation. Thus, we will review only the commonalities. When possible, we will cover important aspects that may not necessarily be used by all glue makefiles. Before we do that, we will begin our discussion with an review of the concepts behind Makefile.inc.in and why it is important.

Basic Concepts

As has been stated previously, the fundamental idea behind the top-level Makefile.inc.in is to hide complexity. More explicitly, this file should contain targets that the user will not execute in most circumstances. By putting such targets in this file, a casual reader need not ever see the complexity of the glue makefile. He or she should only have to open Makefile.in (or the generated Makefile) to get an understanding of how the build works. Of course, some details are encapsulated within Doozer++ makefile stubs, so getting a complete understanding is not necessarily a straightforward process. The extensive comments at the top of and throughout Makefile.in (and documents such as this) are there to fill in the gaps.

While it is very difficult to separate all targets cleanly based on common usage, there are some targets that definitely fit into this file well. Such targets are only called by Doozer++ through a mechanism similar to a C callback. Users of the build system should rarely, if ever, need to execute such targets, especially because Doozer++ may set up a special execution environment for the targets. Callback targets may include the following:

  • The targets used to compile the final binaries for the module's libraries, including static, dynamic, and profiled versions.

  • Any targets used as “pre” and “post” steps of larger targets. Larger targets would include those used for building and installing a module.

Library Name Construction

The responsibility for putting all the pieces together for the final library names rests on this file. Based on variable definitions in make.defs.mk (see below), the final library names must be defined so that Doozer++ knows what to build. The configure script provides the platform-specific details such as file extensions so that the code in Makefile.inc is relatively simple. For example, assuming the existence of a variable $(MY_LIBRARY), the following provides all the information needed to direct the compilation of profiled and non-profiled libraries, of both the static and the dynamic variety:

STATICLIB_EXT=	@STATICLIB_EXT@
DYNAMICLIB_EXT=	@DYNAMICLIB_EXT@

LIBS=		$(MY_LIBRARY)
STATIC_LIBS=	$(LIBS)
DYNAMIC_LIBS=	$(LIBS)

Using the above variables (and more settings from make.defs.mk), we can construct the names of various library versions. These names are then used internally for the targets that build the actual libraries. The name construction is performed as follows:

MY_LIB_STATIC=		$(MY_LIBRARY).$(STATICLIB_EXT)
MY_LIB_DYNAMIC=		$(MY_LIBRARY).$(DYNAMICLIB_EXT)
MY_PROF_LIB_STATIC=	$(MY_LIBRARY)$(PROFLIB_EXT).$(STATICLIB_EXT)
MY_PROF_LIB_DYNAMIC=	$(MY_LIBRARY)$(PROFLIB_EXT).$(DYNAMICLIB_EXT)

Note

The above hinges on the fact that $(PROFLIB_EXT) is non-empty. Otherwise, the targets we will examine in the next section would not be named correctly. If profiled libraries are not supported by the rest of the module's build system, do not use the profiled variants shown above.

Library Build Targets

Once we know the names of the libraries we will build, we can define the targets that actually build them. In nearly every case, these targets are very close to being identical between modules. Only rarely are there module-specific variations. The following shows targets for building static profiled and non-profiled versions of $(MY_LIBRARY):

$(LIBDIR)/$(MY_LIB_STATIC) $(LIBDIR)/$(MY_PROF_LIB_STATIC): $(OBJDIR)/*.$(OBJ(1)EXT)
	@$(SHELL) $(MKINSTALLDIRS) $(LIBDIR)                                        (2)
	$(AR) $(AR_NAME_FLAG)$@ $(OBJDIR)/*.$(OBJEXT)                               (3)
	$(RANLIB) $@                                                                (3)
	cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./                 (4)
1

This defines the targets for the profiled and non-profiled static libraries. Both depend on all the object files in $(OBJDIR).

2

This ensures that the directory where the library binary will be build exists. The variable $(MKINSTALLDIRS) is set via make.defs.mk.

3

These lines together build the static library. The use of $(RANLIB) is optional as it may not be supported or necessary with all compilers. The configure script makes this decision and sets $(RANLIB) accordingly.

4

This makes a symlink to the newly compiled static library in $(LIBDIR_BASE). Based on variable assignments and build targets, the directory named by $(LIBDIR) is always a subdirectory of $(LIBDIR_BASE). It varies based on library type (optimized, profiled, or debugging) and other platform-specific settings.

Note

Most modules in the Juggler Project define their static library target slightly differently than the above. As of this writing, static libraries are not built on Win32 platforms because there is no easy way for the build system to distinguish between a .LIB file associated with a .DLL file and a static .LIB file. To deal with this, these projects use the following variation on the above:

$(LIBDIR)/$(MY_LIB_STATIC) $(LIBDIR)/$(MY_PROF_LIB_STATIC): $(OBJDIR)/*.$(OBJEXT)
ifneq (@OS_TYPE@, Win32)
	@$(SHELL) $(MKINSTALLDIRS) $(LIBDIR)
	$(AR) $(AR_NAME_FLAG)$@ $(OBJDIR)/*.$(OBJEXT)
	$(RANLIB) $@
	cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./
endif

Next, we show similar targets used to build the dynamic versions:

DYLIB_DEPS=	@DYLIB_DEPS@                                                     (1)
DYLIB_PROF_DEPS=	@DYLIB_PROF_DEPS@                                           (1)

$(LIBDIR)/$(MY_LIB_DYNAMIC): $(OBJDIR)/*.$(OBJEXT)
	@$(SHELL) $(MKINSTALLDIRS) $(LIBDIR)
	$(LD) $(LDOPTS) $(DYLIB_NAME_FLAG) $(OBJDIR)/*.$(OBJEXT) $(DYLIB_DEPS)      (2)
ifeq (@OS_TYPE@, Win32)
	cd $(LIBDIR_BASE) && cp $(LIBDIR)/* .                                       (3)
else
	cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./                 (3)
endif

$(LIBDIR)/$(MY_PROF_LIB_DYNAMIC): $(OBJDIR)/*.$(OBJEXT)
	@$(SHELL) $(MKINSTALLDIRS) $(LIBDIR)
	$(LD) $(LDOPTS) $(DYLIB_NAME_FLAG) $(OBJDIR)/*.$(OBJEXT) $(DYLIB_PROF_DEPS) (4)
ifeq (@OS_TYPE@, Win32)
	cd $(LIBDIR_BASE) && cp $(LIBDIR)/* .                                       (5)
else
	cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./                 (5)
endif
1

First, we define variables to hold any dependencies the dynamic libraries have. The first is for the non-profiled dynamic library, and the second is for the profiled dynamic library.

2

This line uses the link command (defined by the configure script) to create the dynamic library. Note that $(DYLIB_DEPS) is at the end of the line so that dependencies are resolved correctly.

3

These lines deal with making copies of or symlinks to the dynamic library binary in $(LIBDIR_BASE). This is similar to the operation performed for the static library, shown above. In this case, we handle the Win32 versus UNIX cases separately. The .LIB file generated automatically with the .DLL (if the library exports symbols) must be present with the .DLL file. On UNIX, it is sufficient to make a link only to the dynamic library.

4

This line again uses the linker to make the profiled dynamic library. Note that it uses $(DYLIB_PROF_DEPS) to resolve dependencies.

5

Again, we make copies of or symlinks to the profiled dynamic library.

Note

The separation of the targets for profiled and non-profiled versions reflects the lack of good support for profiled libraries. The support for building profiled libraries in the Juggler Project was added in late 2001, and it has not reached a level of solid maturity yet. With time, it is hoped that the the above two targets can be merged into one or that there can be better information sharing between the two.

Installation Extension Targets

Most modules in the Juggler Project install more than just libraries and header files. Data files, sample programs, and other useful bits may be installed as well. Even before something can be installed, an installation directory hierarchy usually needs to be in place. Doozer++ offers features for the basic installation and for extending the installation. In this section, we explain how and why modules in the Juggler Project extend the basic Doozer++ installation capabilities.

First, all modules are required to ensure that the installation hierarchy they need exists before installing. This is accomplished by telling Doozer++ that a 'beforeinstall' target exists as follows:

BEFOREINSTALL=	beforeinstall

beforeinstall:
	<create installation hierarchy>

If the variable $(BEFOREINSTALL) is defined, the Doozer++ installation targets will invoke the target(s) defined in $(BEFOREINSTALL) before installing anything (hence the name). (Using this technique is necessary because GNU make provides no way for makefiles to determine if a target has been defined.) Most modules make use of mtree(1) (or the work-alike Perl script) to create the installation hierarchy in a target called 'hier'. More information about the mtree(1) command is available in Appendix A, Helper Scripts.

Next, modules may need to take extra steps before installing specific builds of the libraries (such as those built by the targets 'dbg', 'prof-dso', etc.). This need is indicated as follows:

PREINSTALL=	pre-install

pre-install:
	<perform steps immediately prior to library installation>

If GNU make were more expressive, the above would probably be done using targets such as 'pre-dbg-install', 'pre-prof-dso-install', and so on.

The complement of pre-installation is post-installation. Availability of custom post-installation targets is specified using the $(POSTINSTALL) variable. However, in Doozer++ 1.5, there is some inconsistency with the use of post-installation targets. They do not correspond exactly to the pre-installation targets, and for that reason, some refactoring is needed. In Doozer++ 1.5, the post-installation targets are executed after a full installation is done rather than after each library version is installed. For that reason, the current $(POSTINSTALL) variable should probably be renamed to $(POSTLIBSINSTALL) or some variant thereof. The current post-installation target always installs the module's header files and then runs any custom post-installation steps. It is at this point that most modules install their data files, sample programs, and other necessary/helpful files.

Finally, the complement of $(BEFOREINSTALL) is executed. It is specified using the variable $(AFTERINSTALL). Considering that most modules use $(POSTINSTALL) for data file installation, there is not much left for $(AFTERINSTALL) to do.

Developer Installation Targets

At long last, we have reached the point where we explain how the developer installation is created. All the modules in the Juggler Project are required to set up a developer installation, though they are free to implement it any way they want. The only restriction is that the modules must not touch or otherwise modify the developer installations of previously built modules. As was stated in the section called “The Developer Installation”, the idea here is to create something that looks like a full installation within the developer's build tree. No special steps should be taken that would cause the developer installation to look or behave any differently than a full installation.

As explained in Chapter 4, The “Global” Build, all modules must define targets that set up and tear down the developer installation. The set-up target is always called 'links', and it can be implemented exactly as follows:

  1 instlinks=	$(topdir)/instlinks
    AFTERBUILD=	afterbuild
    
    afterbuild:
  5 	@$(MAKE) links
    
    links:
    ifdef BUILD_TYPE
    	$(MAKE) links-$(BUILD_TYPE)
 10 else
    	$(MAKE) links-all
    endif
    
    links-all:
 15 	@$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" installworld
    
    links-dbg:
    	@$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-debug
    
 20 links-opt:
    	@$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-optim
    
    links-prof:
    	@$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-profiled
 25 
    clean-links:
    ifndef GLOBAL_BUILD
    	rm -rf $(instlinks)
    endif

The above makes use of a key feature Doozer++ 1.5: the $(BUILD_TYPE) variable. When the target 'afterbuild' is invoked, Doozer++ tells the target what type of build was just performed using this variable. Based on that, the 'links' target can execute the right installation.

Note

The $(instlinks) variable (which must be used as the prefix for the developer installation) will be overridden by the global build so that all modules make their developer installations in the same directory. The setting on line 1 of the above block provides the default value. This is useful when the module is being build standalone (i.e., without using the global build).

When performing that installation, another Doozer++ 1.5 feature is used: the BSD-compatible install(1) script. Prior to Doozer++ 1.5, installations were done under the assumption that install-sh might have to be used if a BSD-compatible install(1) command was not available. The install-sh script is only partially compatible with BSD install(1), and because of limitations in that script, installations could be very slow and complex, even when the install(1) command was available. Doozer++ 1.5 ships with a Perl script called bsd-install.pl that is fully compatible with BSD install(1), and it adds a handy extra feature: symlink creation. When the -l option is passed, and the target OS supports symlinks, symlinks to the source files are created instead of making copies of the files. This is exactly what is needed for developer installations, and the above 'links-*' targets make use of that option through the Doozer++-recognized variable $(EXTRA_INSTALL_ARGS).

Include Build and Clean-Up Functionality from Doozer++

The last thing that must be done by Makefile.inc.in is fairly straight forward. The Doozer++ functionality for building everything and for cleaning up must be imported. Most modules make use of the following code as their last few lines:

_clobber:                                                  (1)
	@$(MAKE) cleandepend
	@$(MAKE) clean-links

_LOCAL_CLOBBER=	1                                          (1)

include $(MKPATH)/dpp.libs.mk                              (2)
include $(MKPATH)/dpp.clean.mk                             (3)

CLEAN_DIRS+=	$(BUILDDIR_BASE) $(LIBDIR_NAME)               (3)
CLOBBER_DIRS+=	$(BUILDDIR_BASE) $(LIBDIR_NAME)             (3)
1

This indicates that a local 'clobber' target is available to be executed after the default Doozer++ 'clobber' target.

2

This includes the Doozer++ file dpp.libs.mk which defines all the functionality described in the preceding sections.

3

These lines include the basic Doozer++ clean-up code and extend the list of directories removed by the 'clean' and 'clobber' targets respectively.

Auxiliary Files

We now turn our attention to axillary files used by the glue makefile and the component makefiles (discussed in the section called “Individual Component Makefiles”). Most modules have only two axillary makefiles in their top-level directory: common.defs.mk.in and make.defs.mk.in. We discuss only those two files here; any extra files are project-specific and could probably be merged into make.defs.mk.in or Makefile.inc.in somehow.

common.defs.mk.in

This is a file that comes with Doozer++ (in the examples directory). It provides a (nearly) comprehensive list of all the variables that may be defined by Autoconf macros in Doozer++. The idea is that a project developer can make use of this file with Doozer++ macros so that he or she does not have to know all the variables that may be substituted by the macros. In addition to the variable settings, there is some code at the bottom of this file for handling different ABIs, though at some point, this may move into a formal Doozer++ makefile stub.

make.defs.mk.in

This file performs three simple tasks:

  1. Include common.defs.mk

  2. Provide project-specific customizations that extend settings in common.defs.mk

  3. Provide the basic name of the library (or libraries) that will be compiled

In reality, these two steps are usually reversed in the actual file, but it makes more sense to discuss them using the above ordering.

Include common.defs.mk

Including common.defs.mk is a relatively straightforward operation. As of this writing, all modules have the following as the last line of their make.defs.mk.in file:

include $(topdir)/common.defs.mk

The variable $(topdir) is available through an assignment made earlier in the file:

topdir=	@topdir@

The use of $(topdir), which always has an absolute path, is required so that this file can be included by any other makefile in the module's source tree.

Provide Project-Specific Customizations

Project-specific customizations are made using the extension mechanisms build into variable assignments in common.defs.mk. In all cases, a variable whose name begins with EXTRA_ is used to extend another variable's default settings. For example, $(CXXFLAGS) is extended using $(EXTRA_CXXFLAGS). Extensions can be performed using the += operator, though such operations should be performed after common.defs.mk is included.

Name the Library (or Libraries)

This is actually part of the project-specific customizations, but it is important to distinguish it so that it is not overlooked. The basic idea here is that one or more variables, one for each C/C++ library, will be defined. These variables will contain the basic name of the library without any extensions or version information. Furthermore, the library name will vary correctly based on the target platform. On UNIX-based systems, that means that the library will have the prefix “lib”, and on Win32 platforms, there will be no prefix. For example:

TWEEK_LIBRARY=	@LIB_PREFIX@tweek

The library name variables will be used by Makefile.inc.in when constructing the list of libraries that Doozer++ must build.

Extension

Unless there are dramatic changes to the way Doozer++ does its job at the time of compilation, the file Makefile.in will remain fairly static. Extensions should almost always go in Makefile.inc.in or make.defs.mk.in, depending on the nature of the extensions. User changes should never be made to common.defs.mk.in. This file provides a reasonably complete set of variables for extending its behavior, and make.defs.mk.in serves as an ideal place to perform overrides to any settings made in common.defs.mk.in. The real problem occurs when it is necessary to copy a modified version of common.defs.mk.in from Doozer++. Merging local changes can be hard unless common.defs.mk.in is always imported onto a CVS vendor branch.



[7] Source code used for testing purposes of for sample applications is handled independently of the main code base, usually by a single makefile.

[8] A better name for this file might be proj.defs.mk since its job is to provide project-specific customizations in addition to the common settings provided through the Doozer++ file common.defs.mk.