Each module in the Juggler Project has a glue makefile in its top-level directory that directs compilation of the module's main code base[7]. Every module's top-level makefile employs recursive calls to GNU make to build its code. The recursion occurs based on the source code directory structure. An iterative loop over each subdirectory is used make the recursive call to GNU make. Behind the scenes, all of this occurs within Doozer++ makefile stubs, but it is directed by variable settings in the glue makefile.
In this section, we will discuss how the glue makefiles work so that they may be extended correctly. We begin by explaining the basic structure employed by most of the glue makefiles found in the Juggler Project source tree. Learning how to extend those makefiles follows from that.
In all but one module in the Juggler Project, the basic
structure of the glue makefile is the same: one makefile that
defines high-level targets, and one included makefile that does all
the work. In this subsection, we explain the way these two files
work to make up the glue that holds a module's build system
together. We begin with the simpler of the two:
Makefile.in.
All modules have to have a Makefile.in
in their top-level source directory. The real makefile is
generated as part of executing the configure script. In most
cases, the contents of this file is roughly the same. The file is
divided into sections, and we will examine each section in
turn.
As with most makefiles, the first thing done is the
definition of a default target, and the glue makefile is no
exception. In most cases, the default target is one that is
useful to developers. In Doozer++ terms, that target is
'debug'.
Definition of a default target unfortunately has special
significance if dpp.subdir.mk is used. In order for a recursive build to
execute based on its definition in Doozer++, the file
dpp.subdir.mk must have the special target
called 'default'. Because it is not possible
with GNU make to test if a target has already been defined, a
workaround must be used. Namely, the variable
$(DEFAULT_SET) must be defined if a makefile
that includes dpp.subdir.mk defines its own
'default' target. Thus, most top-level
Makefile.in files have the following as the
first three lines:
default: debug DEFAULT_SET= 1
It is possible that some refactoring of
dpp.subdir.mk might eliminate the need for
this. As of this writing, it is not considered a critical
change.
Since VR Juggler 1.0, the build system has collected
project-wide variable settings into a single file so that it may
be included by all makefiles in the module's core build system.
In VR Juggler 2.0, the (Autoconf-generated) file is called
make.defs.mk[8]. It is always included using the following
line:
include @topdir@/make.defs.mk
Once make.defs.mk has been included,
local variable settings are performed. These can provide further
customizations on variables set in
make.defs.mk, but usually, the variables
are used only within the scope of the glue makefile. That is,
the variables are needed exclusively for directing the process
of building the module source code.
For example, a decision to build a certain subset of the code base may be made at this level. Alternatively, a decision about how to build the code may be made. In most cases, such a decision is whether to build profiled versions of the library (or libraries).
The most important local variable setting is that of the
list of subdirectories in the module. This list is provided
through the $(SUBDIR) variable. Due to the
structure of the various modules in the Juggler Project, there
would be nothing to do without providing Doozer++ with the list of subdirectories. The way
$(SUBDIR) is assigned its value may vary
depending on the specific structure of a given module.
The main purpose of a project's
Makefile.in is the definition of high-level
build targets. These are the targets that users and developers
will run to compile the module. Such targets include
'world', 'release',
'debug', and 'clobber'.
The first two must be defined manually; the third and fourth are
inherited through the use of Doozer++.
The high-level targets make use of low-level targets that
exist in Doozer++ and, usually, in
Makefile.inc.in. The idea is that a casual
(or experienced) user can look at
Makefile.in alone and understand how to
perform common builds of a given module.
Makefile.inc.in, with all its ugly details,
is hidden—more or less.
In the context of the glue makefile, this particular file is often quite complex. If there is any one place in a module's build system that is hard to understand, it is within this file. However, the whole purpose of having this file is to collect all the complex build details into one place that is “out of the way” in some sense.
Due to the complexity and the wide variation in requirements
between modules, we cannot examine a specific
Makefile.inc.in and still provide a general
explanation. Thus, we will review only the commonalities. When
possible, we will cover important aspects that may not necessarily
be used by all glue makefiles. Before we do that, we will begin
our discussion with an review of the concepts behind
Makefile.inc.in and why it is
important.
As has been stated previously, the fundamental idea behind
the top-level Makefile.inc.in is to hide
complexity. More explicitly, this file should contain targets
that the user will not execute in most circumstances. By putting
such targets in this file, a casual reader need not ever see the
complexity of the glue makefile. He or she should only have to
open Makefile.in (or the generated
Makefile) to get an understanding of how
the build works. Of course, some details are encapsulated within
Doozer++ makefile stubs, so getting a complete
understanding is not necessarily a straightforward process. The
extensive comments at the top of and throughout
Makefile.in (and documents such as this)
are there to fill in the gaps.
While it is very difficult to separate all targets cleanly based on common usage, there are some targets that definitely fit into this file well. Such targets are only called by Doozer++ through a mechanism similar to a C callback. Users of the build system should rarely, if ever, need to execute such targets, especially because Doozer++ may set up a special execution environment for the targets. Callback targets may include the following:
The targets used to compile the final binaries for the module's libraries, including static, dynamic, and profiled versions.
Any targets used as “pre” and “post” steps of larger targets. Larger targets would include those used for building and installing a module.
The responsibility for putting all the pieces together for
the final library names rests on this file. Based on variable
definitions in make.defs.mk (see below),
the final library names must be defined so that Doozer++ knows
what to build. The configure script provides the
platform-specific details such as file extensions so that the
code in Makefile.inc is relatively simple.
For example, assuming the existence of a variable
$(MY_LIBRARY), the following provides all the
information needed to direct the compilation of profiled and
non-profiled libraries, of both the static and the dynamic
variety:
STATICLIB_EXT= @STATICLIB_EXT@ DYNAMICLIB_EXT= @DYNAMICLIB_EXT@ LIBS= $(MY_LIBRARY) STATIC_LIBS= $(LIBS) DYNAMIC_LIBS= $(LIBS)
Using the above variables (and more settings from
make.defs.mk), we can construct the names
of various library versions. These names are then used
internally for the targets that build the actual libraries. The
name construction is performed as follows:
MY_LIB_STATIC= $(MY_LIBRARY).$(STATICLIB_EXT) MY_LIB_DYNAMIC= $(MY_LIBRARY).$(DYNAMICLIB_EXT) MY_PROF_LIB_STATIC= $(MY_LIBRARY)$(PROFLIB_EXT).$(STATICLIB_EXT) MY_PROF_LIB_DYNAMIC= $(MY_LIBRARY)$(PROFLIB_EXT).$(DYNAMICLIB_EXT)
The above hinges on the fact that
$(PROFLIB_EXT) is non-empty. Otherwise, the
targets we will examine in the next section would not be named
correctly. If profiled libraries are not supported by the rest
of the module's build system, do not use the profiled variants
shown above.
Once we know the names of the libraries we will build, we
can define the targets that actually build them. In nearly every
case, these targets are very close to being identical between
modules. Only rarely are there module-specific variations. The
following shows targets for building static profiled and
non-profiled versions of
$(MY_LIBRARY):
$(LIBDIR)/$(MY_LIB_STATIC) $(LIBDIR)/$(MY_PROF_LIB_STATIC): $(OBJDIR)/*.$(OBJEXT) @$(SHELL) $(MKINSTALLDIRS) $(LIBDIR)
$(AR) $(AR_NAME_FLAG)$@ $(OBJDIR)/*.$(OBJEXT)
$(RANLIB) $@
cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./
![]()
![]() | This defines the targets for the profiled and
non-profiled static libraries. Both depend on all the
object files in |
![]() | This ensures that the directory where the library
binary will be build exists. The variable
|
![]() | These lines together build the static library. The
use of |
![]() | This makes a symlink to the newly compiled static
library in |
Most modules in the Juggler Project define their static
library target slightly differently than the above. As of this
writing, static libraries are not built on Win32 platforms
because there is no easy way for the build system to
distinguish between a .LIB file
associated with a .DLL file and a static
.LIB file. To deal with this, these
projects use the following variation on the above:
$(LIBDIR)/$(MY_LIB_STATIC) $(LIBDIR)/$(MY_PROF_LIB_STATIC): $(OBJDIR)/*.$(OBJEXT) ifneq (@OS_TYPE@, Win32) @$(SHELL) $(MKINSTALLDIRS) $(LIBDIR) $(AR) $(AR_NAME_FLAG)$@ $(OBJDIR)/*.$(OBJEXT) $(RANLIB) $@ cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./ endif
Next, we show similar targets used to build the dynamic versions:
DYLIB_DEPS= @DYLIB_DEPS@DYLIB_PROF_DEPS= @DYLIB_PROF_DEPS@
$(LIBDIR)/$(MY_LIB_DYNAMIC): $(OBJDIR)/*.$(OBJEXT) @$(SHELL) $(MKINSTALLDIRS) $(LIBDIR) $(LD) $(LDOPTS) $(DYLIB_NAME_FLAG) $(OBJDIR)/*.$(OBJEXT) $(DYLIB_DEPS)
ifeq (@OS_TYPE@, Win32) cd $(LIBDIR_BASE) && cp $(LIBDIR)/* .
else cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./
endif $(LIBDIR)/$(MY_PROF_LIB_DYNAMIC): $(OBJDIR)/*.$(OBJEXT) @$(SHELL) $(MKINSTALLDIRS) $(LIBDIR) $(LD) $(LDOPTS) $(DYLIB_NAME_FLAG) $(OBJDIR)/*.$(OBJEXT) $(DYLIB_PROF_DEPS)
ifeq (@OS_TYPE@, Win32) cd $(LIBDIR_BASE) && cp $(LIBDIR)/* .
else cd $(LIBDIR_BASE) && $(RM_LN) $(notdir $@) && $(LN_S) $@ ./
endif
![]() | First, we define variables to hold any dependencies the dynamic libraries have. The first is for the non-profiled dynamic library, and the second is for the profiled dynamic library. |
![]() | This line uses the link command (defined by the
configure script) to create the dynamic library. Note that
|
![]() | These lines deal with making copies of or symlinks
to the dynamic library binary in
|
![]() | This line again uses the linker to make the profiled
dynamic library. Note that it uses
|
![]() | Again, we make copies of or symlinks to the profiled dynamic library. |
The separation of the targets for profiled and non-profiled versions reflects the lack of good support for profiled libraries. The support for building profiled libraries in the Juggler Project was added in late 2001, and it has not reached a level of solid maturity yet. With time, it is hoped that the the above two targets can be merged into one or that there can be better information sharing between the two.
Most modules in the Juggler Project install more than just libraries and header files. Data files, sample programs, and other useful bits may be installed as well. Even before something can be installed, an installation directory hierarchy usually needs to be in place. Doozer++ offers features for the basic installation and for extending the installation. In this section, we explain how and why modules in the Juggler Project extend the basic Doozer++ installation capabilities.
First, all modules are required to ensure that the
installation hierarchy they need exists before installing. This
is accomplished by telling Doozer++ that a
'beforeinstall' target exists as
follows:
BEFOREINSTALL= beforeinstall beforeinstall: <create installation hierarchy>
If the variable $(BEFOREINSTALL) is
defined, the Doozer++ installation targets will invoke the
target(s) defined in $(BEFOREINSTALL) before
installing anything (hence the name). (Using this technique is
necessary because GNU make provides no way for makefiles to
determine if a target has been defined.) Most modules make use
of mtree(1) (or the work-alike Perl script)
to create the installation hierarchy in a target called
'hier'. More information about the
mtree(1) command is available in Appendix A, Helper Scripts.
Next, modules may need to take extra steps before
installing specific builds of the libraries (such as those built
by the targets 'dbg',
'prof-dso', etc.). This need is indicated as
follows:
PREINSTALL= pre-install pre-install: <perform steps immediately prior to library installation>
If GNU make were more expressive, the above would probably
be done using targets such as
'pre-dbg-install',
'pre-prof-dso-install', and so on.
The complement of pre-installation is post-installation.
Availability of custom post-installation targets is specified
using the $(POSTINSTALL) variable. However,
in Doozer++ 1.5, there is some inconsistency with the use of
post-installation targets. They do not correspond exactly to the
pre-installation targets, and for that reason, some refactoring
is needed. In Doozer++ 1.5, the post-installation targets are
executed after a full installation is done rather than after
each library version is installed. For that reason, the current
$(POSTINSTALL) variable should probably be
renamed to $(POSTLIBSINSTALL) or some variant
thereof. The current post-installation target always installs
the module's header files and then runs any custom
post-installation steps. It is at this point that most modules
install their data files, sample programs, and other
necessary/helpful files.
Finally, the complement of
$(BEFOREINSTALL) is executed. It is specified
using the variable $(AFTERINSTALL).
Considering that most modules use
$(POSTINSTALL) for data file installation,
there is not much left for $(AFTERINSTALL) to
do.
At long last, we have reached the point where we explain how the developer installation is created. All the modules in the Juggler Project are required to set up a developer installation, though they are free to implement it any way they want. The only restriction is that the modules must not touch or otherwise modify the developer installations of previously built modules. As was stated in the section called “The Developer Installation”, the idea here is to create something that looks like a full installation within the developer's build tree. No special steps should be taken that would cause the developer installation to look or behave any differently than a full installation.
As explained in Chapter 4, The “Global” Build,
all modules must define targets that set up and tear down the
developer installation. The set-up target is always called
'links', and it can be implemented exactly as
follows:
1 instlinks= $(topdir)/instlinks AFTERBUILD= afterbuild afterbuild: 5 @$(MAKE) links links: ifdef BUILD_TYPE $(MAKE) links-$(BUILD_TYPE) 10 else $(MAKE) links-all endif links-all: 15 @$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" installworld links-dbg: @$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-debug 20 links-opt: @$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-optim links-prof: @$(MAKE) EXTRA_INSTALL_ARGS=-l prefix="$(instlinks)" install-profiled 25 clean-links: ifndef GLOBAL_BUILD rm -rf $(instlinks) endif
The above makes use of a key feature Doozer++ 1.5: the $(BUILD_TYPE)
variable. When the target 'afterbuild' is
invoked, Doozer++ tells the target what type of build was just
performed using this variable. Based on that, the
'links' target can execute the right
installation.
The $(instlinks) variable (which must
be used as the prefix for the developer installation) will be
overridden by the global build so that all modules make their
developer installations in the same directory. The setting on
line 1 of the above block provides the default value. This is
useful when the module is being build standalone (i.e.,
without using the global build).
When performing that installation, another Doozer++ 1.5
feature is used: the BSD-compatible
install(1) script. Prior to Doozer++ 1.5,
installations were done under the assumption that
install-sh might have to be used if a
BSD-compatible install(1) command was not
available. The install-sh script is only
partially compatible with BSD install(1), and
because of limitations in that script, installations could be
very slow and complex, even when the
install(1) command was available. Doozer++
1.5 ships with a Perl script called
bsd-install.pl that is fully compatible with
BSD install(1), and it adds a handy extra
feature: symlink creation. When the -l option
is passed, and the target OS supports symlinks, symlinks to the
source files are created instead of making copies of the files.
This is exactly what is needed for developer installations, and
the above 'links-*' targets make use of that
option through the Doozer++-recognized variable
$(EXTRA_INSTALL_ARGS).
The last thing that must be done by Makefile.inc.in is fairly straight forward. The Doozer++ functionality for building everything and for cleaning up must be imported. Most modules make use of the following code as their last few lines:
_clobber:@$(MAKE) cleandepend @$(MAKE) clean-links _LOCAL_CLOBBER= 1
include $(MKPATH)/dpp.libs.mk
include $(MKPATH)/dpp.clean.mk
CLEAN_DIRS+= $(BUILDDIR_BASE) $(LIBDIR_NAME)
CLOBBER_DIRS+= $(BUILDDIR_BASE) $(LIBDIR_NAME)
![]()
![]() | This indicates that a local
' |
![]() | This includes the Doozer++ file
|
![]() | These lines include the basic Doozer++ clean-up code
and extend the list of directories removed by the
' |
We now turn our attention to axillary files used by the glue
makefile and the component makefiles (discussed in the section called “Individual Component Makefiles”). Most modules have only two
axillary makefiles in their top-level directory:
common.defs.mk.in and
make.defs.mk.in. We discuss only those two
files here; any extra files are project-specific and could probably
be merged into make.defs.mk.in or
Makefile.inc.in somehow.
This is a file that comes with Doozer++ (in the examples directory).
It provides a (nearly) comprehensive list of all the variables
that may be defined by Autoconf macros in Doozer++. The idea is
that a project developer can make use of this file with Doozer++
macros so that he or she does not have to know all the variables
that may be substituted by the macros. In addition to the variable
settings, there is some code at the bottom of this file for
handling different ABIs, though at some point, this may move into
a formal Doozer++ makefile stub.
This file performs three simple tasks:
Include common.defs.mk
Provide project-specific customizations that extend
settings in common.defs.mk
Provide the basic name of the library (or libraries) that will be compiled
In reality, these two steps are usually reversed in the actual file, but it makes more sense to discuss them using the above ordering.
Including common.defs.mk is a relatively straightforward
operation. As of this writing, all modules have the following as
the last line of their make.defs.mk.in
file:
include $(topdir)/common.defs.mk
The variable $(topdir) is available
through an assignment made earlier in the file:
topdir= @topdir@
The use of $(topdir), which always has
an absolute path, is required so that this file can be included
by any other makefile in the module's source tree.
Project-specific customizations are made using the
extension mechanisms build into variable assignments in
common.defs.mk. In all cases, a variable whose name begins with
EXTRA_ is used to extend another variable's
default settings. For example, $(CXXFLAGS) is
extended using $(EXTRA_CXXFLAGS). Extensions
can be performed using the += operator, though such operations
should be performed after common.defs.mk is
included.
This is actually part of the project-specific customizations, but it is important to distinguish it so that it is not overlooked. The basic idea here is that one or more variables, one for each C/C++ library, will be defined. These variables will contain the basic name of the library without any extensions or version information. Furthermore, the library name will vary correctly based on the target platform. On UNIX-based systems, that means that the library will have the prefix “lib”, and on Win32 platforms, there will be no prefix. For example:
TWEEK_LIBRARY= @LIB_PREFIX@tweek
The library name variables will be used by
Makefile.inc.in when constructing the list
of libraries that Doozer++ must build.
Unless there are dramatic changes to the way
Doozer++ does its job at the time of compilation, the file
Makefile.in will remain fairly static.
Extensions should almost always go in
Makefile.inc.in or
make.defs.mk.in, depending on the nature of the
extensions. User changes should never be made
to common.defs.mk.in. This file provides a reasonably complete set of
variables for extending its behavior, and
make.defs.mk.in serves as an ideal place to
perform overrides to any settings made in
common.defs.mk.in. The real problem occurs when
it is necessary to copy a modified version of
common.defs.mk.in from Doozer++. Merging local
changes can be hard unless common.defs.mk.in is
always imported onto a CVS vendor branch.
[7] Source code used for testing purposes of for sample applications is handled independently of the main code base, usually by a single makefile.
[8] A better name for this file might be
proj.defs.mk since its job is to
provide project-specific customizations in addition to the
common settings provided through the Doozer++ file
common.defs.mk.