Module Build Specification

The module build specification is designed around Phil Lord's build specification at BuildSpec and uses the AntMerge facility

Abstract

The following defines a standard method of defining build scripts to avoid code replication

Requirements

• myGrid is being developed on different sites, by different people
• Although myGrid should exist as a single entity, it is also clear that different parts of the project will be useful to other groups.
• Other groups should be able to use parts of myGrid without getting the whole thing

From this it appears therefore that myGrid should consist of a number of modules, which are largely independent, or where dependencies are clearly enumerated.

• In order to have a unified build however, the different modules need to have a build system which can be unified, as well as operating "stand alone"
• Many of the modules will share external dependencies. These should not be duplicated between them, as this will lead to both download bloat, and also, however careful we are with version numbers, external dependency version conflict
• "Stand-alone" build should also allow developers to build just the subsystems within myGrid that they are actively developing, to reduce the time spent on the code-compile-test cycle. This will also reduce the amount of space they need to use, if they use multiple work file copies of their modules, as some people like to
• The build system should also be the common point of entry for the unit test system.
• It may also be appropriate to use it as a common point of entry for the unit test system.
• Likewise deployment and installation. And possibly even launch of the system.

Overall Design

The best way to address these issues is to define a single unified directory structure which all of the projects should follow. There will be a globally defined build file this will have preset attributes. Each project should also design a single build file with standard entry points (targets) that they wish to ammend or add to the global build file. These individual build files can then be directed "from above" by the top level build file to perform a unified build. Here I call the individual build files "module build files", and the top level build file "the main build file".

This addresses a number of issues, the subsystems each have a local build file for unit testing and for allowing distribution of each component. The build file, both in module and in the main file, become unified in design and structure and allows for more familiarity. It wis also remove the duplication that is currently occuring within the build files. This allows for an easier updating process and management

The systems uses a self building module (AntMerge) to build each component, the AntMerge module calls a default build file and a local build file to create a new build file which is used to generate the code.

The AntMerge files need to be installed on build machine, these are available from CVS at ViewCVS

The AntMerge, once built, can be used to generate and build the required modules

Directory Structure

The basic directory structure that I am proposing for myGrid is shown in the following table. I have somewhat mixed up end directory structure and directory structure in the CVS. Those directories marked with † should be present in the CVS, those without will be generated directories.

Directory or Filename	Usage
ext/	All external dependencies should be found here
Licenses/ †	Location of all the license documents should be here. There may be quite a few of these, as external dependencies will need them
doc/ †	Top Level documentation which applies to all modules. Would probably include stuff like this document, general help documentation. These should be in text format or pdf only. Also probably an "AUTHORS" document.
javadoc/	Unified Javadoc repository, created from all of the source.
build/	Used for temporary files during the build. What this includes will depend on the "build mode".
build/test-results/	JUnit test results should go in here.
build/classes/	Generated class files.
lib/	For jar files. What this includes will depend on the "build mode".
samples/ †	Examples which run and do cute things.
src/ †	All source code should be placed in here. By source we mean, anything that is need to generate the functional software. And not anything else. This would exclude documentation, and also "source" code that is actually built from something else.
bootbuild.xml †	This file will be used to bootstrap the build process do not edit.
version.txt †	This file will initially be set to 0.0.1 This is used by anthill to track the version numbers of the automated build do not edit. Format is major.minor.build

Standard Targets

There will be a number of standard build targets that each module build file will need.

Target	Depends	Usage
build	compile	Build everything, compile and documentation
compile	none	Compile everything that needs to be compiled for functional code. Not documentation. Compile means generate .class files, and not .jar files.
doc	none	Generate any documentation which needs generating.
javadoc	none	Generate javadoc.
jar	compile	Package everything up into .jars in the lib/ directory
dist	jar	Generate all of the distribution files, both zip and tar.gz
test	compile	Run complete test sets on files in build/ directory.
clean	none	Delete everything that is generated. This should return the system to what appears out of the CVS.
deploy	compile	Create the web apps structure within the build
install	deploy	Install webapps to tomcat, using the admin upload client. There are also reload, remove, and list targets using the ant tasks shipped with tomcat
gather	prepare	Gathers all of the required jars to ext/

A couple of these target could probably do with some further subdivision, in particular "clean", which probably needs a separate "clean_classes", which I've found very useful during development. Likewise dist, could depend on dist-zip, and dist-tgz.

Using the module build

1. Download the AntMerge files from ViewCVS
2. Build AntMerge system using ant (see AntMerge for full instructions)
   There is a bootbuild.xml created in each module, this is used by the automated AntHill build system.
   You will need to download the zip file from AntMerge to 'bootstrap' the system for the first time
3. Each project needs to edit the module-in.xml to replace the default settings. An example is in ViewCVS
4. You may need to generate the default.xml in etc to do this run the command
   antmerge --mergefile="antmerge.xml" --mainfile="default-in" --output="default.xml"
   in the etc directory
5. Once edited with the module requirements the follwing command will create the build file
   antmerge --mergefile="module-in.xml" --mainfile="etc/default" --output="build.xml"

The build.xml does not need to be checked into CVS as the file can be generated at buildtime each time, this means that the build file will be current and upto date

Automated top level build

See ProjectBuild

-- RichCawley - 26 Nov 2002