---------------------------------
Globus Toolkit 2.x CVS Procedures
---------------------------------

The Globus 2.0 effort marked the first widespread use of CVS branches
for the globus team.  Many (later painful) mistakes were made in the
management of the first branches.  This document stands to provide a
procedure to follow for releases and all CVS operations that will
hopefully prevent pontential problems.

The commands used for the common cvs operations are at the end of this
document.  Throughout this document, we will refer to the head of the
CVS repository as the 'trunk' (aka 'the main branch').

The trunk is no longer to be considered a dumping ground.  The trunk is
intended to be the place that releases are cut from.  This implies that
whatever is in the trunk should be (somewhat) guaranteed to work.  This
requires everything that gets committed to the trunk be put through a
full set of tests.  These tests would at least include a successful
build and execution on two different architectures (probably linux and
solaris) using the tools we normally build with (currently cvs-build
-everything).  See Globus Project 2.x Testing Procedures for more
details.

All active work is to be done on branches.  The majority (supposed to be
all) of our work is related to a specific campaign.  So, a branch should
be created for every campaign.  See below for info on creating a branch. 
These branches are intended to be used only by the small team
participating in a particular campaign.  The branch itself should be
created over all modules.  Any work on a branch should not be merged
with the trunk until it has gone through an  'alpha' phase and is
considered 'beta' (release candidate) material.

Small bug fixes may not need a branch.  Please use these guidelines and
common sense with regard to committing bug fixes to the trunk.  You
should follow the same testing as everything else.

It will be necessary to perform merges (see below) periodically
thoughout the life of a branch in order to shift the branch point and
maintain compatibility with the rest of the trunk.  Typical scenarios
include:

-- The transition from alpha to beta
-- A recent commit to the trunk is incompatible with your branch
   (like a change to globus_common)
-- The branch is getting 'old' (more than two months since creation)

In any of these situations, it will be necessary to create a new branch
off of the trunk and merge the previous branch into that one.  The old
branch should be discontinued immediately after this shift.  Note that
the 'beta' branch is  intended to be a very short-lived branch.  It is
there only to be sure that your code works properly with the latest
changes in the trunk.

When a campaign has been completed and is ready for beta release, the
'beta' branch can be merged into the trunk.  Before this is done, be
sure to update the package data files.  Specifically, updating the
Major, Minor, Age as required and the Version_Stability Release="beta".
Once again, the campaign should be considered stable and complete
(passes all testing procedures in Globus Project 2.x Testing Procedures)
before merging into the trunk.
------------------
Release Procedures
------------------

We propose following the Linux kernel model of making two collections of
packages available simultaneously, with two different release version
numbers.  The release version numbers would take the form of X.y.z, or
major, minor, and incremental.  Right now, our major number would be 2,
and our minor number is 0, making for version 2.0 of the Globus Toolkit.

The even value of y specifies that this collection of packages is part
of a stable release.  The even line will progress only with incremental
releases as bug fixes are applied to the stable release.  Odd values of
y mean that the collection of packages is part of an experimental
release.  The odd line will progress as new campaigns (new
functionality) are merged into the trunk (signifying their completion).

This even/odd version number designation has been used very successfully
by the Linux kernel.

In order to discuss how we would make it work, here is a diagram of how
the FTP tree might look after this proposal had been in place for a
while.  All version numbers are hypothetical!

Proposed FTP structure:

Toolkit/2.0/2.0.13
	Bundles
	README (contains CVS logs about updates from last release)
	packages/  (contains the packages that make up these bundles)
        updates/   (update packages which have been released since
        		2.0.13 was finalized.  There is one README per
        		package, describing what changed)

Toolkit/2.0/2.0.14
	Bundles
	README (This is basically the concatenation of the READMEs
		from updates to the last incremental version number)
	packages/  (the packages for these bundles.  Basically the
			packages from the updates directory of 2.0.13,
			along with the other packages from the 2.0.13
			packages directory.  An incremental release is
			equivalent to the previous release + updates.)
	updates/   (Starts empty)

Toolkit/2.0/all-latest
	This is the collection of the most recent version of every
	package we've got.  People can look in here if they're
	interested in getting the latest version of package Foo, but
	don't want to hunt down what release it came out in

Toolkit/2.1/2.1.30
	Bundles
	README
	packages/
	
	The idea here is to leverage the automation we will put in
	place, and follow the same structure.  These bundles should be
	guaranteed to build at least on our machines.  :-)

Toolkit/2.1/2.1.31
	Bundles
	README
	packages/

	As campaign branches get merged back in, the development
	version will increment.


Here are the processes and procedures which will make this work. 

To start with, we just have the CVS trunk.  When it is determined that a
stable release is due, we create a branch for it.  Right now that
corresponds to the Globus 2.0 branch (which happens to be named
globus-beta-branch, but is really the 2.0 branch now).

In the new scheme, the branch for Globus X.y would be named stable_x_y. 
The first thing done on this new branch is to set all the package's
Version_Stability Release="production".  Release bundles/packages will
be made from the stable branch by an automated script.  Any time a bug
is found and fixed, it will be applied to the stable branch, and
whenever an updated package is created (this may or may not be
one-to-one with bugs fixed), it will be placed in the updates/
subdirectory.

At some point, enough of the packages in the current stable branch will
have updates, and we will decide to provide an incremental, toolkit-wide
release containing all of the updates thus far.  The stable_x_y will
then be tagged with stable_x_y_(z+1) and a new release cut from that
tag.  Each succesive release from the stable branch will increment the
value of z in the tag.  We refer to such releases as "incremental
releases."

In parallel to that, the trunk continues with the current experimental
release.  Individual campaigns may complete and merge their changes into
the trunk.  There may also be bug fixes to the trunk (all bugs fixed in
the stable branch should get fixed in the trunk as well, if still
applicable).

On a weekly basis (monday?) a new experimental release is cut if any
change has occurred to the trunk.  The trunk is first tagged with
experimental_x_y_z and the release cut from that tag (using the same
tools as before).

At some point, it will be decided that it is time for a new stable
release, encompassing the features added in the experimental releases. 
To do so, a new stable branch is created, the same way we made the
Globus 2.0 branch.  This is the operation that increments the value of
y.  I.e., for a while we will be incrementing 2.0.z and 2.1.q.  At some
point, the last 2.1.q will become 2.2.0, on its own branch.

When this happens, a new experimental release (2.3.0 in this example)
will be created (as a tag on the trunk).

We will only be supporting one stable release at a time, so the creation
of a new stable release (with an incremented y value) means that the
previous stable and experimental releases will no longer be updated,
although they will remain available in FTP archives.  The new
experimental release set up at this time will most likely be empty until
a campaign is merged in. 

We will not be offering the same level of support for experimental
releases as for stable releases.  Although we will encourage submission
of bug reports for the experimental releases, there won't be any
guarantee of turnaround time on the bug fixes (although, as always, it
will be faster if good debugging information/patches are supplied). 
Simply stated, new development may take priority over specific bug fixes
in the experimental line.  Conversely, since there is no new development
in the stable line, bug fixes have a much higher priority, and should be
attended to promptly.

We should have a tool that will create a release directory structure
quickly and easily (ie, gbuild -release=stable -version=2.0.14... 
gbuild -release=experimental -version=2.1.397) to keep the cost of
maintenance low.

---------------------
Common CVS Operations
---------------------

creating a branch
-----------------

In general, all branches should be from the trunk.  It is possible, to
branch from a branch, but, once the above procedures are in place, it
should never be necessary.

First we need to mark the branch point.  This allows the set of changes
on only this branch to be applied in merges.  (This may seem like the
default operation, but there are some gotchas)

    cvs rtag <branch_point> all

Then, create the actual branch:

    cvs rtag -b -r <branch_point> <branch_name> all

Once the branch is created, be sure to add this branch to dirt's
branch.def file.  See http://www.mcs.anl.gov/~link/dirt.txt for more
info.

checking out a branch
---------------------

cvs co -r <branch> all

merging
-------

These instructions apply to merging branches into the trunk or into
another branch (aka, shifting a branch)

Checkout the destination of the merge.

mkdir cvs_dest_branch_wd
cd cvs_dest_branch_wd
if dest is trunk
    cvs co all
else
    cvs co -r <dest_branch_name> all

cvs -q up -d -j <source_branch_point> -j <source_branch_name>

(the -j <source_branch_point> may not be necessary if the branch was made off
trunk.  I can not remember all of the issues that make it necessary.  If you
get a message like the following:

U some_file
cvs update: file some_file exists, but has been added in revision some_branch

rerun the merge for that file only with the following command:

cvs -q up -j <some_branch> some_file
)

To add a little confidence to the success of the merge, you should walk
through the differences between what you're about to commit and the branch
you merged in.

cvs -q diff -u -b -N -r <source_branch_name> >differences

Now resolve conflicts, test and commit.

Note: -kk can be used in the above merge command to prevent conflicts
from occurring because of differences in CVS keyword expansions.  (ie,
$Revision: 1.9$)  There is a problem with using it when new files end up
being merged in.  If those new files are committed as is, they will
perminently have the -kk attributed to them, and CVS keywords will never
be expanded.

So, either don't use -kk if new files are added or use the following
procedure to remove the -kk from these new files after merging them. 
From the top of a recently merged checkout:

find . -name Entries -print -exec grep '\/0\/' {} \;

in every Entries file with an entry similar to the following

/<filename>/0/dummy timestamp/-kk/

simply edit it to read

/<filename>/0/dummy timestamp//

or use the following script

for file in `find . -name Entries` ; do
    mv $file $file.orig
    (cat $file.orig | sed 's/-kk//' > $file)
done

Note, that this problem does not affect files that already exist in
repository, only new files as the result of a merge.

rollbacks
---------

In general, a rollback is accomplished by a reverse merge.  It can be
used to remove any range of changes.

To rollback a single file:

cvs up -j <version after bad commit> -j <version before bad commit>
<file>

dirt
----

if you made changes to a dirt monitored package while the branch name
was not defined in branch.def, you'll need to refresh the dirt info for
the branch

for module in "<modules that need dirt adjusting>" ; do
    files=`find $module -name dirt.sh`
    cvs commit -f $files
done