Atlas Software Installation at PDSF

The ATLAS area on pdsf has the following structure

/auto/atlas/mirror - mirrored libraries 

In this area there are directories that preserve the CERN ATLAS structure:

/auto/atlas/mirror/software  (srt mirror).
and
/auto/atlas/mirror/offline/external (Gaudi, Geant4, tarred and pulled over etc)

/auto/atlas/sw/lhcxx - LHC++ software

/auto/atlas/sw/lhcb  - Gaudi
/auto/atlas/sw/lhcb/cvs - Gaudi repository

/auto/atlas/local - local cvs repository and libraries built locally

Similar to mirror it has subdirectories:
/auto/atlas/local/software - for the repository and ATLAS software
/auto/atlas/local/offfline/external - for locally built external packages


/auto/atlas/tools - setup scripts

Setup Scripts
==============
I added a script setup-atlas-new.csh to /auto/atlas/tools.
It is supposed to be executed after setup-general.csh.
Depending on the first argument (either local or mirror) it
will setup one of the two environments available currently on 
PDSF.

to set up environment that accesses mirrored libraries:
source /auto/atlas/tools/setup-general.csh
source /auto/atlas/tools/setup-atlas-new.csh mirror

to access locally build libraries:
source /auto/atlas/tools/setup-general.csh
source /auto/atlas/tools/setup-atlas-new.csh local

Mirroring of the ATLAS release

To prepare mirroring of the ATLAS release one needs:

/afs/cern.ch/atlas/software/bin/srt mirror -i -r `pwd` -d

This mirrors the srt itself and prepares the dist directory to hold releases. The original srt (from afs) needs to be used to do the mirroring - mirrored srt knows that it is a mirror and mirroring with a mirror is not allowed so srt mirror won't work for that. There is no need to have any environmental variables to set up for performing the task listed above.

To mirror a release, one still has to use the afs version of: /afs/cern.ch/atlas/software/bin/srt, but already local srt libraries could be used. For that one needs to define $SRT_HOME

setenv SRT_HOME /auto/atlas/mirror/software/lib/srt/0.3.2

(0.3.2) is the current version of srt in use.

Then a release is mirrored:

/afs/cern.ch/atlas/software/bin/srt mirror -c -r `pwd` releaseNumber architecture

in our case to mirror 1.1.4 for linux we do:
/afs/cern.ch/atlas/software/bin/srt mirror -c -r `pwd` 1.1.4 i586-pc-linux-gnu/egcs-1.1

currently we have the following releases mirrored:
0.0.41
1.1.1
1.1.2
1.1.3
1.1.4	414 MB
1.1.5   439 MB 
1.1.6

We need to establish a policy on how to drop releases.


Local copy of LHC++
===================

We currently have version 2.0.1 and 3.2.0 (388Mbytes) available under
/auto/atlas/sw/lhcxx. 
Installation is straightforward if you follow instructions:
http://wwwinfo.cern.ch/asd/lhc++/installation.html

Laurent VACAVANT from EP/HC  and
Iwona Sakrejda From NERSC (ISakrejda@lbl.gov)

registered for lbl to use nag_c and the objectivity.

Following a web page:
http://wwwinfo.cern.ch/asd/lhc++/installation.html
I performed instalation from:
/afs/cern.ch/sw/lhcxx/export 

and our local copy is at:
/auto/atlas/sw/lhcxx

But the version I installed has no includes. Here is a message I got
from CERN library support:
*******
 We are not allowed to redristribute the NAG C library include files
 outside Cern.
 That's because use of header files requires a NAG C developer license:
 Cern has a site license but we can't assume external sites as one as
 well.
 Our group is currently negotiating with NAG to provide such licenses
 to external institutes, but this will taks some time.

 Best regards
 Lysiane Besson/Program Library
*******

So for now I just added includes by hand to
/auto/atlas/sw/lhcxx/specific/redhat61/Nag_C/5.0/includes


Mirroring of the ATLAS cvs repository and Gaudi cvs repository:
===============================================================

Mirroring of both the ATLAS and Gaudi cvs repositories
 with the CVSup/d software has been set up.

The server runs at cern on atlas006.cern.ch.
Relevant software exists in ~sakrejda/public/cvsup.16.1
A cron job was set up to monitor the server and restart it if
the server is killed for some reason (computer reboot etc).
A script for the cron job is at:
~sakrejda/public/cvsup.16.1/watch.sh

A local version of the Atlas repository resides at:
/auto/atlas/local/software/cvs
It needs 351MB. 

There is a file:
/auto/atlas/local/software/cvs/CVSROOT/avail
It contains 1 line:
unavail
This way the repository is read-only.

A local version of Gaudi repository resides at:
/auto/atlas/sw/lhcb/cvs
It needs 120MB.

There is a file:
/auto/atlas/sw/lhcb/cvs/CVSROOT/avail
It contains 1 line:
unavail
This way the repository is read-only.

Currently bo repositories are updated every 4 hours by a cron job running on pdsflx003

pdsflx003 66% crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.17005 installed on Thu Nov  9 13:32:44 2000)
# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)
SHELL=/bin/csh
20  0-23/4 * * * /auto/atlas/local/acvsup
50  0-23/4 * * * /auto/atlas/sw/lhcb/acvsup

Log of the cron job goes to:
/auto/atlas/local/cvsuplog for the ATLAS repository and /auto/atlas/sw/lhcb/cvsuplog for Gaudi.

Building ATLAS release based on a local repository:

I mirrored srt to have it in the /auto/atlas/local/software area. There is no procedure published to install a "primary" (not mirrored) srt.

reminder, to mirror srt:

 /afs/cern.ch/atlas/software/bin/srt mirror -i -r `pwd` d

The next step is to verify (and correct if necessary) the loaction of install. srt assumes that install is in /usr/bin/local. If you have it installed in another location you can correct for it by editing:

/auto/atlas/mirror/software/lib/srt/0.3.2/fragments/i586-pc-linux/egcs-1.0/machine

Then you need to retrieve a tagged version of a release from the repository. First set the CVSROOT variable to point to your repository:

setenv CVSROOT /auto/atlas/local/software/cvs

and retrieve a revision you need:

srt [srt options] mirror -n -r ROOT DIST

I created:

/auto/atlas/local/software/dist

then in:

/auto/atlas/local/software

I ran:

/afs/cern.ch/atlas/software/bin/srt mirror -n -r `pwd` 1.3.0

If you "fixed" your mirror of srt, so it does not know it's a mirror you need to go to

/auto/atlas/local/software/dist

and:

/auto/atlas/local/software/bin/srt new -p 1.3.0 offline-01-03-00

Currently location of cern library is hardwired in some of the GNUmakefile.in files. I use the following line to identify them:

find 1.3.0/ -name GNUmakefile.in -exec grep \/cern\/ {} \; -print

In 1.3.3 the following files had to be modified:

1.3.0/atrig/CTrig/GNUmakefile.in
1.3.0/atrig/LVL1/t1muon/GNUmakefile.in
1.3.0/atrig/LVL2/t2muon/GNUmakefile.in
1.3.0/atrig/LVL2/t2sct/GNUmakefile.in
1.3.0/atrig/TrigSim/AtrigDump/GNUmakefile.in
1.3.0/atrig/TrigSim/AtrigUtils/GNUmakefile.in
1.3.0/graphics/Atlantis/GNUmakefile.in

Make sure that cernlib and srt are in your PATH when you log in, as the build dispatches processes to other nodes and they expect it. So you need the following lines in your .cshrc file:

setenv SRT_HOME /auto/atlas/local/software/lib/srt/0.3.2
setenv SRT_ARCH i586-pc-linux-gnu
setenv SRT_CONF egcs-1.1
setenv SRT_TARGET i586-pc-linux-gnu/egcs-1.1
setenv SRT_SITE /auto/atlas/local/software/srt-local-config
setenv PATH ${PATH}:/auto/atlas/local/software/bin
setenv CERN /usr/local/cern
setenv CERN_LEVEL 2000
setenv PATH ${PATH}:${CERN}/${CERN_LEVEL}/bin

Then in :

/auto/atlas/local/software/

srt setup

cd dist/1.3.0

I edited boot.mk replacing list of hosts by my local host. Then following Maya's instructions I ran:

make -f boot.mk release
make -f boot.mk release-post-clean

Then to freeze the relaease:

cd /auto/atlas/local/software/
/afs/cern.ch/atlas/software/bin/srt mirror -f -r `pwd` 1.3.0

or with the "fixed" version of srt

cd /auto/atlas/local/software/dist/1.3.0
srt freeze

Have not been successful for a while.....

Now we have the following releases build locally:
1.1.3
1.1.5
1.1.6
1.2.0
1.2.1


Mirroring of the Gaudi repository:
==================================

Mirroring of the Gaudi cvs repository with the CVSup/d software has been set up.
A local version of the repository resides at:

/auto/atlas/sw/lhcb/cvs

It requires 97MB of disk space.Currently it is updated every 3 hours by a cron job running on pdsflx02.

Log of the cron job goes into:
/auto/atlas/sw/lhcb/cvsuplog



Installing Gaudi from CERN:
===========================


pdsflx02 74% cp /afs/cern.ch/atlas/offline/external/GaudiRoot/Gaudi.tar.gz .
pdsflx02 75% cp /afs/cern.ch/atlas/offline/external/GaudiRoot/packages.tar.gz .
pdsflx02 76% cp /afs/cern.ch/atlas/offline/external/GaudiRoot/tools.tar.gz .

unzip and untar


Installing local version of CMT
================================

Web page for distributions (v1r7):
http://www.lal.in2p3.fr/SI/CMT/v1r7/CMTDownload.html

Then select both
http://www.lal.in2p3.fr/SI/CMT/v1r7/CMTv1r7i386_linux22.tar.Z
and
http://www.lal.in2p3.fr/SI/CMT/v1r7/CMTv1r7.tar.Z

For ATLAS at PDSF CMT was installed at:
/auto/atlas/sw/contrib/CMT

Save it and untar

(tar -xvZf)

then go to CMT/v1r7/mgr and install:
./INSTALL
gamke


And it's ready for use....

Gaudi (v6r2) Installation


Untar (tar -xvzf) in 
/auto/atlas/local/offline/external/Gaudi 

a release  that D. Quarrie posted at CERN in the ATLAS afs area:
/afs/cern.ch/atlas/project/Gaudi/install/0.6.2.tar.gz 

Make sure you have CMT v1r7.

Set the following environmental variables:
setenv CERN /usr/local/cern
setenv CERN_LEVEL 2000
setenv CMTBASE /auto/atlas/sw/contrib/CMT
setenv CMTVERS v1r7
setenv CMTCONFIG i386_linux22
source $CMTBASE/$CMTVERS/mgr/setup.csh
setenv PATH   `/auto/atlas/tools/addpath.pl PATH ${CMTBASE}/mgr`
setenv CMTPATH /auto/atlas/local/offline/external/Gaudi/0.6.2
setenv CMTSITE LBNL
setenv LHCBCERNINC ${CERN}/${CERN_LEVEL}/include/
setenv SITEROOT /auto/atlas/
setenv ROOTSYS /auto/atlas/sw/ROOT/v2.25
setenv LD_LIBRARY_PATH /auto/atlas/sw/ROOT/v2.25/i386_linux22/root/lib
setenv PATH ${PATH}:${CERN}/${CERN_LEVEL}/bin


Xerces had to be installed in 
$EXT_DIR
(tar -xvzf  ~quarrie/Atlas/xerces-c.tar.gz).
This is the same place xerces is installed at CERN.

in EXTLIB/v2r3p1/cmt/ requirements were modified to account
for the PDSF configuration:
at the end of PYTHON_DIR
    LBNL   "/usr" 
  
at the end of EXT_DIR:
    LBNL   "${SITEROOT}/local/offline/external"

at the end of ROOT_DIR:
    LBNL   "${SWROOT}/ROOT"

Then do the following:

     cd Athena/v1r8p1/cmt
     cmt broadcast rm -f '*.make'
     cmt broadcast cmt config
     cmt broadcast make clean
     cmt broadcast make 


AND IT WAS MADE!!!!!!!!

Additional packages from offline/external
=========================================

The following packages were tarred and installed in  
/auto/atlas/mirror/offline/external
tar -cvzf ~/public/pythia_6.129.tar.z --exclude pythia_6.129/installed/sparc-sun-solaris2.5.1 pythia_6.129
  
tar -cvzf ~/public/isajet_7.51.tar.z -X ~/exclude.txt isajet_7.51
 
tar -cvzf ~/public/4.09.tar.z --exclude 4.09/installed/sparc-sun-solaris2.5.1 4.09

Building Athena at based on /auto/atlas/mirror and my local cvs
================================================================

On pdsf system install is in:

/usr/bin/install

so I had to edit:

/auto/atlas/mirror/software/lib/srt/0.3.2/fragments/i586-pc-linux/egcs-1.0/machine 

I replaced:
# Installation
INSTALL         = /usr/local/bin/gnu/install -c

with:

# Installation
INSTALL         = /usr/bin/install -c

Then I followed Dave's instructions:

srt new src 1.1.3
cd src
cvs co -d ZebraTDRCnv -r MayTutorial_01_01_00 offline/Event/ZebraTDRCnv
cvs co -d GaudiSequencer -r Gaudi5-00-00-00 offline/Control/GaudiSequencer
cvs co -d GaudiInterface -r Gaudi5-00-00-00 offline/External/GaudiInterface
cvs co -d ControlExamples -r Gaudi5-00-00-00 offline/Control/ControlExamples

cd ..
mkdir build
mkdir run
cd build
../src/configure
make clean
make install

results from these 3 commands are in:
/auto/atlas/sakrejda/mathena/build
clean.doc  configure.doc  install.doc

The third one contains references to the /afs area although I have my 
local Gaudi - needs to be chcked.
also the following variables are being defined:
ATLAS_EXTERNAL=/afs/cern.ch/atlas/offline/external
LHCXX_ARCH=redhat61
CMTCONFIG=i386_linux22
GAUDIROOT=/afs/cern.ch/atlas/offline/external/GaudiRoot
need to find out where and modify to reflect local configuration

But otherwise procedure seemed to be succesful.

To run:
cd ../run
cp ../src/ControlExamples/GaudiExamplesCommon/share/* .
source setup.csh

But then I don't know what I am supposed to run....


Geant 4 Installation
Instructions for installation of Geant 4 are available from the 

Geant 4 Web pages at CERN. The Geant 4 package is 
installed at CERN in ATLAS area in 
/afs/cern.ch/atlas/offline/external/geant4

So on PDSF it was placed in:
/auto/atlas/local/offline/external/geant4

From the Geant 4 
Downloads 
page the source,the 
libraries and 
the data files for:



neutron scattering processes


low energy electromagnetic processes


photon evaporation


radio-active decay hadronic processes (including co-working data set for photon evaporation)

have been downloaded.

The source and the libraries tar files unpacked into: 
/auto/atlas/local/offline/external/geant4/geant4.3.0

and the data files into:
/auto/atlas/local/offline/external/geant4/data

Following Geant 4 Installation Guide
the following environmental variables have been defined:
setenv G4SYSTEM Linux-g++
setenv G4INSTALL /auto/atlas/local/offline/external/geant4/geant4.3.0
setenv CLHEO_BASE_DIR /auto/atlas/sw/lhcxx/specific/redhat61/CLHEP/1.6.0.0

All other variables were set by default (as the  manual describes them).

The manual does not mention it but the build expects to find cernlib in
the path. So I also defined:
setenv CERN /usr/local/cern
setenv CERN_LEVEL 2000
setenv PATH ${PATH}:${CERN}/${CERN_LEVEL}/bin

Then from:
$G4INSTALL/source

gmake was issued and it keeps going and going and going....