ADA and GridPP
David Adams
Original: August 2, 2004
  Update: August 5, 2004.

GridPP is expected to be a major contributor to ADA (ATLAS Distributed
Analysis) described at http://www.usatlas.bnl.gov/ADA. Foremost is
is a python-based interface for job submission and monitoring,
presumably delivered as part of the GANGA project. There are many other
components required to provide ATLAS with an end-to-end analysis
and we additionally look to the GridPP for these components. Most
notable is a software package managment service.

User interface for job submission
---------------------------------
ADA describes ATLAS data in terms of datasets that may include either
event or non-event (e.g. histograms) data. There are transformations
that act on an existing dataset to produce new datasets and jobs that
describe the invocation of a transformation to produce a physical
dataset. The user interface should provide:

1. Means to browse and select from previously defined transformations.

2. Means to create a new transformation by modifying an existing one.
   Note that an ADA transformation is decomposed into an application
   plus a task and it is the latter that users typically modify.

3. Means to construct a dataset from an existing file or collection
   of datasets. This need not appear in the first release.

4. Means to browse and select from existing datasets.

5. Means to submit a job specified as a transformation, dataset and
   user preferences. The job is submitted to an analysis service.

6. Means to select an appropriate anlysis service.

7. Means to interrogate the analysis service or job catalog for the
   status of a previously submitted job. This status may include a
   reference to the partial or complete output dataset.

8. Means to examine a dataset, e.g. see the number of events in an
   event dataset or view the histograms in a histogram datset.

All the above (and more) should be availble from an easy-to-use command
line and an intuitive GUI.

The above is implemented in Python and long term should be very
portable, i.e. can be run on almost any platform. In the short term,
a python wrapper around the DIAL C++ classes can provide a quick
implementation of the command line interface restricted to those
platforms where the DIAL libraries are available. The functionality
of the former may be restricted.

User interface for job monitoring
---------------------------------
ADA provides mechanisms for cataloging both active and inactive jobs.
At present one contacts an analysis service with a job ID and can
ask the service for the ID's of known jobs.  The existing GANGA job
monitoring interface looks attractive here. A user should be able to
get a view of all (or just his or her) high-level jobs submitted over
some time period and then drill down to view the subjobs for any or
these. The view should include essentially everything in the AJDL
job description. The view should be dynamic, i.e. updated as active
job progress.

Package management service
--------------------------
ADA plans to make use of a package management service to locate and
install software packages at a site. This will free users and site
managers from installing software by hand at predefined locations.
Given a package name, this service could do the following:

1. Indicate if the packge is installed.

2. If so, indicate the location (directory).

3. If not, indicate whether it is availble for installation.

4. Install the package.

The service would also manage disk space, deleting packages in
accordance with site and VO policies.

The service should be developed in cooperation with gLite.