From white@fnal.gov Tue Nov 16 18:05:00 1999
Date: Mon, 15 Nov 1999 11:04:50 -0600
From: white@fnal.gov
To: sam-design@fnal.gov
Subject: Station design document - more comments

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "US-ASCII" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Igor, et. al.

I attach a marked up copy of Rich's marked up copy of Igor's station design
doc -- for Igor to take and incorporate parts of in the final note in cvs.

I have four comments which relate to requirements I think we have.  I cannot
immediately assure myself that there is an implementation for each of these
issues which is adequate and will scale.  I think some small tests may be
required to put these matters to rest.

1)It is a requirement that global optimizer be able to make global resource
decisions, based on it's tier of knowledge -- ie. assuming other components
have done the level of optimization they are capable of doing.   It is a
change to the as-yet-unimplemented Optimizer/Resource Management design to
present the entire future file transfer requests - before it can be
guaranteed that there is disk space to carry them out.  I don't think the
Optimizer can do its job unless the requested files it is dealing with
really can be delivered imminently.  So that the statement that CM simply
tells Optimizer about all the files in the future and gets authorization for
them does not seem to work that simply.  It seems that Optimizer must also
know something about max cache space available (if all disposable files were
actually erased) and min space available (if no disposable files were
erased) as well as some figure of merit for disposable files.  That is the
weighing which must go on in the end - - between which is the resource in
greatest demand at that time -- robot arm/tape drive  or  disk space.
This means that Optimizer must have some sort of snapshot picture of the
state of Cache at all of its Stations.   I think this is appropriate in
fact, for a global optimizer.
I think we should assure ourselves that this works by going through some
gedanken experiments with situations where files are to be re-used in the
future, project masters can estimate when they might be free, robot arm
either is or is not the scarcest resource, etc.

2) It is a requirement that a fragmented disk cache have tools to easily see
what is going on and which files are pinned, which in use by which project
etc.  We need this for debugging and for making the users comfortable.  We
also need it for recovery from crashes.  The only implementation mechanism
that has been suggested for this is to use the unix file system as a part of
this mechanism.   What does this cost?  Is this scalable - where are locks
and bottlenecks?  If you have a person running a project with one tiny
file - over and over - what overheads of cache management would there be and
how would these compare with the other overheads of communicating with PM
and opening a file - given that the actual work to read a few bytes is
nothing.  I think that limiting case needs to be studied and understood --
in order to assure ourselves that in the limit the overheads of dynamic
cache management are acceptable.

3) When files are to be copied from one station to another, rather than
refetched from tape,  the authorization for this needs also to go through a
global optimizer.  However, in this case there are 2 cache managers involved
in the transfer -- the initiating one and the affected cache manager for the
source file.  The protocol for this form of file reservation/pinning and
cache management is not clear to me from your note.  Perhaps we can go
through it in detail.

4) Requests for space in the cache in order to write files which must be
flushed through to tape at some rate, in order to not block the cache -- are
not really considered in the design.   It is certainly possible to separate
out these two for the time being and address this later -- with a totally
separate cache for now for write and with no possibility that recently
written files will be re-used -- they would simply have to be refetched.
However, it might be worth some thought and discussion at this stage --
since again the protocol with the global optimizer comes into play as not
only the ability to garner global robot resources is at stake, but the rate
of filling of the cache, and therefore it lack of availability for other
activities must be weighed.

Couple of further comments.  It is clear that we have a number of other
'resource management' parameters and characteristics to specify.  Perhaps
the project and station parameters which we are discussing should be
components of a separate "Configuration and Resource Management"  cvs
package - - which station uses (and global optimizer uses) rather than
embedded into Station Servers package.

As I noted somewhere in the text  - we might want more of a 'batch'
interface for changing file locations.
In general I think we need to do a careful estimate of numbers of
transactions between components in the system and to the database.   I think
the rates are still low, but we nevertheless should write this down -
together with our assumptions about numbers of projects/hour,
consumers/project, rates of consumption, mounts/hour, etc.
I think Sinisa should start getting involved in this part of the Station
design - some days of his were foreseen to be in station work.

We will have to sooner, rather than later, introduce roles for
individuals -- for the admininstrative commands like sam lock...  Julie
should take note and we should consider the operational issue of how these
roles are assigned and by whom.

Vicky


    [ Part 2, Text/HTML (Name: "station.html")  656 lines. ]
    [ Unable to print this part. ]