Minutes for SAM Nov 24, 1998 meeting 
------------------------------------

Present: Julie, Matt, Lee , Heidi, Igor, Steve, Rich

Topics: status and plans

We discussed the plan for the next 6 weeks ( Through Jan 15). The 
items on the plan are as follows:

   1.Bring up our db server machines and make a production version
     of the database. 
   2.Build command line interface for consumer, with example script. 
   3.Complete project editing and creation machinery. 
   4.Work out farm interactions with sam and get sam to work with farm
     batch system. 
   5.Create system to verify import scripts. 
   6.Set up one of the sam nodes as an import station. 
   7.Complete  station design and station disk responsibilities, including
     proxies with other stations. 
   8.Put together a db browser. 
   9.Iterate with D0 on sam interfaces: command line, gui, administrative. 
  10.Connect with the d0 d0om and d0 framework. 
  11.Work on Batch needs for SAM analysis-type environment. 
  12.Prepare for von Ruden Review (Jan 25-28).

Discussion of the plan:

Steve  is working on making sam db server multi-threaded or multi-process
so it can handle many clients simultaneously. He asked what the performance
needs for the server are and if we intend to continue to develop it in Python.
We discussed that we would continue to develop IDL  interfaces and the 
current single threaded version is adequate for the next few weeks/months. 
We will eventually need it to be more perform-ant and would consider  
moving to c++ if needed.

We briefly discussed the db server machines (sun 3500 and 4500) and people
think it is good to have a real production platform for keeping the db 
needed for the D0 MC challenge. Phase I of the MC challenge is starting
now, and we will have 10's to 100's of files to manage early next year.
This is a way to test everything with d0 users. In the summer, Phase II will
start and we plan to have 1000's of files ready for October next year. 
(Ruth and I spoke with Dane about managing the  hardware, and I talked
Dave Fagan this morning (11/25) about getting ready to install these.) 

We went over the command line interface (CLI) and Heidi showed some examples
of what she thought would be useful for the farm system. She and Igor will meet
to discuss this further.  (See her mail in sam_design for more details.)

Concerning point 3, we said that we need to finish up the connections for the
GUI interface. Heidi suggested that the physical streams and L3 triggers will 
be important for the MC challenge. 

I described, briefly, the farm test system and we discussed using their
batch system with the sam system. 

We talked about what is needed for the "verify" of the import python scripts.
There are two aspects, 1. verify that the script python is ok, and 2. 
check that the import information is consistent with itself, and with 
what is already in the data base. We think we may be able to simply have
people run the script on the real database, and not do the commit. We will 
return the error messages and they can keep trying until they 
get the import info correct. This can be done by mailing the scripts to
a test address which returns a log file. When the script works, they
can send the script to an official place, and we will do the official
import, with the commit, when the data is ready.

Short discussion about sam import station. This means we will make one of the
sam testbed nodes an import station, where data can be shipped from 
outside fir import into the sam system.

Matt thought there may be some tools available for browsing the db. 

Concerning use of a batch system for analysis computing. People from 
Platform Computing (LSF batch system) will be at the lab at some undecided
data in the next 2 months. (Two possible dates currently are Dec. 15, and
January 12). A few of us should attend this meeting. We may be able to use
a batch system as part of our station management?? We need to think how/if
SAM needs to communicate with a batch system   under which 
consumers are running.
 
Julie presented some changes she has been working on in the db schema
wrt users and groups. We had a long discussion about where to get the 
userid, groupid etc. from. The easy way is to get it from Yolanda's 
lab data base. We might want to screen this if we get it there or just
enter users as they apply for db accounts. There 
were a couple of relations which needed to be added to the db. Please look
at the current schema for details of  these modifications. There was a lot
more discussion which I glazed over on.

Steve has added exceptions to the db server. They are documented in the IDL 
files.

Rich is interesting in working with the interfaces to the db server and 
how to use them. Also expanding the capabilities of the info server. Next 
Tuesday, he will  give a demo of the current sam admin and we can 
discuss it and what we would like to see in the future. 

Matt wanted to know where the official sam page is now? runIIcomputing/sam
I am not sure if this is automatically updated, but changes should be made to 
the sam cvs area and they will be incorporated asap.

Happy Holiday,

Lee