SAMGrid Plone

I've loaded the event_partitons and sequence_blocks tables with enough data to
run.  Jerry has tested this and it appears to be working fine.

Hi,
    This is supposed to be V7 testing and validation for D0 online. This
is basically the environment Dehong defined for the tests. The metadata
that was declared used the V7 metadata format which I can provide if
necessary, and since that was declared successfully I would be surprised
if a V5 environment would have worked (unless V7 is backwards compatible
for metadata). Also, I learned very recently the event catalog
conventions have changed from V5 to V7, and looking at the contents of
the file on disk it is definitely V7. I also noticed that running sam
get metadata returns the V7 format for the metadata, so I suspect the db
server is V7. Can which version of the db server is running in
dlsam_dev. Note these tests are from host d0oll and not d0olf which has
a special V7 fss/stager running for the tests in the development
environment. Also, this is an attempt to store a new event catalog, not
read a old catalog or store a catalog for an older file.
Here is what I see in the test environment  and to me it looks like the
software at least is V7:
d0oll.fnal.gov} echo $SAM_DB_SERVER_NAME
SAMDbServer.dlsam_dev:SAMDbServer
d0oll.fnal.gov} sam get metadata --file=Stream_1_0191249065_001.raw
ImportedDetectorFile({
             'fileName' : 'Stream_1_0191249065_001.raw',
               'fileId' : 1188386L,
             'fileType' : 'importedDetector',
           'fileFormat' : 'unknown',
             'fileSize' : SamSize('192.87MB'),
                  'crc' : CRC('unknown crc value', 'unknown crc type'),
    'fileContentStatus' : 'unknown',
           'eventCount' : 672L,
             'dataTier' : 'raw',
           'firstEvent' : 5L,
            'lastEvent' : 1320L,
            'startTime' : SamTime(1144896365.0),
              'endTime' : SamTime(1144896436.0),
    'applicationFamily' : ApplicationFamily(appFamily='datalogger',
appName='datalogger', appVersion='0.0'),
                'group' : 'online daq',
           'datastream' : 'stream_1',
        'filePartition' : 1L,
    'lumBlockRangeList' : LumBlockRangeList([LumBlockRange(4294901760L,
4294901782L)]),
    'runDescriptorList' :
RunDescriptorList([RunDescriptor(runType='physics data taking',
runNumber=191249065)]),
    })
compare this to what is seen from the online production environment
which looks like V5 always did:
d0olf.fnal.gov} echo $SAM_DB_SERVER_NAME
SAMDbServer.dlsam_prd:SAMDbServer
d0olf.fnal.gov} sam get metadata --file=daq_test_0000217491_182.raw
             File Type:  Raw Data File
             File Name:  daq_test_0000217491_182.raw
               File ID:  10473974
             File Size:  490265303 [B]
              CRC Data:  1508421560L [adler 32 crc type]
       File Start Time:  04/13/2006 20:56:45
         File End Time:  04/13/2006 21:00:20
       Physical Stream:  daq_test
      File Format Info:  unknown
           First Event:  158666898
            Last Event:  158787508
          Total Events:  2723
        Process Family:  datalogger
          Process Name:  datalogger
       Process Version:  1
             Node Name:  d0mino.fnal.gov
            Work Group:  online daq
             User Name:  d0run
            Run Number:  217491
              Run Type:  physics data taking
    Minimum Luminosity:  4642381
    Maximum Luminosity:  4642382
        File Partition:  182
So I am fairly confident that the testing is under a V7 environment for
the client software side at least. Can the V7 client work against V5 DB
servers? When I try to run a test like that on d0olf (setting up V5
client software but pointing to the dlsam_dev db server I get a traceback:
d0olf.fnal.gov} setup sam_config -q dlsam_dev
d0olf.fnal.gov} sam get metadata --file=Stream_1_0191249065_001.raw
SamUserApiExceptionManager> OUCH, a new kind of exception that we don't
recognize.
sys.exc_info()[0] = Fnorb.orb.CORBA.MARSHAL, sys.exc_info()[1] = Minor:
1096024074L Completed: COMPLETED_NO
Traceback (most recent call last):
  File "/usr/products/sam_user_api/NULL/v5_1_0_7/bin/sam", line 14, in ?
    sys.exit( SamUserApi.main(sys.argv) )
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/SamUserApi.py",
line 145, in main
    return CommandInterface.mainDispatch( commandSuite, args )
  File
"/usr/products/sam_common/NULL/v5_1_1_4/src/python/CommandInterface.py",
line 56, in mainDispatch
    returnStatus = theCommand.dispatch()
  File
"/usr/products/sam_common/NULL/v5_1_1_4/src/python/CommandDefinition.py",
line 604, in dispatch
    returnStatus = self.dispatchHandler(argDict, argList)
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/samMetadata.py",
line 73, in cliDispatch
    result = self.getImplementation()(argDict, argList)
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/samMetadata.py",
line 94, in implementation
    raise SamUserApiExceptionManager.handleExcInfo(sys.exc_info())
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/samMetadata.py",
line 87, in implementation
    metadata = proxy().getFileMetaDataWithCRC(filename)
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/SamUserApiServerProxy.py",
line 179, in __call__
    self.servant = self.proxyConstructor()
  File
"/usr/products/sam_user_api/NULL/v5_1_0_7/src/python/SamUserApiServerProxy.py",
line 272, in proxyConstructor
    self.servant = self.factory.createServant(self.interfaceType)
  File
"/usr/products/sam_common/NULL/v5_1_1_4/stub/SAMDbServer/__init__.py",
line 48520, in createServant
    apply(request.invoke, args, kw)
  File "/usr/products/fnorb/Linux-2-4/v1_1b_12/Fnorb/orb/DII.py", line
160, in invoke
    (forwarded, self.__results) = client.synchronous(self, args)
  File "/usr/products/fnorb/Linux-2-4/v1_1b_12/Fnorb/orb/GIOPClient.py",
line 146, in synchronous
    result = self.__process_reply(request, reply)
  File "/usr/products/fnorb/Linux-2-4/v1_1b_12/Fnorb/orb/GIOPClient.py",
line 435, in __process_reply
    raise self.__unmarshal_system_exception(cursor)
Fnorb.orb.CORBA.MARSHAL: Minor: 1096024074L Completed: COMPLETED_NO
d0olf.fnal.gov} echo $SAM_DB_SERVER_NAME
SAMDbServer.dlsam_dev:SAMDbServer
sam-it@fnal.gov wrote:
>Ticket: #1853: cannot store catalogs to development database from D0 Online,
Topic: D0, Status: accepted, Importance: Medium, Classification: User Problem
>Ticket URL: https://plone3.fnal.gov/SAMGrid/tracking/1853/
>---------------------------------------------------------------------------
>Replying to this automated message will record a follow up to this ticket.
Please adhere to the following:
>  Don't change the message subject. Don't quote any message text. Don't cc to a
mailing list. Don't include a signature.
>  See https://plone3.fnal.gov/SAMGrid/documents for help.
>---------------------------------------------------------------------------
>
>#3 comment 2006-04-13 07:16 PM (swhite)
>---------------------------------------------------------------------------
>Comment:
>    Some unknown person has deleted data from various SAM tables.  We are
working on
>    reloading the tables.  Past file related data is most likely lost.  Since
its
>    only dev I'm more interested in getting future loads to work.
>    **
>    Ugg, your using v5.  I have not even approached reloading that table.  It
data
>    was not what was in the production db.  Is v5 really necessary since D0 is
>    moving to v7?
>
>#2 accept 2006-04-13 05:34 PM (emailsubmitter)
>---------------------------------------------------------------------------
>Change: status: "pending" -> "accepted"
>Change: assignees: "[]" -> "['D0SAMonshift']"
>Change: topic: "" -> "D0"
>
>#1  2006-04-13 05:34 PM (Anonymous User)
>---------------------------------------------------------------------------
>Comment:
>    Hi,
>        For the past couple of days I have not been able to store event
>    catalogs from the online system to the development database.
>    d0oll.fnal.gov} echo $SAM_DB_SERVER_NAME
>    SAMDbServer.dlsam_dev:SAMDbServer
>    Here is the error message I am seeing:
>    importEventsUsingCatFile('/projects/gug/junk/buf22/archive/cat/cat-
>    store/Stream_1_0191249065_001.cat'):
>    ORA-01403: no data found
>    ORA-06512: at "SAMDEV.GROUP_SEQUENCE", line 24
>    ORA-06512: at line 1
>    I checked and at least the catalog file on disk is valid python and I
>    can access the data:
>    d0oll.fnal.gov} cp
>    /projects/gug/junk/buf22/archive/cat/cat-store/Stream_1_0191249065_001.cat
>    a.py
>    d0oll.fnal.gov} python
>    Python 2.4.2 (#1, Dec  2 2005, 11:17:28)
>    [GCC 3.4.3] on linux2
>    Type "help", "copyright", "credits" or "license" for more information.
>    >>> import a
>     >>> a.TheEventList[0].getLevel3()
>    '0x0000001800000017000000160000001500000014000000130000001200000011L'
>     >>>
>    Could this be related to the database problems earlier this week? The
>    ORA errors sound like a potential issue on the database side, maybe a
>    missing table? Can someone investigate this please?
>    --
>    -Jerry->
>    gug@fnal.gov
>    Pepe's Theory of everything: "Under the right circumstances, things
happen."
>
>---------------------------------------------------------------------------
>Original Issue Description:
>Hi,
>    For the past couple of days I have not been able to store event
>catalogs from the online system to the development database.
>d0oll.fnal.gov} echo $SAM_DB_SERVER_NAME
>SAMDbServer.dlsam_dev:SAMDbServer
>Here is the error message I am seeing:
>importEventsUsingCatFile('/projects/gug/junk/buf22/archive/cat/cat-
>store/Stream_1_0191249065_001.cat'):
>ORA-01403: no data found
>ORA-06512: at "SAMDEV.GROUP_SEQUENCE", line 24
>ORA-06512: at line 1
>I checked and at least the catalog file on disk is valid python and I
>can access the data:
>d0oll.fnal.gov} cp
>/projects/gug/junk/buf22/archive/cat/cat-store/Stream_1_0191249065_001.cat
>a.py
>d0oll.fnal.gov} python
>Python 2.4.2 (#1, Dec  2 2005, 11:17:28)
>[GCC 3.4.3] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
>
>
>>>>import a
>>>>
>>>>
> >>> a.TheEventList[0].getLevel3()
>'0x0000001800000017000000160000001500000014000000130000001200000011L'
> >>>
>Could this be related to the database problems earlier this week? The
>ORA errors sound like a potential issue on the database side, maybe a
>missing table? Can someone investigate this please?
>--
>-Jerry->
>gug@fnal.gov
>Pepe's Theory of everything: "Under the right circumstances, things happen."
>
>
--
-Jerry->
gug@fnal.gov
Pepe's Theory of everything: "Under the right circumstances, things happen."
Untrusted issue submission: no verification possible

Some unknown person has deleted data from various SAM tables.  We are working on
reloading the tables.  Past file related data is most likely lost.  Since its
only dev I'm more interested in getting future loads to work.
**
Ugg, your using v5.  I have not even approached reloading that table.  It data
was not what was in the production db.  Is v5 really necessary since D0 is
moving to v7?

Hi,
    For the past couple of days I have not been able to store event
catalogs from the online system to the development database.
d0oll.fnal.gov} echo $SAM_DB_SERVER_NAME
SAMDbServer.dlsam_dev:SAMDbServer
Here is the error message I am seeing:
importEventsUsingCatFile('/projects/gug/junk/buf22/archive/cat/cat-
store/Stream_1_0191249065_001.cat'):
ORA-01403: no data found
ORA-06512: at "SAMDEV.GROUP_SEQUENCE", line 24
ORA-06512: at line 1
I checked and at least the catalog file on disk is valid python and I
can access the data:
d0oll.fnal.gov} cp
/projects/gug/junk/buf22/archive/cat/cat-store/Stream_1_0191249065_001.cat
a.py
d0oll.fnal.gov} python
Python 2.4.2 (#1, Dec  2 2005, 11:17:28)
[GCC 3.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import a
 >>> a.TheEventList[0].getLevel3()
'0x0000001800000017000000160000001500000014000000130000001200000011L'
 >>>
Could this be related to the database problems earlier this week? The
ORA errors sound like a potential issue on the database side, maybe a
missing table? Can someone investigate this please?
--
-Jerry->
gug@fnal.gov
Pepe's Theory of everything: "Under the right circumstances, things happen."