Overview: The SAM CORBA Infrastructure
In the summer of 2003, with new requirements being introduced by CDF
and other experiments beginning to use SAM, the SAM team was granted the
opportunity to redesign the interfaces and implementation of the SAM DBServer.
As a result of this process, we now have in place a CORBA infrastructure that
should help to streamline the development of new methods and interfaces for
communication between the various SAM elements.
At the time of this writing (spring 2004), the new infrastructure has
not been deployed in production. The components of SAM that are written in c++
(sam_station, sam_client_cpplib, etc) have not been reimplemented using the new
concepts (except where necessary for communication with the rewritten
dbServer). However, the components of SAM that are based in python
(sam_user_pyapi, sam_admin_pyapi, sam_mis_pyapi) are well underway in the
process of being reimplemented with the new infrastructure.
Philosophy in a Nutshell:
The driving concepts and philosophies behind the CORBA infrastructure
can be summarized as follows:
- All interfaces are defined in the package sam_idl.
- Data structures and constants should be defined in
sam_idl/sam_corba_base as part of the SAM module, available to all clients.
- Data structures should be designed for re-use between
components.
- Data structure names should include "Struct" or "Union", as
appropriate, within the IDL definition, for clarity (e.g., SamDictionaryStruct,
UserIdentifierUnion, etc.)
- we are still using structures defined outside of
sam_corba_base; these should be phased out as the station is
retooled.
- constants that come from the database itself (e.g.,
appropriate status values or crc types, etc.) should ideally be generated from
the DB; however, this is not practical at the present time, hence the constants
are hard-coded into the IDL and we hope they stay in synch with the
database.
- Every data structure should have a corresponding wrapper class, which
has a corba representation matching the IDL signature, extended with methods as
appropriate to the object.
- The base class for all python wrappers is found in
sam_common_pylib/SamCorba/SamIdlStructWrapperBase.py. Every wrapper includes a
pyRepr() method and a corbaRepr() method, which are used for
automarshalling.
- The derived python wrappers are found under
sam_common_pylib/SamStruct. (Wrappers for CORBA objects which are defined
outside of sam_corba_base, such as the legacy objects defined by the station,
are generally found in sam_common_pylib/SamLegacyStruct).
- The corresponding c++ wrappers are found in sam_lib (should
be renamed to sam_common_cpplib).
- The python/c++ name of the object drops the "Struct" or "Union"
from the object name -- at this level, an object is an object, and we do not
need to know how it is constructed. This is the abstract python/c++
representation of a CORBA object. So a CORBA SamDictionaryStruct becomes a
SamDictionary in python or c++; a CORBA UserIdentifierUnion becomes a
UserIdentifier, etc.
- Note, the c++ code has only implemented wrappers for the
structures it uses; this is true for the python code as well, but the dbServer
is written in python, hence nearly all structures are implemented.
- Marshalling and unmarshalling the
data should be transparent to the user. Client applications should never need
to know anything about how to convert from useful python data structures and
their CORBA representation (which is notoriously difficult to get right,
especially with compound objects).
- All python client code marshalling/unmarshalling is handled
within the sam_common_pylib/SamCorba/SamServerProxy.py classes, which
marshal/unmarshal data structures, results, and exceptions. Users pass in a
high-level python object, and the results of a call are a high-level python
object. This simplifies the client code a very great deal.
- All python server code (dbserver) marshalling/unmarshalling is
handled within the sam_db_srv/src/ServantMethodWrapper.py classes, which are
called when the dbServant is created. The dbserver implementation arguments are
high-level python objects, and the return values are high-level python objects.
This simplifies the dbServer implementations a very great deal.
- Automarshalling is transparent in c++, because the compiler will
cast the object into the appropriate underlying CORBA structure before making
the CORBA call.
- All exceptions that a method can raise should be declared in the IDL.
Exceptions are treated as "just another object" in terms of marshalling and
unmarshalling.
- Constructors of wrapped objects must be able to accept several
different "signatures" of input arguments:
- CORBA object (create a high-level object from the CORBA
datastream)
- Existing instance (create a new object from an existing one)
- Argument lists or argument dictionaries
- constructor overloading in c++ makes this easy
- a bit more difficult in python, since we must inspect the
argument list and determine internally how we are being called, which carries
with it some level of performance hit; at the time of this writing, we are
investigating the performance of various different constructor "template"
algorithms.
Marshalling and
Unmarshalling: What do we mean?
When we in the SAM world speak of marshalling and unmarshalling, we are
extending the meaning beyond merely the CORBA marshalling/unmarshalling. The
CORBA meaning is:
- CORBA marshal:
- Convert a CORBA object into a byte stream and send down the
wire.
- CORBA unmarshal:
- Receive a byte stream and convert it into the corresponding CORBA
object.
In the SAM world, we prefer to think of the higher-level objects
(especially for python applications):
- SAM marshal:
- Convert a high-level python object into it's CORBA representation
for CORBA marshalling.
- SAM unmarshal:
- Convert a CORBA unmarshalled obect into its high-level python
representation
In pictures, it looks something like the following:
client client server server
HighLevelObject CORBA Object NETWORK CORBA Object HighLevelObject
----------------------------------------------------------------------------------------
HLO CO=HLO.corbaRepr() ..bagO'Bytes.. CO HLO.pyRepr()
|-----------------------------------|
corba corba
marshall unmarshal
|============================================================================|
SAM SAM
marshal unmarshal
Critical Points in the CORBA Infrastructure for
Python
Most SAM developers will never need to get down and dirty with the
CORBA infrastructure, because they'll be able to find an expert (namely, me) to
get down and dirty on their behalf. However, for those times when I'm on
vacation or not answering my phone, here are the critical points and a
corresponding brain dump of related info.
- sam_common_pylib/SamCorba/SamIdlStructWrapperBase.py:
- base class wrappers are here, as well as the marshal and
unmarshal functions used by the automarshallers.
- python recursive import problems, hence all the base classes that
need to know about each other are in one great big file; this should be cleaned
up, really... but so far, I've had no luck.
- set DEBUG flag (at the top of the file) for verbose output in
marshalling
- sam_common_pylib/SamCorba/SamServerProxy.py:
- the client marshal/unmarshal code is here.
- also here: an attempt to close unused dbServer connections; I
don't think it is working (python garbage collection at shutdown does not seem
to be very reliable).
- sam_db_srv/src/ServantMethodWrapper.py:
- the dbServer marshal/unmarshal code is here.
- the dbServer also takes special care to make sure that any
exception it raises was declared in the IDL, see the exceptionHandler
- sam_common/SamStruct/*.py:
- each wrapper has a selftest, which calls the dbServer method
testArbitrarySamDataStructure. If you're testing a new data structure, remember
to rebuild sam_idl_pylib, rebuild sam_common_pylib, and then RESTART the
dbserver for testing, or you'll get very strange results.
- use "gmake test" in this directory to test all wrappers. You must
have a dbserver running, of course. The wrappers should all be tested, and
should all pass.
Author: Lauri Loebel Carpenter
Last update by: $Author:
lauri $
Updated on: $Date: 2005/04/22 16:06:46 $
