Use cases for analysis are enumerated here. They are divided into currently supported use cases and use cases that are reasonable but not available. The use cases can be broken into components. These components are listed and linked to instructions on how to execute each of the components.
o sam declare Note: requires knowledge of dataset and fileset. fileset
can be some fake that is done for 10 files at time
and is used by CDF'ers for accounting
Later:
o sam add location < permanent disk>
o samadmin mark file availability status
o sam admin mark file status
... do verification
o copy to tape by hand with encp.
o sam add location < tape>
o special command to add cookie and checksum
o sam delete location
o samadmin mark file availablility status
o samadmin mark file status
o sam declare
Note: Requires knowledge of dataset and fileset. fileset
can be some fake that is done for 10 files at time
and is used by CDF'ers for accounting
Also requires the exact size in kbytes.
o use a python script to put in the CRC checksum
Note: Talks to dbserver so it does not need a password to the db.
o sam add location
Note: Location must be added as a valid location with sam_admin command
o Run concatenation, use sam store to tape for concatenated output
o sam store < permanent disk> o samadmin mark file availability statuso sam admin mark file status ... do verification o copy to tape by hand with encp. o sam add location < tape> o special command to add cookie. (Checksum should be there) o sam delete location o samadmin mark file availablility status o samadmin mark file status
For this case, you can obtain the information on the location and names of files, and use that to just create a list that you put into your tcl file yourself and read from disk. These are the necessary steps:
Let's get started with a test. The goal is to store a file from fcdfdata016.fnal.gov with 1 event.
Here is what to do:ssh fcdflnx2 ssh sam@fcdfdata016 # Get around firewalls cd storetest cp rs-1ev-test-031105-0811.root \ # put in the UNIX time to dl-1ev-test-031105-<time>.root # make the file unique somehow cp rs-1ev-test-031105-0811.py dl-1ev-test-031105-<time>.py # #Edit the file rs-1ev-test-031105-0811.py #(1) Where you see "filename" put # in "dl-1ev-test-031105-<time>.root". #(2) Replace occurances of my username (stdenis) with yours. # # Setup the sam product and create an enviroment variable for the # station name setup sam export SAM_STATION=cdf-sam # # Do the store. Note that you specify the location in pnfs space. # sam store --descrip=dl-1ev-test-031105-<time>.py --source=. \ --dest=/pnfs/cdf/sam/test
You should get back a message like:
[sam@fcdfdata016 storetest]$ sam store --descrip=rs-1ev-test-031105-0820.py --source=. --dest=/pnfs/cdf/sam/test >>>>>> Loading user description of file(s) from rs-1ev-test-031105-0820.py >>>>>> Destination for rs-1ev-test-031105-0820.root is /pnfs/cdf/sam/test >>>>>> Validating metadata for rs-1ev-test-031105-0820.root >>>>>> Declaring 1 file(s) >>>>>> File rs-1ev-test-031105-0820.root has been declared with ID=2314195 >>>>>> Initial file meta-data has been committed >>>>>> Submitting rs-1ev-test-031105-0820.root to FSS at station cdf-sam >>>>>> Submitted all requests, now waiting for callbacks
At this point you will wait for the copy and store to occur. It can take a couple of minutes.
You will get a messsage when it is ok that says:
File store complete. 1
You can check that it was stored ok with:
sam@fcdfdata016 storetest]$ sam locate rs-1ev-test-031105-0811.root
['/pnfs/cdf/sam/test,vo3257'] [sam@fcdfdata016 storetest]$
Once this works, you can try to store from your remote location.
Note that this is a TEST area we are writing to.
The proper way is to use the procedures that code so that handle
automatic creation of
directories in pfns space so that the --dest argument above will in fact
be some kind of dummy argument that tells the code to pick the location.
The file tested here cannot be read back. For the complete testing of
writing and reading, you should refer to the instructions
below.
The easiest way to store a file for CDF is to use a script from DHMods This calls the sam store commands decribed in the basic store test but it also harvests all the metadata from the file and your enviroment so that you don't have to write a metadata file like that described in the basic metadata creation instructions or the advanced description of adding metadata.
The command is evoked by checking out the DHMods package from the cdfsoft enviroment. A help commands is provided:
/DHMods/bin/samStoreCdfFile -help.which gives
Store regular CDF files in SAM Usage: samStoreCdfFileThis will store a file according to a dataset id you provide. To obtain the destination, you need to run a separate script. where-to-store.shpossible options are: --help - this message --file= - full name of the file to be stored - mandatory --dataset= - CDF dataset assigned to the file - mandatory --pnfs= - file destination - mandatory --station= - local SAM station, may be set via $SAM_STATION - mandatory --host= - hostname for the local SAM station, may be set via $SAM_HOST_NAME - mandatory --description= - any description of the file --html= - reference to the description of data --storeoptions= - auxiliary options to be forwarded to the "sam store" command --rename - rename file according CDF convention -v - verbose output -t
When you store a file with SAM, you store some data with it in order to find it later. A filename is unique in all places and times in SAM so if you choose "test" for your filename, you are likely to find you get an error. Therefore you should choose a filename that is unique -- usually by adding the unix time stamp and your station name or location. You should never use the filename with wildcards to search for files: that is what the metadata are used for.
The minimalist version of a metadata file contains the program name, version, nubmer of events, time produced, your name, where it was produced, the type of run, the group, stream and some descriptive text that may be the same for your private dataset. You can also choose your own "cdf dataset" name -- but beware that you choose something that someone else won't also pick or you will mix up your files with them. (This will be made impossible in a few months, but right now, it is a problem.) So putting your kerberos principle as the first part of any cdf dataset name is a good idea. You can also put a reference to some web page where additional information on the dataset is kept. Finally, if you are storing Monte Carlo files, you should NOT use the cdf dataset at the time of the file storage, but rather at the time of the Monte Carlo Request.
An example of this is shown here; while it says Monte Carlo and Generator, this is only a temporary kludge that you must use for all files. Use this anyway for a real file or ntuple. Also, the parameters chosen are for Monte Carlo -- they should be declared in the Monte Carlo Request. They are left here for illustration that parameters can be anything.
from import_classes import *
appfamily=AppFamily('generator', '1.00', 'generator')
filename = 'rs-1ev-test-031106-1910.root'
t = SAMMCFile(filename,Events(1, 2, 2),
"generated",
appfamily,
"01/21/2003 10:59:09",
"01/21/2003 11:20:08",
18,
{
'Global':
{ 'ProducedByName':'mrenna',
'OriginName':'fermilab',
'Phase':'unspecified',
'FacilityName':'fixed-target-farm',
'ProducedForName':'mrenna',
'RunType':'Monte Carlo',
'GroupName':'cdf',
'Stream':'m',
'Description':'test mc',
},
'CDF':
{ 'DataSet':'stink2',
'html':'http://cepa.fnal.gov/personal/mrenna/',
}
'Generated' :{
'AppFamily':'generator',
'FirstEvent':'1',
'AppVersion':'1.00',
'LastEvent':'2',
'NumRecords':'2',
'AppName':'generator',
'TotalEvents':'2',
'RunNumber':54321,}
}
)
For a Monte Carlo file, the request system is working so use this. Note
the use of "requestid" in the "global" category:
from import_classes import *
appfamily=AppFamily('generator', '1.00', 'generator')
filename = 'rs-1ev-test-031106-1910.root'
t = SAMMCFile(filename,Events(1, 2, 2),
"generated",
appfamily,
"01/21/2003 10:59:09",
"01/21/2003 11:20:08",
18,
{
'Global':
{ 'ProducedByName':'mrenna',
'OriginName':'fermilab',
'Phase':'unspecified',
'FacilityName':'fixed-target-farm',
'ProducedForName':'mrenna',
'RunType':'Monte Carlo',
'GroupName':'cdf',
'Stream':'m',
'Description':'test mc',
'requestid':'27'
},
'CDF':
{ 'DataSet':'stink2',
'html':'http://cepa.fnal.gov/personal/mrenna/',
},
'Pythia':
{ 'cdfrelease':'testpycdfrelease',
'collider':'testpycollider',
'comments':'testpycomments',
'decaytable':'testpydecaytable',
'energy':'testpyenergy',
'et_jet_cut':'testpyet_jet_cut',
'fact_scale':'testpyfact_scale',
'lamqcd5':'testpylamqcd5',
'numrecords':'testpynumrecords',
'partons':'testpypartons',
'pdf':'testpypdf',
'physicsprocess':'testpyphysicsprocess',
'picobarns':'testpypicobarns',
'qcd_order':'testpyqcd_order',
'qcd_power':'testpyqcd_power',
'qed_order':'testpyqed_order',
'qed_power':'testpyqed_power',
'ranseed1':'testpyranseed1',
'ranseed2':'testpyranseed2',
'renorm_scale':'testpyrenorm_scale',
'runnumber':'testpyrunnumber',
'useevtgen':'testpyuseevtgen',
'useqq':'testpyuseqq',
'validated':'testpyvalidated',
'version':'testpyversion',
'webpage':'testpywebpage',
},
'Herwig' :
{ 'cdfrelease':'testhercdfrelease',
'collider':'testhercollider',
'comments':'testhercomments',
'decaytable':'testherdecaytable',
'energy':'testherenergy',
'et_jet_cut':'testheret_jet_cut',
'fact_scale':'testherfact_scale',
'lamqcd5':'testherlamqcd5',
'numrecords':'testhernumrecords',
'partons':'testherpartons',
'pdf':'testherpdf',
'physicsprocess':'testherphysicsprocess',
'picobarns':'testherpicobarns',
'qcd_order':'testherqcd_order',
'qcd_power':'testherqcd_power',
'qed_order':'testherqed_order',
'qed_power':'testherqed_power',
'ranseed1':'testherranseed1',
'ranseed2':'testherranseed2',
'renorm_scale':'testherrenorm_scale',
'runnumber':'testherrunnumber',
'validated':'testhervalidated',
'version':'testherversion',
'webpage':'testherwebpage',
},
'Alpgen' :{
'collider':'testalpcollider',
'comments':'testalpcomments',
'dr_jj_cut':'testalpdr_jj_cut',
'dr_lj_cut':'testalpdr_lj_cut',
'energy':'testalpenergy',
'et_jet_cut':'testalpet_jet_cut',
'et_lep_cut':'testalpet_lep_cut',
'fact_scale':'testalpfact_scale',
'lamqcd5':'testalplamqcd5',
'll_mass_cut':'testalpll_mass_cut',
'numrecords':'testalpnumrecords',
'partons':'testalppartons',
'pdf':'testalppdf',
'physicsprocess':'testalpphysicsprocess',
'picobarns':'testalppicobarns',
'qcd_order':'testalpqcd_order',
'qcd_power':'testalpqcd_power',
'qed_order':'testalpqed_order',
'qed_power':'testalpqed_power',
'ranseed1':'testalpranseed1',
'ranseed2':'testalpranseed2',
'renorm_scale':'testalprenorm_scale',
'runnumber':'testalprunnumber',
'validated':'testalpvalidated',
'version':'testalpversion',
'webpage':'testalpwebpage',
'weight':'testalpweight',
},
'Madgraph' :{
'collider':'testmadcollider',
'comments':'testmadcomments',
'dr_jj_cut':'testmaddr_jj_cut',
'dr_lj_cut':'testmaddr_lj_cut',
'energy':'testmadenergy',
'et_jet_cut':'testmadet_jet_cut',
'et_lep_cut':'testmadet_lep_cut',
'fact_scale':'testmadfact_scale',
'lamqcd5':'testmadlamqcd5',
'll_mass_cut':'testmadll_mass_cut',
'numrecords':'testmadnumrecords',
'partons':'testmadpartons',
'pdf':'testmadpdf',
'physicsprocess':'testmadphysicsprocess',
'picobarns':'testmadpicobarns',
'qcd_order':'testmadqcd_order',
'qcd_power':'testmadqcd_power',
'qed_order':'testmadqed_order',
'qed_power':'testmadqed_power',
'ranseed1':'testmadranseed1',
'ranseed2':'testmadranseed2',
'renorm_scale':'testmadrenorm_scale',
'runnumber':'testmadrunnumber',
'validated':'testmadvalidated',
'version':'testmadversion',
'webpage':'testmadwebpage',
'weight':'testmadweight',
},
'Generated' :{
'AppFamily':'generator',
'FirstEvent':'1',
'AppVersion':'1.00',
'LastEvent':'2',
'NumRecords':'2',
'AppName':'generator',
'TotalEvents':'2',
'RunNumber':54321,}
}
)
When you have store metadata for a file, you can retrieve it with the command:
sam get metadata --file=<myfile>as illustrated for the more complex example below.
You can see all the parameter types by the following sam command:
sam translate constraints --dim="help"which returns
Dimensions can be used to query files based on the SAM meta-data, Run Config data, or MCRun parameters. This style of help shows all the available dimensions. Or, you can view a more limited subset of dimensions by using the type option: --dim=help --type=<typeName> Where <typeName> is one of the following: alpgen, cdf, cdfsim, datafile, datasetdef, dfc, herwig, madgraph, mc, mcrun, pythia, runFollowing the instructions above, we issue the folloing command for the typeName "cdf":
[stdenis@nglas05 ~]$ sam translate constraints --dim="help" --type=cdfand we obtain
To use dimension queries, specify dimensions and constraints combined
with and/or/minus operators, as in these examples:
--dim='pythia.topmass 75 and simulated.numrecords > 404'
--rpn='pythia.topmass 75 simulated.numrecords > 404 and'
--dim='pythia.topmass 50-100 or global.originname nixhef'
--dim='(data_tier digitized and appl_name d0reco and version preco03.07.00) \
minus generated.decay > 12'
Available dimensions (not case sensitive):
CDF.DATASET : cdf file catalog dataset
CDF.FILESET : cdf file catalog fileset
CDF.HTML : web page with further information about this file
The syntax then maps into the metadata storage as indicated in the
example below. This example is a very complex and
somewhat nonsensical one that demonstrates a number of
parameters that are available.
from import_classes import *
appfamily=AppFamily('generator', '1.00', 'generator')
filename = 'rs-1ev-test-031106-1910.root'
t = SAMMCFile(filename,Events(1, 2, 2),
"generated",
appfamily,
"01/21/2003 10:59:09",
"01/21/2003 11:20:08",
18,
{
'Global':
{ 'ProducedByName':'mrenna',
'OriginName':'fermilab',
'Phase':'unspecified',
'FacilityName':'fixed-target-farm',
'ProducedForName':'mrenna',
'RunType':'Monte Carlo',
'GroupName':'cdf',
'Stream':'m',
'Description':'test mc',
},
'CDF':
{ 'DataSet':'stink2',
'html':'http://cepa.fnal.gov/personal/mrenna/',
},
'Pythia':
{ 'cdfrelease':'testpycdfrelease',
'collider':'testpycollider',
'comments':'testpycomments',
'decaytable':'testpydecaytable',
'energy':'testpyenergy',
'et_jet_cut':'testpyet_jet_cut',
'fact_scale':'testpyfact_scale',
'lamqcd5':'testpylamqcd5',
'numrecords':'testpynumrecords',
'partons':'testpypartons',
'pdf':'testpypdf',
'physicsprocess':'testpyphysicsprocess',
'picobarns':'testpypicobarns',
'qcd_order':'testpyqcd_order',
'qcd_power':'testpyqcd_power',
'qed_order':'testpyqed_order',
'qed_power':'testpyqed_power',
'ranseed1':'testpyranseed1',
'ranseed2':'testpyranseed2',
'renorm_scale':'testpyrenorm_scale',
'runnumber':'testpyrunnumber',
'useevtgen':'testpyuseevtgen',
'useqq':'testpyuseqq',
'validated':'testpyvalidated',
'version':'testpyversion',
'webpage':'testpywebpage',
},
'Herwig' :
{ 'cdfrelease':'testhercdfrelease',
'collider':'testhercollider',
'comments':'testhercomments',
'decaytable':'testherdecaytable',
'energy':'testherenergy',
'et_jet_cut':'testheret_jet_cut',
'fact_scale':'testherfact_scale',
'lamqcd5':'testherlamqcd5',
'numrecords':'testhernumrecords',
'partons':'testherpartons',
'pdf':'testherpdf',
'physicsprocess':'testherphysicsprocess',
'picobarns':'testherpicobarns',
'qcd_order':'testherqcd_order',
'qcd_power':'testherqcd_power',
'qed_order':'testherqed_order',
'qed_power':'testherqed_power',
'ranseed1':'testherranseed1',
'ranseed2':'testherranseed2',
'renorm_scale':'testherrenorm_scale',
'runnumber':'testherrunnumber',
'validated':'testhervalidated',
'version':'testherversion',
'webpage':'testherwebpage',
},
'Alpgen' :{
'collider':'testalpcollider',
'comments':'testalpcomments',
'dr_jj_cut':'testalpdr_jj_cut',
'dr_lj_cut':'testalpdr_lj_cut',
'energy':'testalpenergy',
'et_jet_cut':'testalpet_jet_cut',
'et_lep_cut':'testalpet_lep_cut',
'fact_scale':'testalpfact_scale',
'lamqcd5':'testalplamqcd5',
'll_mass_cut':'testalpll_mass_cut',
'numrecords':'testalpnumrecords',
'partons':'testalppartons',
'pdf':'testalppdf',
'physicsprocess':'testalpphysicsprocess',
'picobarns':'testalppicobarns',
'qcd_order':'testalpqcd_order',
'qcd_power':'testalpqcd_power',
'qed_order':'testalpqed_order',
'qed_power':'testalpqed_power',
'ranseed1':'testalpranseed1',
'ranseed2':'testalpranseed2',
'renorm_scale':'testalprenorm_scale',
'runnumber':'testalprunnumber',
'validated':'testalpvalidated',
'version':'testalpversion',
'webpage':'testalpwebpage',
'weight':'testalpweight',
},
'Madgraph' :{
'collider':'testmadcollider',
'comments':'testmadcomments',
'dr_jj_cut':'testmaddr_jj_cut',
'dr_lj_cut':'testmaddr_lj_cut',
'energy':'testmadenergy',
'et_jet_cut':'testmadet_jet_cut',
'et_lep_cut':'testmadet_lep_cut',
'fact_scale':'testmadfact_scale',
'lamqcd5':'testmadlamqcd5',
'll_mass_cut':'testmadll_mass_cut',
'numrecords':'testmadnumrecords',
'partons':'testmadpartons',
'pdf':'testmadpdf',
'physicsprocess':'testmadphysicsprocess',
'picobarns':'testmadpicobarns',
'qcd_order':'testmadqcd_order',
'qcd_power':'testmadqcd_power',
'qed_order':'testmadqed_order',
'qed_power':'testmadqed_power',
'ranseed1':'testmadranseed1',
'ranseed2':'testmadranseed2',
'renorm_scale':'testmadrenorm_scale',
'runnumber':'testmadrunnumber',
'validated':'testmadvalidated',
'version':'testmadversion',
'webpage':'testmadwebpage',
'weight':'testmadweight',
},
'Generated' :{
'AppFamily':'generator',
'FirstEvent':'1',
'AppVersion':'1.00',
'LastEvent':'2',
'NumRecords':'2',
'AppName':'generator',
'TotalEvents':'2',
'RunNumber':54321,}
}
)
The metadata for a file stored in this way can then be examined with the command:
sam get metadata --file='rs-1ev-test-031106-1910.root'
which returns
File Type: SAMMC Data File
File Name: rs-1ev-test-031106-1910.root
File ID: 2318128
File Size: 307446 [B]
CRC Data: 525925219L [adler 32 crc type]
File Start Time: 01/21/2003 10:59:09
File End Time: 01/21/2003 11:20:08
Physical Stream: m
File Format Info: unknown file format
First Event: 1
Last Event: 2
Total Events: 2
Application Family: generator
Application Name: generator
Application Version: 1.00
Import Process ID: 0
Node Name: fcdfdata016.fnal.gov
Work Group: cdf
User Name: sam
Produced For: mrenna
Produced By: mrenna
Origin Location: fermilab
Origin Facility: fixed-target-farm
Physics Channel:
Description: test mc
MC Phase: unspecified
Run Number: 54321
Run Type: monte carlo
Run Start Time: 01/21/2003 10:59:09
Run End Time: 01/21/2003 11:20:08
Run Description: test mc
Run CM Energy: 0.0
Parent Files: []
Split: 0
Merge: 0
Key: collider = testalpcollider (Category: alpgen)
Key: comments = testalpcomments (Category: alpgen)
Key: dr_jj_cut = testalpdr_jj_cut (Category: alpgen)
Key: dr_lj_cut = testalpdr_lj_cut (Category: alpgen)
Key: energy = testalpenergy (Category: alpgen)
Key: et_jet_cut = testalpet_jet_cut (Category: alpgen)
Key: et_lep_cut = testalpet_lep_cut (Category: alpgen)
Key: fact_scale = testalpfact_scale (Category: alpgen)
Key: lamqcd5 = testalplamqcd5 (Category: alpgen)
Key: ll_mass_cut = testalpll_mass_cut (Category: alpgen)
Key: numrecords = testalpnumrecords (Category: alpgen)
Key: partons = testalppartons (Category: alpgen)
Key: pdf = testalppdf (Category: alpgen)
Key: physicsprocess = testalpphysicsprocess (Category: alpgen)
Key: picobarns = testalppicobarns (Category: alpgen)
Key: qcd_order = testalpqcd_order (Category: alpgen)
Key: qcd_power = testalpqcd_power (Category: alpgen)
Key: qed_order = testalpqed_order (Category: alpgen)
Key: qed_power = testalpqed_power (Category: alpgen)
Key: ranseed1 = testalpranseed1 (Category: alpgen)
Key: ranseed2 = testalpranseed2 (Category: alpgen)
Key: renorm_scale = testalprenorm_scale (Category: alpgen)
Key: runnumber = testalprunnumber (Category: alpgen)
Key: validated = testalpvalidated (Category: alpgen)
Key: version = testalpversion (Category: alpgen)
Key: webpage = testalpwebpage (Category: alpgen)
Key: weight = testalpweight (Category: alpgen)
Key: dataset = stink2 (Category: cdf)
Key: html = http://cepa.fnal.gov/personal/mrenna/ (Category: cdf)
Key: appfamily = generator (Category: generated)
Key: appname = generator (Category: generated)
Key: appversion = 1.00 (Category: generated)
Key: firstevent = 1 (Category: generated)
Key: lastevent = 2 (Category: generated)
Key: numrecords = 2 (Category: generated)
Key: runnumber = 54321 (Category: generated)
Key: totalevents = 2 (Category: generated)
Key: facilityname = fixed-target-farm (Category: global)
Key: groupname = cdf (Category: global)
Key: originname = fermilab (Category: global)
Key: phase = unspecified (Category: global)
Key: producedbyname = mrenna (Category: global)
Key: producedforname = mrenna (Category: global)
Key: cdfrelease = testhercdfrelease (Category: herwig)
Key: collider = testhercollider (Category: herwig)
Key: comments = testhercomments (Category: herwig)
Key: decaytable = testherdecaytable (Category: herwig)
Key: energy = testherenergy (Category: herwig)
Key: et_jet_cut = testheret_jet_cut (Category: herwig)
Key: fact_scale = testherfact_scale (Category: herwig)
Key: lamqcd5 = testherlamqcd5 (Category: herwig)
Key: numrecords = testhernumrecords (Category: herwig)
Key: partons = testherpartons (Category: herwig)
Key: pdf = testherpdf (Category: herwig)
Key: physicsprocess = testherphysicsprocess (Category: herwig)
Key: picobarns = testherpicobarns (Category: herwig)
Key: qcd_order = testherqcd_order (Category: herwig)
Key: qcd_power = testherqcd_power (Category: herwig)
Key: qed_order = testherqed_order (Category: herwig)
Key: qed_power = testherqed_power (Category: herwig)
Key: ranseed1 = testherranseed1 (Category: herwig)
Key: ranseed2 = testherranseed2 (Category: herwig)
Key: renorm_scale = testherrenorm_scale (Category: herwig)
Key: runnumber = testherrunnumber (Category: herwig)
Key: validated = testhervalidated (Category: herwig)
Key: version = testherversion (Category: herwig)
Key: webpage = testherwebpage (Category: herwig)
Key: collider = testmadcollider (Category: madgraph)
Key: comments = testmadcomments (Category: madgraph)
Key: dr_jj_cut = testmaddr_jj_cut (Category: madgraph)
Key: dr_lj_cut = testmaddr_lj_cut (Category: madgraph)
Key: energy = testmadenergy (Category: madgraph)
Key: et_jet_cut = testmadet_jet_cut (Category: madgraph)
Key: et_lep_cut = testmadet_lep_cut (Category: madgraph)
Key: fact_scale = testmadfact_scale (Category: madgraph)
Key: lamqcd5 = testmadlamqcd5 (Category: madgraph)
Key: ll_mass_cut = testmadll_mass_cut (Category: madgraph)
Key: numrecords = testmadnumrecords (Category: madgraph)
Key: partons = testmadpartons (Category: madgraph)
Key: pdf = testmadpdf (Category: madgraph)
Key: physicsprocess = testmadphysicsprocess (Category: madgraph)
Key: picobarns = testmadpicobarns (Category: madgraph)
Key: qcd_order = testmadqcd_order (Category: madgraph)
Key: qcd_power = testmadqcd_power (Category: madgraph)
Key: qed_order = testmadqed_order (Category: madgraph)
Key: qed_power = testmadqed_power (Category: madgraph)
Key: ranseed1 = testmadranseed1 (Category: madgraph)
Key: ranseed2 = testmadranseed2 (Category: madgraph)
Key: renorm_scale = testmadrenorm_scale (Category: madgraph)
Key: runnumber = testmadrunnumber (Category: madgraph)
Key: validated = testmadvalidated (Category: madgraph)
Key: version = testmadversion (Category: madgraph)
Key: webpage = testmadwebpage (Category: madgraph)
Key: weight = testmadweight (Category: madgraph)
Key: cdfrelease = testpycdfrelease (Category: pythia)
Key: collider = testpycollider (Category: pythia)
Key: comments = testpycomments (Category: pythia)
Key: decaytable = testpydecaytable (Category: pythia)
Key: energy = testpyenergy (Category: pythia)
Key: et_jet_cut = testpyet_jet_cut (Category: pythia)
Key: fact_scale = testpyfact_scale (Category: pythia)
Key: lamqcd5 = testpylamqcd5 (Category: pythia)
Key: numrecords = testpynumrecords (Category: pythia)
Key: partons = testpypartons (Category: pythia)
Key: pdf = testpypdf (Category: pythia)
Key: physicsprocess = testpyphysicsprocess (Category: pythia)
Key: picobarns = testpypicobarns (Category: pythia)
Key: qcd_order = testpyqcd_order (Category: pythia)
Key: qcd_power = testpyqcd_power (Category: pythia)
Key: qed_order = testpyqed_order (Category: pythia)
Key: qed_power = testpyqed_power (Category: pythia)
Key: ranseed1 = testpyranseed1 (Category: pythia)
Key: ranseed2 = testpyranseed2 (Category: pythia)
Key: renorm_scale = testpyrenorm_scale (Category: pythia)
Key: runnumber = testpyrunnumber (Category: pythia)
Key: useevtgen = testpyuseevtgen (Category: pythia)
Key: useqq = testpyuseqq (Category: pythia)
Key: validated = testpyvalidated (Category: pythia)
Key: version = testpyversion (Category: pythia)
Key: webpage = testpywebpage (Category: pythia)
Request ID: 0
Data Tier: generated
[stdenis@nglas05 ~]$ sam translate constraints --dim="cdf.dataset stink2"to obtain:
Files: sm-store-test1.root rs-1ev-test-031106-1810.root rs-1ev-test-031106-1840.root rs-1ev-test-031106-1844.root rs-1ev-test-031106-1844-1.root rs-1ev-test-031106-1845.root rs-1ev-test-031106-1845-1.root rs-1ev-test-031106-1847.root rs-1ev-test-031106-1855.root rs-1ev-test-031106-1910.root rs-1ev-test-031114-2251.root File Count: 11 Average File Size: 300 Total File Size: 3300 Total Event Count: 22
For Monte Carlo, you do not use the dataset to do the dimension. You
use the request id. This is because you will have looked for the
Monte Carlo based on some description and then found some samples
that are interesting. With each sample is a request id and when
you want to use that sample, you ask for the file based on this
id. The request ID is guaranteed to be unique. For example, if
the request given by requestId=27 is what you want,
the sam translate constraints will be done as follows:
sam translate constraints --dim="GLOBAL.REQUESTID 27"yielding
Files: samTueMay25133949CDT2004.root samTueMay25145356CDT2004.root File Count: 2 Average File Size: 300 Total File Size: 600 Total Event Count: 4
from SamUserApiImportClasses import *
datatier='reconstructed'
appfamily=AppFamily('generator', '1.00', 'generator')
dict={
'cdf':{
'dataset':'stink2',
'html':'http://cepa.fnal.gov/personal/mrenna/',
},
'Pythia':
{ 'cdfrelease':'testpycdfrelease',
'collider':'testpycollider',
'comments':'testpycomments',
'decaytable':'testpydecaytable',
'energy':'testpyenergy',
'et_jet_cut':'testpyet_jet_cut',
'fact_scale':'testpyfact_scale',
'lamqcd5':'testpylamqcd5',
'numrecords':'testpynumrecords',
'partons':'testpypartons',
'pdf':'testpypdf',
'physicsprocess':'testpyphysicsprocess',
'picobarns':'testpypicobarns',
'qcd_order':'testpyqcd_order',
'qcd_power':'testpyqcd_power',
'qed_order':'testpyqed_order',
'qed_power':'testpyqed_power',
'ranseed1':'testpyranseed1',
'ranseed2':'testpyranseed2',
'renorm_scale':'testpyrenorm_scale',
'runnumber':'testpyrunnumber',
'useevtgen':'testpyuseevtgen',
'useqq':'testpyuseqq',
'validated':'testpyvalidated',
'version':'testpyversion',
'webpage':'testpywebpage',
},
'Herwig' :
{ 'cdfrelease':'testhercdfrelease',
'collider':'testhercollider',
'comments':'testhercomments',
'decaytable':'testherdecaytable',
'energy':'testherenergy',
'et_jet_cut':'testheret_jet_cut',
'fact_scale':'testherfact_scale',
'lamqcd5':'testherlamqcd5',
'numrecords':'testhernumrecords',
'partons':'testherpartons',
'pdf':'testherpdf',
'physicsprocess':'testherphysicsprocess',
'picobarns':'testherpicobarns',
'qcd_order':'testherqcd_order',
'qcd_power':'testherqcd_power',
'qed_order':'testherqed_order',
'qed_power':'testherqed_power',
'ranseed1':'testherranseed1',
'ranseed2':'testherranseed2',
'renorm_scale':'testherrenorm_scale',
'runnumber':'testherrunnumber',
'validated':'testhervalidated',
'version':'testherversion',
'webpage':'testherwebpage',
},
'Alpgen' : {
'collider':'testalpcollider',
'comments':'testalpcomments',
'dr_jj_cut':'testalpdr_jj_cut',
'dr_lj_cut':'testalpdr_lj_cut',
'energy':'testalpenergy',
'et_jet_cut':'testalpet_jet_cut',
'et_lep_cut':'testalpet_lep_cut',
'fact_scale':'testalpfact_scale',
'lamqcd5':'testalplamqcd5',
'll_mass_cut':'testalpll_mass_cut',
'numrecords':'testalpnumrecords',
'partons':'testalppartons',
'pdf':'testalppdf',
'physicsprocess':'testalpphysicsprocess',
'picobarns':'testalppicobarns',
'qcd_order':'testalpqcd_order',
'qcd_power':'testalpqcd_power',
'qed_order':'testalpqed_order',
'qed_power':'testalpqed_power',
'ranseed1':'testalpranseed1',
'ranseed2':'testalpranseed2',
'renorm_scale':'testalprenorm_scale',
'runnumber':'testalprunnumber',
'validated':'testalpvalidated',
'version':'testalpversion',
'webpage':'testalpwebpage',
'weight':'testalpweight',
},
'Madgraph' :{
'collider':'testmadcollider',
'comments':'testmadcomments',
'dr_jj_cut':'testmaddr_jj_cut',
'dr_lj_cut':'testmaddr_lj_cut',
'energy':'testmadenergy',
'et_jet_cut':'testmadet_jet_cut',
'et_lep_cut':'testmadet_lep_cut',
'fact_scale':'testmadfact_scale',
'lamqcd5':'testmadlamqcd5',
'll_mass_cut':'testmadll_mass_cut',
'numrecords':'testmadnumrecords',
'partons':'testmadpartons',
'pdf':'testmadpdf',
'physicsprocess':'testmadphysicsprocess',
'picobarns':'testmadpicobarns',
'qcd_order':'testmadqcd_order',
'qcd_power':'testmadqcd_power',
'qed_order':'testmadqed_order',
'qed_power':'testmadqed_power',
'ranseed1':'testmadranseed1',
'ranseed2':'testmadranseed2',
'renorm_scale':'testmadrenorm_scale',
'runnumber':'testmadrunnumber',
'validated':'testmadvalidated',
'version':'testmadversion',
'webpage':'testmadwebpage',
'weight':'testmadweight',
},
'Global':{
'phase':'undefined',
'stream':'notstreamed',
'description':'junk',
'producedforname':'stdenis',
'runtype':'monte carlo',
'groupname':'test',
},
'Generated':{
'useevtgen':'on',
'collisionenergy':'1960.0',
'decay':'tauola',
'generator':'pythia',
'pdflibfunc':'CTEQ5',
},
}
This request is obviously nonsense but demonstrates the generators and
parameters available. In a real case, one of these would be chosen
with sensible values.The request is then sent by the following command:
sam create request --dictfile=req3.py --group=test --numEvents=1and one obtains a request id:
RequestId 32This request id is then entered in the file metdata under the global parameter category. The request can be queried:
sam get request details --requestId=32This returns all the info:
Request Detail ID:21 Family: generator Application Name: generator Version: 1.00 Request Detail Status: okay Proj Snap ID: 0 Request ID: 32 Installation ID: 0 Param Type: runtype Value: monte carlo Category: global Data_tier: reconstructed Description: Param Type: stream Value: notstreamed Category: global Data_tier: reconstructed Description: Param Type: description Value: junk Category: global Data_tier: reconstructed Description: Param Type: producedforname Value: stdenis Category: global Data_tier: reconstructed Description: Param Type: groupname Value: test Category: global Data_tier: reconstructed Description: Param Type: useevtgen Value: on Category: generated Data_tier: reconstructed Description: Param Type: collisionenergy Value: 1960.0 Category: generated Data_tier: reconstructed Description: Param Type: dataset Value: stink2 Category: cdf Data_tier: reconstructed Description: Param Type: html Value: http://cepa.fnal.gov/personal/mrenna/ Category: cdf Data_tier: reconstructed Description: Param Type: dr_lj_cut Value: testmaddr_lj_cut Category: madgraph Data_tier: reconstructed Description: Param Type: qed_order Value: testmadqed_order Category: madgraph Data_tier: reconstructed Description: Param Type: pdf Value: testmadpdf Category: madgraph Data_tier: reconstructed Description: Param Type: webpage Value: testmadwebpage Category: madgraph Data_tier: reconstructed Description: Param Type: et_jet_cut Value: testmadet_jet_cut Category: madgraph Data_tier: reconstructed Description: Param Type: version Value: testmadversion Category: madgraph Data_tier: reconstructed Description: Param Type: energy Value: testmadenergy Category: madgraph Data_tier: reconstructed Description: Param Type: comments Value: testmadcomments Category: madgraph Data_tier: reconstructed Description: Param Type: numrecords Value: testmadnumrecords Category: madgraph Data_tier: reconstructed Description: Param Type: ranseed2 Value: testmadranseed2 Category: madgraph Data_tier: reconstructed Description: Param Type: ranseed1 Value: testmadranseed1 Category: madgraph Data_tier: reconstructed Description: Param Type: ll_mass_cut Value: testmadll_mass_cut Category: madgraph Data_tier: reconstructed Description: Param Type: collider Value: testmadcollider Category: madgraph Data_tier: reconstructed Description: Param Type: qcd_order Value: testmadqcd_order Category: madgraph Data_tier: reconstructed Description: Param Type: renorm_scale Value: testmadrenorm_scale Category: madgraph Data_tier: reconstructed Description: Param Type: validated Value: testmadvalidated Category: madgraph Data_tier: reconstructed Description: Param Type: dr_jj_cut Value: testmaddr_jj_cut Category: madgraph Data_tier: reconstructed Description: Param Type: picobarns Value: testmadpicobarns Category: madgraph Data_tier: reconstructed Description: Param Type: fact_scale Value: testmadfact_scale Category: madgraph Data_tier: reconstructed Description: Param Type: lamqcd5 Value: testmadlamqcd5 Category: madgraph Data_tier: reconstructed Description: Param Type: et_lep_cut Value: testmadet_lep_cut Category: madgraph Data_tier: reconstructed Description: Param Type: runnumber Value: testmadrunnumber Category: madgraph Data_tier: reconstructed Description: Param Type: physicsprocess Value: testmadphysicsprocess Category: madgraph Data_tier: reconstructed Description: Param Type: qcd_power Value: testmadqcd_power Category: madgraph Data_tier: reconstructed Description: Param Type: weight Value: testmadweight Category: madgraph Data_tier: reconstructed Description: Param Type: partons Value: testmadpartons Category: madgraph Data_tier: reconstructed Description: Param Type: qed_power Value: testmadqed_power Category: madgraph Data_tier: reconstructed Description: Param Type: qed_order Value: testpyqed_order Category: pythia Data_tier: reconstructed Description: Param Type: pdf Value: testpypdf Category: pythia Data_tier: reconstructed Description: Param Type: et_jet_cut Value: testpyet_jet_cut Category: pythia Data_tier: reconstructed Description: Param Type: version Value: testpyversion Category: pythia Data_tier: reconstructed Description: Param Type: comments Value: testpycomments Category: pythia Data_tier: reconstructed Description: Param Type: numrecords Value: testpynumrecords Category: pythia Data_tier: reconstructed Description: Param Type: cdfrelease Value: testpycdfrelease Category: pythia Data_tier: reconstructed Description: Param Type: ranseed1 Value: testpyranseed1 Category: pythia Data_tier: reconstructed Description: Param Type: collider Value: testpycollider Category: pythia Data_tier: reconstructed Description: Param Type: decaytable Value: testpydecaytable Category: pythia Data_tier: reconstructed Description: Param Type: ranseed2 Value: testpyranseed2 Category: pythia Data_tier: reconstructed Description: Param Type: qcd_order Value: testpyqcd_order Category: pythia Data_tier: reconstructed Description: Param Type: useevtgen Value: testpyuseevtgen Category: pythia Data_tier: reconstructed Description: Param Type: renorm_scale Value: testpyrenorm_scale Category: pythia Data_tier: reconstructed Description: Param Type: validated Value: testpyvalidated Category: pythia Data_tier: reconstructed Description: Param Type: partons Value: testpypartons Category: pythia Data_tier: reconstructed Description: Param Type: picobarns Value: testpypicobarns Category: pythia Data_tier: reconstructed Description: Param Type: fact_scale Value: testpyfact_scale Category: pythia Data_tier: reconstructed Description: Param Type: lamqcd5 Value: testpylamqcd5 Category: pythia Data_tier: reconstructed Description: Param Type: energy Value: testpyenergy Category: pythia Data_tier: reconstructed Description: Param Type: runnumber Value: testpyrunnumber Category: pythia Data_tier: reconstructed Description: Param Type: webpage Value: testpywebpage Category: pythia Data_tier: reconstructed Description: Param Type: qcd_power Value: testpyqcd_power Category: pythia Data_tier: reconstructed Description: Param Type: physicsprocess Value: testpyphysicsprocess Category: pythia Data_tier: reconstructed Description: Param Type: useqq Value: testpyuseqq Category: pythia Data_tier: reconstructed Description: Param Type: qed_power Value: testpyqed_power Category: pythia Data_tier: reconstructed Description: Param Type: dr_lj_cut Value: testalpdr_lj_cut Category: alpgen Data_tier: reconstructed Description: Param Type: qed_order Value: testalpqed_order Category: alpgen Data_tier: reconstructed Description: Param Type: pdf Value: testalppdf Category: alpgen Data_tier: reconstructed Description: Param Type: webpage Value: testalpwebpage Category: alpgen Data_tier: reconstructed Description: Param Type: et_jet_cut Value: testalpet_jet_cut Category: alpgen Data_tier: reconstructed Description: Param Type: version Value: testalpversion Category: alpgen Data_tier: reconstructed Description: Param Type: energy Value: testalpenergy Category: alpgen Data_tier: reconstructed Description: Param Type: comments Value: testalpcomments Category: alpgen Data_tier: reconstructed Description: Param Type: numrecords Value: testalpnumrecords Category: alpgen Data_tier: reconstructed Description: Param Type: ranseed2 Value: testalpranseed2 Category: alpgen Data_tier: reconstructed Description: Param Type: ranseed1 Value: testalpranseed1 Category: alpgen Data_tier: reconstructed Description: Param Type: ll_mass_cut Value: testalpll_mass_cut Category: alpgen Data_tier: reconstructed Description: Param Type: collider Value: testalpcollider Category: alpgen Data_tier: reconstructed Description: Param Type: qcd_order Value: testalpqcd_order Category: alpgen Data_tier: reconstructed Description: Param Type: renorm_scale Value: testalprenorm_scale Category: alpgen Data_tier: reconstructed Description: Param Type: validated Value: testalpvalidated Category: alpgen Data_tier: reconstructed Description: Param Type: dr_jj_cut Value: testalpdr_jj_cut Category: alpgen Data_tier: reconstructed Description: Param Type: picobarns Value: testalppicobarns Category: alpgen Data_tier: reconstructed Description: Param Type: fact_scale Value: testalpfact_scale Category: alpgen Data_tier: reconstructed Description: Param Type: lamqcd5 Value: testalplamqcd5 Category: alpgen Data_tier: reconstructed Description: Param Type: et_lep_cut Value: testalpet_lep_cut Category: alpgen Data_tier: reconstructed Description: Param Type: runnumber Value: testalprunnumber Category: alpgen Data_tier: reconstructed Description: Param Type: physicsprocess Value: testalpphysicsprocess Category: alpgen Data_tier: reconstructed Description: Param Type: qcd_power Value: testalpqcd_power Category: alpgen Data_tier: reconstructed Description: Param Type: weight Value: testalpweight Category: alpgen Data_tier: reconstructed Description: Param Type: partons Value: testalppartons Category: alpgen Data_tier: reconstructed Description: Param Type: qed_power Value: testalpqed_power Category: alpgen Data_tier: reconstructed Description: Param Type: qed_order Value: testherqed_order Category: herwig Data_tier: reconstructed Description: Param Type: pdf Value: testherpdf Category: herwig Data_tier: reconstructed Description: Param Type: et_jet_cut Value: testheret_jet_cut Category: herwig Data_tier: reconstructed Description: Param Type: version Value: testherversion Category: herwig Data_tier: reconstructed Description: Param Type: comments Value: testhercomments Category: herwig Data_tier: reconstructed Description: Param Type: numrecords Value: testhernumrecords Category: herwig Data_tier: reconstructed Description: Param Type: cdfrelease Value: testhercdfrelease Category: herwig Data_tier: reconstructed Description: Param Type: ranseed1 Value: testherranseed1 Category: herwig Data_tier: reconstructed Description: Param Type: collider Value: testhercollider Category: herwig Data_tier: reconstructed Description: Param Type: decaytable Value: testherdecaytable Category: herwig Data_tier: reconstructed Description: Param Type: ranseed2 Value: testherranseed2 Category: herwig Data_tier: reconstructed Description: Param Type: qcd_order Value: testherqcd_order Category: herwig Data_tier: reconstructed Description: Param Type: renorm_scale Value: testherrenorm_scale Category: herwig Data_tier: reconstructed Description: Param Type: validated Value: testhervalidated Category: herwig Data_tier: reconstructed Description: Param Type: partons Value: testherpartons Category: herwig Data_tier: reconstructed Description: Param Type: picobarns Value: testherpicobarns Category: herwig Data_tier: reconstructed Description: Param Type: fact_scale Value: testherfact_scale Category: herwig Data_tier: reconstructed Description: Param Type: lamqcd5 Value: testherlamqcd5 Category: herwig Data_tier: reconstructed Description: Param Type: energy Value: testherenergy Category: herwig Data_tier: reconstructed Description: Param Type: runnumber Value: testherrunnumber Category: herwig Data_tier: reconstructed Description: Param Type: webpage Value: testherwebpage Category: herwig Data_tier: reconstructed Description: Param Type: qcd_power Value: testherqcd_power Category: herwig Data_tier: reconstructed Description: Param Type: physicsprocess Value: testherphysicsprocess Category: herwig Data_tier: reconstructed Description: Param Type: qed_power Value: testherqed_power Category: herwig Data_tier: reconstructed Description: Param Type: phase Value: undefined Category: global Data_tier: reconstructed Description: Param Type: generator Value: pythia Category: generated Data_tier: reconstructed Description: Param Type: decay Value: tauola Category: generated Data_tier: reconstructed Description: Param Type: pdflibfunc Value: CTEQ5 Category: generated Data_tier: reconstructed Description:Requests can be listed:
sam list requests --requestIdGt=0giving
[1, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [2, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [3, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [4, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [5, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [6, 'new', 'mc-request', 1, 'sam', 'test', 10000, 'none'] [7, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [8, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [9, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [10, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [11, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [12, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [13, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [14, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [15, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [16, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [17, 'new', 'mc-request', 1, 'lauri', 'test', 1, 'none'] [18, 'new', 'mc-request', 1, 'lauri', 'test', 1, 'none'] [19, 'new', 'mc-request', 1, 'lauri', 'test', 1, 'none'] [20, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [21, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [22, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [23, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [24, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [25, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [26, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [27, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [28, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [29, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [30, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [31, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none'] [32, 'new', 'mc-request', 1, 'sam', 'test', 1, 'none']This functionality will soon be available on web pages, just as they are in D0. It is also possible to use
sam modify request ...To modify the number of events, the status of the request and so on.
[stdenis@nglas05 ~]$ sam locate sm-store-test1.rootwhich yields
['/pnfs/cdfen/filesets/SM/SMTest,ia3937']This is dull -- just the location in enstore, along with the "enstore cookie" (don't ask).
Here is a more intersting one (common file used in all tests, so it gets around):
[stdenis@nglas05 ~]$ sam locate gb01defd.0001exo0yielding
['/pnfs/cdfen/filesets/GI/GI05/GI0500/GI0500.0,ia3638', 'ncdf68.fnal.gov:/scratch/sam/cache1/boo', 'nglas09.fnal.gov:/cdf/scratch/sam/cache/cdfyale/prd/boo', 'lf7.ph.gla.ac.uk:/localhome/sam/cache1/boo', 'tuhept.phy.tufts.edu:/home/sam/cache1/boo', 'cdf3.uchicago.edu:/cdf/data3a/boo', 'nglas07.fnal.gov:/cdf/scratch/sam/pro/boo', 'nglas08.fnal.gov:/cdf/scratch/sam/prd/boo', 'testwulf.hpcc.ttu.edu:/home/sam/cache1/prd/boo', 'matrix.physics.ox.ac.uk:/eweak/disk1/sam/boo', 'cdfg.ph.gla.ac.uk:/data3/sam/prd/boo', 'cdf001.ucsd.edu:/cdf/data01/cdf001/cdf-sam/cache/boo', 'nglas03.fnal.gov:/data3/sam/prd/boo', 'nglas10.fnal.gov:/data/nglas10/a/sam/prd/boo', 'nglas05.fnal.gov:/data3/sam/pro/boo', 'nglas04.fnal.gov:/data3/sam/pro/boo', 'nglas06.fnal.gov:/data1/sam/prd/boo', 'fcdfdata016.fnal.gov:/data1/cdf-sam/cache/cdf-scotgrid-2/prd/boo', 'pccdf2.ts.infn.it:/cdf3/sam_cache/boo', 'dcap://cdfdca-door01:dcap://cdfdca.fnal.gov:25125/pnfs/fnal.gov/usr//cdfen/filesets/GI/GI05/GI0500/GI0500.0', 'dcap://cdfdca-door03:dcap://cdfdca.fnal.gov:25137/pnfs/fnal.gov/usr//cdfen/filesets/GI/GI05/GI0500/GI0500.0', 'dcap://cdfdca-door02:dcap://cdfdca.fnal.gov:25136/pnfs/fnal.gov/usr//cdfen/filesets/GI/GI05/GI0500/GI0500.0', 'dcap://cdfdca-door04:dcap://cdfdca.fnal.gov:25138/pnfs/fnal.gov/usr//cdfen/filesets/GI/GI05/GI0500/GI0500.0', 'cdfsam.cnaf.infn.it:/cdf/data/data001/SAM-100GB-cache/boo']
The syntax of the sam translate has been shown elsewhere. This can then be used to define a dataset as follows:
sam create dataset definition --defname=stink2 --group=test \
--defdesc='test mc param store with mrenna' \
--dim='cdf.dataset stink2'
A definition can be examined with
sam translate constraints --dim="DATASET_DEF_NAME stink2"This gives:
[stdenis@nglas05 ~]$ sam translate constraints --dim="DATASET_DEF_NAME stink2" Files: sm-store-test1.root rs-1ev-test-031106-1810.root rs-1ev-test-031106-1840.root rs-1ev-test-031106-1844.root rs-1ev-test-031106-1844-1.root rs-1ev-test-031106-1845.root rs-1ev-test-031106-1845-1.root rs-1ev-test-031106-1847.root rs-1ev-test-031106-1855.root rs-1ev-test-031106-1910.root File Count: 10 Average File Size: 300 Total File Size: 3000 Total Event Count: 20If the result is not what was expected, then the definition can be redone and the previous definition will be overwritten.
After you are satisfied, you can either run a project or use:
sam create dataset --defname=stink2This ensures that your defintion cannot be changed.
First, a cdf dataset has a more modern (especially in Grid) concept of a "data collection". That is a group of files that are common in their properties. For our implementation of SAM in CDF, we have maintained this as a parameter although more sophisticated ways of handling this are being hammered out.
A SAM dataset definition corresponds to a selection of files meeting some criteria set by a variety of parameters that describe them based on the declarations made in the metadata.
A very simple way through the morass is to use the parameter cdf.dataset in defining a sam dataset and that is the end of the story. Using more sophisticated combinations of parameters requires care that one has specified the collection of files uniquely. Tools exist to allow you to examine files you care to inspect, but this is indeed a complex operation.
Once a dataset definition is made, it can be used to specify the files to be delivered to a project. When that delivery has been done, sam keeps permanent record of the project that was run and it is possible to always go back to find out what files were used. This is called a "dataset" within the context of sam or a "project snapshot" within the context of sam.
When a dataset definition is made, it is possible to immediately take a snapshot of the files that satisfy the requirements specified in the definition with "sam create dataset". Once this is done the definition is frozen. This is useful if you want to make sure that your definition is not modified - by someone else!