Sudhir Malik - February 2009
February 27
February 26
February 25
Craft Dataset for both Data and MC: use CMSSW_2_2_5
for MC:
there is a TrackerPointing skim available, the dataset is:
/TkAlCosmics4T/Summer08_COSMMC_21X_V1_v1/RECO
When using MC a different global tag should be used: COSMMC_21X_V1
The data location is (also at FNAL) here
for DATA:
The following is not in dbs ( I could not find)
/Cosmics/Commissioning08_CRAFT_ALL_V9_225_ReReco_FromTrackerPointing_v2/RECO
Ntuples are available for this sample (using V00-00-06 of
DPGAnalysis/SiPixelNtuplizer). They have been processed and will be copied to our castor area here:
/castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer
What I could find is one with "v1". It is here
The skim below is now at Fermilab
/Cosmics/Commissioning08_CRAFT_ALL_V9_SuperPointing_225-v3/RAW-RECO
and is here
Summer08 samples are here
Winter09 samples are here
CRAFT data is here
CRUZET4 data is here
February 24
While running this script as a condor job, I ran into an issue.
The condor log files says
000 (101282.000.000) 02/24 12:01:40 Job submitted from host: <131.225.205.246:32993>
...
001 (101282.000.000) 02/24 12:02:19 Job executing on host: <131.225.190.21:32973>
...
006 (101282.000.000) 02/24 12:03:27 Image size of job updated: 321592
...
006 (101282.000.000) 02/24 12:04:27 Image size of job updated: 421980
...
006 (101282.000.000) 02/24 12:05:27 Image size of job updated: 1391088
...
006 (101282.000.000) 02/24 12:06:27 Image size of job updated: 2374412
...
009 (101282.000.000) 02/24 12:07:27 Job was aborted by the user.
The system macro SYSTEM_PERIODIC_REMOVE expression '(JobRunCount > 10 || ImageSize > 3000000 )' evaluated to TRUE
So the job just hangs, there is no feedback email about any problem. When a job finishes, I get an email about it but not in this
case. The only difference between running and hanging is coming from running on all files instead of a few. This is vague
but it hangs around sometimes after ~5K, other times ~7K events. Since I had to run over 100K events, I split the
job into 15 parts. The idea is to then add the output root files. The root files actually are not big. For 7K events it is just 0.5Mb.
I am not sure if there is memory leak.
Charles gave a python snippet which I can include in my config files. It bascially runs in this way. Suppose I have
1000 files to run over. I have their names as 1000 lines in a text file. I can these into say 20 sections of 50 files each.
I can then run the job 20 times. Each time I change the section number I am running over.
Here it is
import FileUtils
numSections = 20
currentSection=1 # will be 2, then 3 .. 20
myLongList = FileUtils.loadListFromFile ('text.file')
myShortList = FileUtils.sectionNofTotal (myLongList,
currentSection,
numSections)
myList = FileUtls.sectionNofTotal( FileUtils.loadListFromFile ('tetxt'),
currentSection,
numSections)
And then submit the job. The next time, change :
currentSection=2
and resubmit the job, etc.
February 23
I had a chat with Benedikt today. Showed him slides with what I did and questions I
had. Also had chat with Adam Everett at Purdue about it.
February 20
In a meeting and then lunch
February 19
I was able to put on selectedLayer1Muons. But later I learnt from benedikt that I has to put
the cut before selection
Putting cut after selection
I made a tag in the RecoMuonValidator.cc file as follows
P_cut =pset.getUntrackedParameter("P_cut");
and use it like this:
===========================================
// Analyzer reco::Muon
for(View::const_iterator iMuon = muonColl.begin();
iMuon != muonColl.end(); ++iMuon) {
int trkNTrackerHits = 0, glbNTrackerHits = 0;
int staNMuonHits = 0, glbNMuonHits = 0;
float p=iMuon->p();
if(p > P_cut)
cout<< "MOMENTUM=" << iMuon->p() << endl;
=================================
by defining P_cut in the ../python/RecoMuonValidator_cfi.py
as below
==============================
# muonLabel = cms.InputTag("muons"),
muonLabel = cms.InputTag("selectedLayer1Muons"),
P_cut = cms.untracked.double(20.0), #momentum P >20
===============================
Putting cut before selection
I did the following in RecoMuonValidator_cfi.py
======================
from PhysicsTools.PatAlgos.selectionLayer1.muonSelector_cfi import selectedLayer1Muons
selectedLayer1Muons.cut = cms.string('pt > 10. & abs(eta) < 2.4')
recoMuonValidator = cms.EDFilter("RecoMuonValidator",
MuonServiceProxy,
tpSelector = muonTPSet,
simLabel = cms.InputTag("mergedtruth","MergedTrackTruth"),
trkMuLabel = cms.InputTag("generalTracks"),
staMuLabel = cms.InputTag("standalonemuons:UpdatedAtVtx"),
glbMuLabel = cms.InputTag("globalMuons"),
# muonLabel = cms.InputTag("muons"),
muonLabel = cms.InputTag("selectedLayer1Muons"),
==========================
February 18
Updated the StarterKit twiki with modified python script that works with cosmics data.
See the twiki (version 42)
February 17
Updated CMS Google Calendars. They are all here. Liz added them to her google calendars. Seh found ouut that
to add them you have to let the other person know its ICAL address and and to view them on web you need HTML address.
Here they are
LPCUserSupportPhysics
LPCUserSupportComputing
LPCUserSupportOffline
Miscellaneous meetings
But in keep in mind
We are 7 hours behind CERN time EXCEPT during "From 8 March- 29 March (6 hours behind)" and "25 Oct -1 Nov (8 hours behind)"
My info is based on the day light saving time change in Europe and US as follows
Europe
----------
Start: Last Sunday in March at 1 am UTC
End: Last Sunday in October at 1 am UTC
U.S. and Canada beginning in 2007:
----------------------------------------------
Start: Second Sunday in March
End: First Sunday in November
February 16
WISDOM TOOTH EXTRACTION
February 13
Working on CMS Google calendars
My PhEDEx requests was done:
The pixel cosmics CRAFT DATA is here
at FNAL.
Andreiy's request for Summer 08 Ztautau and Zmumu are here
For my comments yesterday benedikt suggested that I apply some cut to see if
selectedLayer1Muons
are really being selected.
Shall do that next week.
February 12
Working on PAT muon validation
To Benedikt,
So this is what I did so far.
1. In ../python/RecoMuonValidator_cfi.py, I commented/uncommented as below
simLabel = cms.InputTag("mergedtruth","MergedTrackTruth"),
trkMuLabel = cms.InputTag("generalTracks"),
staMuLabel = cms.InputTag("standalonemuons:UpdatedAtVtx"),
glbMuLabel = cms.InputTag("globalMuons"),
# muonLabel = cms.InputTag("muons"),
muonLabel = cms.InputTag("selectedLayer1Muons"),
When I just made this change, in the final DQM_V0001_R000000001__GlobalValidation__Test__RECO.root file, I got empty histograms(like NMuons, NSim, SimEta etc.)
in RecoMuon_MuonAssoc/Muon folder
This means that I did not get "muons" unlike before but I also did not get "selectedLayer1Muons". I got nothing.
2. Then I also added the following lines in the main config file which is located here.
http://cmslxr.fnal.gov/lxr/source/Validation/RecoMuon/test/muonValidation_cfg.py
In this ( in aditin to step 1) I load the PAT layers 0+1 and processed/added thesepath as follows in this main config file.
# PAT Layer 0+1
process.load("PhysicsTools.PatAlgos.patLayer0_cff")
process.load("PhysicsTools.PatAlgos.patLayer1_cff")
process.p=cms.Path(
process.patLayer0
+ process.patLayer1)
process.muonValidation_step = cms.Path(cms.SequencePlaceholder("mix")+process.recoMuonValidation)
process.schedule = cms.Schedule(process.p,
process.raw2digi_step,
# process.validation_step,
process.muonValidation_step,
process.MEtoEDMConverter_step,process.outpath)
After doing this I got all the plots like before and they looked identical to "muons" as I was seeing originally.
Does it mean I am now seeing "selectedLayer1" muons instead of "muons"
How do I be sure?
February 11
Working on PAT muon validation. Wrote email to benedikt
Hi! Benedikt,
For pat MuonValidation, I am trying to do a very simple thing but have problem.
In the MuonValidation there are several pieces of code. After lot of digging, I thought
of plugging pat::Muon in RecoMuonValidator.cc. I included the following header below to define pat
#include "DataFormats/PatCandidates/interface/Muon.h"
and also add pat::Muons collection and commented out the muons it was originally asking for
The original code is at
http://cmslxr.fnal.gov/lxr/source/Validation/RecoMuon/src/RecoMuonValidator.cc
and the piece that I changed is below
However, I get the error when compiling that I have cut & pasted at the bottom of this email.
What should I do??
I have seen that there is this kind of line is some analyzers
void RecoMuonValidator::analyze(const edm::Event& iEvent, const edm::EventSetup& iSetup)
but here it is different. I am not sure what is the difference between "iEvent" above and "event" in the snippet below.
Thanks.
regards,
Sudhir
CODE SNIPPET
====================================
void RecoMuonValidator::analyze(const Event& event, const EventSetup& eventSetup
{
if ( ! theDQM ) {
LogError("RecoMuonValidator") << "DQMService not initialized\n";
return;
}
// Get TrackingParticles
Handle simHandle;
event.getByLabel(simLabel_, simHandle);
const TrackingParticleCollection simColl = *(simHandle.product());
// Get Muon Tracks
Handle > trkMuHandle;
event.getByLabel(trkMuLabel_, trkMuHandle);
View
Benedikts reply:
And one addition - why do you want to use a edm::View ? You can read a collection of PAT muons already by using View. Which is the whole purpose
of using Views.
Cheers,
Benedikt
Liz's reply
There is no difference between iEvent and event. These are formal argument names (this is a fundamental software language concept, google it) and indeed the
compiler error message "In member function `virtual void RecoMuonValidator::analyze(const edm::Event&, const edm::EventSetup&)':" deletes them. The error is
probably due to using an iterator as a pointer which only accidentally works sometimes. The original author is also making copies for no good reason. This is not
a shining example of code quality. You need to take a different strategy or get help from someone who knows C++ to help fix this before you reuse it.
February 10
Worked on the problem that Yvonne and Andrius (ePAT participants) had.
The problem was with Module 8
The problem is
Hello!
I have a problem with the example of the homework. I just copy and pasted the good-muon-selecter and though the selectedLayer1Muons do exist (checked with
EventContentAnalyser), the modul cant find them.
The error-message and Source-Code is given below. It is a really simple structure. I was thinking a lot about it but cant find my mistake.
Thanks!
Yvonne
error-message:
%MSG
Begin processing the 10th record. Run 1, Event 560, LumiSection 666677 at 07-Feb-2009 10:29:38 CET
%MSG-w ScheduleExecutionFailure: PostModule 07-Feb-2009 10:29:38 CET Run: 1 Event: 560
an exception occurred and all paths for the event are being skipped:
---- ScheduleExecutionFailure BEGIN
ProcessingStopped
---- ProductNotFound BEGIN
getByLabel: Found zero products matching all criteria
Looking for type: edm::OwnVector >
Looking for module label: selectedLayer1Muons
Looking for productInstanceName:
cms::Exception going through module CandSelector/goodMuons run: 1 event: 560
---- ProductNotFound END
Exception going through path g
---- ScheduleExecutionFailure END
Solution was
saw yours and Andrius' posting. I am not sure if the example you are following is right. Well it is on the twiki, so I don't blame you. First of all the you
should make sure that the root file you run on has *_selectedlayer1Muons_*_* objects or not. If you were to open your root file below you will see that it does
not have anything listed as selectedLayer1. So your script will keep complaining about selectedLayer1Muons
/store/relval/CMSSW_2_2_0/RelValTTbar/GEN-SIM-RECO/IDEAL_V9_v3/0000/0A99230C-20BA-DD11-B5B2-000423D99614.root'
I am attaching a script that does skimming. You can try it. I am runnig it at Fermilab cmslpc from the directory ../PatAlgos/test/.. area.
I have copied the script (MyMuonSelctor.py) and the root file (PatAnalyzerSkeletonSkim.root) and the output root file ("outputfile.root") to the area below
/afs/cern.ch/user/m/malik/scratch0/
1. The script runs on "PatAnalyzerSkeletonSkim.root" and produces another skimmed file called "outputfile.root" which has your selctions like
_selectedLayer1Muons_ePAT ( ePAT comes from process = cms.Process("ePAT") in teh scripts)
It is able to run because PatAnalyzerSkeletonSkim.root has selctedLayer1 information
You will see that I have other branches also that I keep in the scripts like
outputCommands = cms.untracked.vstring('drop *',
'keep *_selectedLayer1Muons_ePAT*_*',
'keep *_selectedLayer1Electrons_ePAT*_*',
the user response was:
On Feb 11, 2009, at 7:26 AM, Using Physics Analysis Toolkit (PAT) in your analysis wrote:
Dear Sudhir,
thanks for your example. I just changed in my file the "CandidateSlector " into your "PATMuonSelector" and everythings works.
The SelectedLayer1-Particles exist, because I run PatLayer1 over my initial Root-File. So this was not the problem.
I think there is something wrong with the Candidate Selector given in the TWiki.
Anyway, I will use your modul and can now finish my exercise 8.
Thanks a lot for your help!
Regards, Yvonne
February 09
The StarterKit in CMSSW_2_2_3 has a script called cosmics_cfg.py which is suppose to run
on CRUZET data. When I run this scripts I get errors. I summarised them in email to hypernews here. I tried to remove the modules it complaint but gave up for today and it did
not work.
Giovanni replied this
Sal says
We don't really have any clients for this at the moment. There are
more developed ways of looking at cosmics that are more appropriate.
This was done at the request of some people, and they are no longer
using it. So I think it's safe to drop it.
February 06
Helped user with the folllowing PhEDEx Request
Request 48348 : Transfer Request :
Dataset /Ztautau/Summer08_IDEAL_V11_redigi_v1/GEN-SIM-RECO 72 88.92 GB
Dataset /Zmumu/Summer08_IDEAL_V11_redigi_v1/GEN-SIM-RECO 148 177.20 GB
220 266.12 GB
T3_US_FNALLPC : pending approval
Helped user with his ROOT macro to plot pixel hits in disk and barrel
Another user wanted to know where to dump his big *.root files. He was unable to write them
to
/uscms_data/d2/lpcll
. Told him an alternative /uscmst1b_scratch/lpc1/3DayLifetime
February 05
A user at Texas Tech is having the following problem (not related with the StarterKit) itself
The problem is :
I would like to learn PAT on the TTU computer.
I participated in JTERM III at Fermi Lab. I could follow the
WorkBookAnalysisStarterKit (https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookAnalysisStarterKit)
on the cmslpc without any error.
However during this week I tried to do the same thing again on TTU
computer with CMSSW 2_2_3 it did not work .
This the error message I got when I did this command "cmsRun
PatAnalyzerSkeleton_cfg.py >& skoutput.txt &".
WARNING: to run on Summer08AODSIM from 2.2.X requirs to drop Particl eFlow,
so PAT will switch from PFTau to CaloTau
Replaced tauTrigMatchHLT1Tau.src: cms.InputTag("allLayer0Taus") =>
cms.InputTag( "allLayer0CaloTaus")
Replaced tauTrigMatchHLTLooseIsoTauMET30L1MET.src:
cms.InputTag("allLayer0Taus") => cms.InputTag("allLayer0CaloTaus")
Replaced tauTrigMatchHLTDoubleIsoTauTrk3.src:
cms.InputTag("allLayer0Taus") => c ms.InputTag("allLayer0CaloTaus")
WARNING: to run on Summer08AODSIM from 2.2.X requirs to drop ParticleFlow,
so PAT will switch from PFTau to CaloTau
CORAL/RelationalPlugins/sqlite Error SQLiteStatement::prepare 5
database is l ocked
CORAL/RelationalPlugins/sqlite Error SQLiteStatement::fetchNext 21
database i s locked
%MSG-s CMSException: 03-Feb-2009 16:01:24 CST pre-events
cms::Exception caught in cmsRun
---- Configuration BEGIN
Error occured while creating PoolDBESSource
---- Conditions BEGIN
PoolDBESSource::fillrecordToIOVInfo: tag CSCBadChambers_realCal has
empty iov to ken meaning requested tag or data do not exist
---- Conditions END
---- Configuration END
%MSG
-bash-3.00$
[1]- Exit 73 cmsRun PatAnalyzerSkeleton_cfg.py >&skoutput.txt
What should I do to get it work?
How should I start studying the PAT?
Thank you
After digging around I found a following response to a similar problem ( but it does not help the user). So I am not sure what to do.
You're getting conditions out of an SQLite database that is installed
in the software installation area. SQLite uses file locks to mediate
read/write access to the SQLite database (even if it is only going to
be read), and file locks are notorious for having various failure
modes over distributed file systems. I'd guess that the most common
failure mode is using NFS to cross-mount the filesystem and not
enabling NFS locking (most Linux distributions seem to have it off by
default, even when NFS is turned on). You can
check with the command
/sbin/chkconfig --list nfslock
which should print something like
nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off
If those are all "off" on the system you're running on, then that's
probably
the problem. In that case, someone with root access on that system
needs to enable and start the nfslock service.
While googling I found that people suggested(not sure what they imply though
---The preferred workaround for this problem is to copy the DB Release tarball to the job work directory
----Either move your working directory to /tmp or into AFS space. On the batch system just go to the local working directory using the $TMPDIR
variable.
February 04
Reviewed documentation for Module 8.
A user running Module 6 had problem with his analyzer.
He was unable to read "selectedLayer1" pat objects.
After lot of digging around the mistake being made
was very simple. The order of his path in config file should
have been patLayer0->patLayer1->his analyzer where as he
had the patLayer1->his analyzer switched around.
February 03
Got following responses from Adam Everett and Colin Bernet regarding PAT validation
Adam Everett wrote:
My first question is: what is the purpose of pat::muon validation? Is it to compare pat::muons to reco::muons? To compare pat::muons to TrackingParticles /
MCTruth? or to make some representative pat::muon distributions as some sort of DQM job?
I am not as familiar with PAT as I should be, but I suspect that you will not need to make any new validation modules if you want to examine the same histograms as
we use for offline. I think that the object inheritance will allow you to use MultiTrackValidator or RecoMuonValidator by simply changing the configuration files.
During the creation of the PAT-tuple, you should be able to do the validation module and be sure to "keep *_MEtoEDMConverter_*_*" to the output file. Then the
harvesting and postValidation step is trivial.
Colin Berett:
Colin Bernet wrote:
Thanks, I'll be very happy to contact you, certainly at the beginning of
next week.
In the meanwhile, I propose that you have a look at the validation
system of the particle flow. I think it is more or less representative,
and the documentation exists.
https://twiki.cern.ch/twiki/bin/view/CMS/SWGuidePFValidation
Very briefly, I think that we should reuse the validation system of the
POGs, and just develop new validation tools to check what is specific to
PAT:
- selections
- matching to gen, and to trigger
February 02
Finished work on the twiki above twiki to CMSSW_2_2_3 WorkBookWriteFrameworkModule.
Added a histogram to the Demoanalyzer iteself. Now it plots a distribution of number of tracks per event
Also a user having problem with his FWLite macro was solved. We were blaming it to problems of compiling it in CINT (initially)
whereas the later problem was with the macro itself. I looked carefully at his macro. I found missing
#TH2.h
and
#include "DataFormats/TrackReco/interface/Track.h"
in his header file.
and
fwlite::Handle > h_track;
h_track.getByLabel(ev,"generalTracks");
Actually there were data format problem running it earlier since the root file was not included while opeing root (as suggested by FWLite experience)
which I had fixed. But realised after carefully looking at the error message that now the problems were just with the macro itself.
The macro that now actually here
Earlier Chris Jones had suggested
ROOT can actually compile it for you. Instead of doing
.x sk_fwlite_MyonPtPxPyPzEtaPhi.C
do
.x sk_fwlite_MyonPtPxPyPzEtaPhi.C++
The only caveat is you should open your .root file before loading the macro since that will bring the necessary libraries into
your ROOT session and the the .x ***.C++ command will link those loaded libraries to the code that ROOT compiles.
Hope that helps,
Chris
and
Strictly speaking, it appears that your problem is actually in ROOT
code and not actually in CMS code. This is because you are running
your code under CINT and CINT just uses the dictionaries for CMS
objects and those dictionaries are generated via ROOT code. One
possible way around this could be to compile your code instead of
using the CINT interpreter since CINT is known to have 'issues'.
Charles had suggested
Using uncompiled Root code (a.k.a. Cint, C+, ...) is generally evil* and should be avoided for all but the simplest of
tasks. :-)
* http://www.parashift.com/c++-faq-lite/big-picture.html#faq-6.15
Another user had the following question:
I'm fighting with Module 6 Exercise 1.
In patLayer1_fromAOD_full.cfg.py from PhysicsTools/PatAlgos/test
(they refer to it as configuration file for standard PAT Layer1 production) what InputTag I should use for electronTag and
jetTag?
For example in previous module 5 I used
process.verySimplePATAnalysis = cms.EDFilter("VerySimplePATAnalysis",
electronTag = cms.untracked.InputTag("selectedLayer1Electrons"),
jetTag = cms.untracked.InputTag("selectedLayer1Jets")
)
here, in module 6 I'm going to use
process.verySimplePATAnalysis = cms.EDAnalyzer("VerySimplePATAnalysis",
electronTag = cms.untracked.InputTag("?????????????????"),
jetTag = cms.untracked.InputTag("?????????????????")
)
There was a question from ePAT trainee and few other people to get run and event info running FWLite macro
The snippet that does it is as follows
fwlite::Event ev(file);
for( ev.toBegin();
! ev.atEnd();
++ev) {
// cout << "Event Number and Run Number =" << ev.id() << endl;
// cout << "Run Number =" << ev.id().run() << endl;
cout << "Event Number =" << ev.id().event() << "=" << m <
Go to January's's log
Last modified: Tue Nov 4 11:07:59 CST 2008