TestSkim080310

Intro

Test with FDR data

Directory /local/workdir/mm-s2/ and subdirectories

Links

To submit errors: https://savannah.cern.ch/bugs/?func=additem&group=panda

To see all Pathena jobs submitted by Marco Mambelli: http://gridui02.usatlas.bnl.gov:25880/server/pandamon/query?ui=user&name=Marco%20Mambelli

Setup

I followed setup instructions from https://twiki.cern.ch/twiki/bin/view/Atlas/UseAtlasSoftwareProjectsKit

[marco@tier2-06 mm-s2]$ . /local/inst/pandajs/setup.sh
[marco@tier2-06 mm-s2]$ source /share/app/atlas_app/atlas_rel/13.0.40/cmtsite/setup.sh -tag=13.0.40.2,AtlasOffline
[marco@tier2-06 mm-s2]$ source /share/app/atlas_app/atlas_rel/13.0.40/AtlasOffline/13.0.40/AtlasOfflineRunTime/cmt/setup.sh

Event collection query and preparation

I executed several queries. Some streams listed in the browser are not valid and I receive several Oracle "Unknown error":

• "Data quality" is not working
• "Temporal cut" may not work
• fdr08_run1_StreamJet_o1_r12_t2 (279685 events) produces queries that work when counting the events but fail when trying to retrieve them

Queries on fdr08_run1_MinBias_o1_r12_t1 with no luminosity or time constrain work. I did 3 queries (NLooseElectron>0/1/2) obtaining 21/2/0 events and retrieved the collections with wget

The collection you are querying is: fdr08_run1_MinBias_o1_r12_t1

Your query is : SELECT count(*) FROM fdr08run1MinBiaso1r12t1 WHERE NLooseElectron>0

...
> wget http://cern.ch/test-service-tags-output/fdr08_run1_MinBias_o1_r12_t1_1205191685_user.root

Transforming the collection not to fail CollSplitByGUID.exe (CollCopy used for rel 12 is no more in rel 13, have to use CollAppend)

[marco@tier2-06 mm-s2]$ CollAppend.exe -src MinBias_query2 RootCollection -dst mycoll_query2 RootCollection -queryopt 'SELECT RunNumber, EventNumber'
...
>> Creating query for the source collection
pool::Exception: Column with name `SELECT' does NOT exist. ( POOL : "CollectionDescription::column" from "CollectionBase" )
[marco@tier2-06 mm-s2]$ CollAppend.exe -src MinBias_query2 RootCollection -dst mycoll_query2 RootCollection -queryopt 'RunNumber, EventNumber'
...
CollAppend: Finished appending input collection(s) `MinBias_query2:RootCollection' to output collection(s) `mycoll_query2:RootCollection' 
[marco@tier2-06 split1]$ ./CollSplitByGUID.exe -src mycoll_query2 RootCollection
./CollSplitByGUID.exe: error while loading shared libraries: liblcg_MultiCollection.so: cannot open shared object file: No such file or directory

Unable to split. The split script is not compatible with release 13 (missing library error)

Skim

Anyway trying to skim:

[marco@tier2-06 split1]$ CollListFileGUID.exe -src mycoll_query2 RootCollection 
Warning in : 
                     The StreamerInfo of class TBranch read from file mycoll_query2.root
                     has the same version (=10) as the active class but a different checksum.
                     You should update the version to ClassDef(TBranch,11).
                     Do not try to write objects with the current class definition,
                     the files will not be readable.

>> Creating query for the collection
>> Executing the query 
AE518AFE-B8DF-DC11-9588-0030487CD750

9A68D349-B8DF-DC11-A23A-0030487CBA0A

-----hanging ctrl+C necessary-----

[marco@tier2-06 split1]$ curl http://uct2-grid1.uchicago.edu:8000/dq2/lrc/PoolFileCatalog/?guids=AE518AFE-B8DF-DC11-9588-0030487CD750+9A68D349-B8DF-DC11-A23A-0030487CBA0A
Error. GUIDs not found[marco@tier2-06 split1]$ 

Setup pathena

[marco@tier2-06 pathena]$ export CMTPATH=`pwd`:${CMTPATH}
[marco@tier2-06 pathena]$ export CVSROOT=:ext:mambelli@atlas-sw.cern.ch:/atlascvs; export CVS_RSH=ssh; 
[marco@tier2-06 pathena]$ export PATHENA_GRID_SETUP_SH=/local/inst/pandajs/setup.sh 
[marco@tier2-06 pathena]$ cmt co PhysicsAnalysis/DistributedAnalysis/PandaTools;
# ================= working on package PandaTools version PandaTools-00-00-08 (At head)  path PhysicsAnalysis/DistributedAnalysis in /local/workdir/mm-s2/split1/pathena/PhysicsAnalysis/DistributedAnalysis/PandaTools
  # get top files 
cvs update: Updating .
Creating setup scripts.
Creating cleanup scripts.
[marco@tier2-06 pathena]$ cd PhysicsAnalysis/DistributedAnalysis/PandaTools/cmt
[marco@tier2-06 cmt]$ source setup.sh
[marco@tier2-06 cmt]$ make
...
installation done
------> (constituents.make) install_python_modules done
 all ok.
[marco@tier2-06 cmt]$ cd ../../../...
[marco@tier2-06 pathena]$ mkdir run; cd run
[marco@tier2-06 run]$ pathena --shipInput --inputFileList input_list --outDS user.MarcoMambelli.test080310 myJobOption13.py 
extracting run configuration
ConfigExtractor > Input=COLL
ConfigExtractor > InputFiles mycoll_query2.root 
ConfigExtractor > Output=AOD
ConfigExtractor > Output=META StreamAODFileMetaData StreamAOD
archive sources
archive InstallArea
post sources/jobO
Warning in : no dictionary for class pair is available
Warning in : 
                     The StreamerInfo of class TBranch read from file /local/workdir/mm-s2/split1/pathena/run/mycoll_query2.root
                     has the same version (=10) as the active class but a different checksum.
                     You should update the version to ClassDef(TBranch,11).
                     Do not try to write objects with the current class definition,
                     the files will not be readable.

Warning in : no dictionary for class AttributeListLayout is available
submit
===================
 JobID  : 19
 Status : 0
  > build
    PandaID=8286928
  > run
    PandaID=8286929

Job http://gridui05.usatlas.bnl.gov:25880/server/pandamon/query?job=8286929 failed with execute script: Running athena failed : 24832.

Examining the log file I'm suspecting a disk problem at BNL: https://savannah.cern.ch/bugs/?34424

Next step is local execution.

Out of the 4 GUID present for each event:
1. is unknown   
2. is a RAW file
3. is a AOD file belonging to multiple DS (one is the official merge)
4. is a ESD file

The CollAppend command keeps only the unknown GUID (is it the TAG?) in the
ouput file.

Using the "-queryopt StreamAOD_ref" option CollListFileGUID.exe lists
only the unknown GUID (TAG?) and the GUID of RAW file

In release 12 the behavior was quite different.

Cut Paste

. /osg/app/atlas_app/atlas_rel/13.0.40/cmtsite/setup.sh -tag=13.0.40.2,AtlasProduction
. /osg/app/atlas_app/atlas_rel/13.0.40/AtlasProduction/13.0.40.2/AtlasProductionRunTime/cmt/setup.sh
. /share/osg-client/setup.sh 

export CMTPATH=`pwd`:${CMTPATH}
export CVSROOT=:ext:mambelli@atlas-sw.cern.ch:/atlascvs; export CVS_RSH=ssh; 
export PATHENA_GRID_SETUP_SH=/share/osg-client/setup.sh
cmt co PhysicsAnalysis/DistributedAnalysis/PandaTools

cd PhysicsAnalysis/DistributedAnalysis/PandaTools/cmt
source setup.sh
make

pushd PhysicsAnalysis/DistributedAnalysis/PandaTools/cmt
source setup.sh
make
popd

cd ../../../..
mkdir run; cd run
pathena 

-- MarcoMambelli - 10 Mar 2008