MarcoWR0814

Monday

• FDR phone meeting (phone EsNet)
• UofC ATLAS meeting (3rd HEP)

Tuesday

• MWT2 meeting (Robert's office)
• Pathena code exploration: trf runAthena is the ultimate responsible for file transfer and is probably downloaded from BNL

Wednesday

• ATLAS TAG DB meeting (phone CERN)
• ATLAS facilities meeting (phone EsNet)
• Documents about TAGs:
  • https://twiki.cern.ch/twiki/bin/view/Atlas/ELSSIDevelopersManual
  • https://twiki.cern.ch/twiki/bin/view/Atlas/AutomaticTagLoading
  • https://twiki.cern.ch/twiki/bin/view/Atlas/AtlasEventTagTools
  • https://twiki.cern.ch/twiki/bin/view/Atlas/TagForEventSelection
  • https://twiki.cern.ch/twiki/bin/view/Atlas/TagNavigatorTool
  • https://twiki.cern.ch/twiki/bin/view/Atlas/ELSSIDevelopersManual
  • https://twiki.cern.ch/twiki/bin/view/Atlas/PHPDevelopersNotes
  • https://twiki.cern.ch/twiki/bin/view/Atlas/TagNavigatorToolGangaPlugin
• TODO:
  • list of files processed (total/successful/failed)
  • with the collection
  • filter on the guid:
    • show me the tag but filter on this catalog

• POOL utilities:
  • work at CERN
  • nightly
  • Jack's setup: ~/maxilist/LCG/POOL_2_7_4/setup274.sh (to source)
  • /afs/cern.ch/sw/lcg/app/releases
  • links:
    • http://pool.cern.ch/
    • http://lcgapp.cern.ch/lxr/POOL/source/
    • http://pool.cvs.cern.ch/cgi-bin/pool.cgi/pool/?search=None&hideattic=1#dirlist

why pool_insertFileToCatalog?
pool_insertFileToCatalog reads the giud from the file

[lxplus218] /afs/cern.ch/user/m/mambelli/prog/LCG/POOL_2_7_4/src > cd Utilities/cmt/[lxplus218] /afs/cern.ch/user/m/mambelli/prog/LCG/POOL_2_7_4/src/Utilities/cmt > cmt configCreating setup scripts.Creating cleanup scripts.[lxplus218] /afs/cern.ch/user/m/mambelli/prog/LCG/POOL_2_7_4/src/Utilities/cmt > . ./setup.sh

POOL_2_7-patches compiles, later releases do not

-query support eventnumber== ...
There is no way to query by GUID
To select multiple events, a OR list should be use d

lxplus218] /afs/cern.ch/user/c/cranshaw/maxidisk/LCG > ls
cmtnotes  dev1  POOL_2_4_3  POOL_2_4_3-ref  POOL-2-6-2  POOL_2_6_3  POOL_2_7_0  POOL_2_7_4
[lxplus218] /afs/cern.ch/user/c/cranshaw/maxidisk/LCG > cd POOL_2_7_4
[lxplus218] /afs/cern.ch/user/c/cranshaw/maxidisk/LCG/POOL_2_7_4 > ls
include  setup274.sh  setup-nightly.sh  slc4_ia32_gcc34_dbg  src
[lxplus218] /afs/cern.ch/user/c/cranshaw/maxidisk/LCG/POOL_2_7_4 > 

• TODO
  • write utility to play with GUIDs (filter, ...)

For future reference, here's how I managed the server restart on atldbdev01:                                                                                              
                                                                                                                                                                          
ssh mkenyon@lxplus.cern.ch                                                                                                                                                
ssh tagexsrv@atldbdev01 (Florbela and I have the password, if you need it)                                                                                                
cd ~/public/TagExtraction                                                                                                                                                 
./TagServices.sh restart                                                                                                                                                  
                                                                                                                                                                          
                                                                                                                                                                          
So, what you have to remember here is that, although we're restarting a service on atldbdev01, the files are all located at                                               
/afs/cern.ch/user/t/tagexsrv/publicTagExtraction. It's a bit confusing, because there are a similar set of files at                                                       
atldbdev01:/home/tagbrowser/public/TagExtraction.                                                                                                                         
                                                                                                                                                                                                        
                                                                                                                                                                                                        
Thanks, Mike. ?I note now that Caitriana's documentation actually _says_ this (not in the same place as the instructions for restarting the lxgate instance,                                            
but still...):                                                                                                                                                                                          
                                                                                                                                                                                                        
      7. The services can then be run using the TagServices.sh script, found in ~/public/TagExtraction. This is a basic start/stop/restart script which                                                 
      is used as:                                                                                                                                                                                       
                                                                                                                                                                                                        
./TagServices.sh start | stop | restart                                                                                                                                                                 

Thursday

• GLUE schema discussion about resources (Burt, Xin)
• Force Pathena to perform skim without shared file system
  • SUSY validation has input dataset but no local files (-inDS, no -shipInputs), e.g. PathenaSessionTry3 : all files (except jobOption) are in dataset. Check with Robert
  • 2 event/2files skim: http://cern.ch/test-service-tags-output/fdr08_run1_MinBias_o1_r12_t1_1207336538_user.root
    • fdr08_run1_MinBias_o1_r12_t1 and RunNumber>=3070 and RunNumber<=3079 and NLooseElectron>1

Friday

• cont UIUC-HEP registration
• cont force Pathena to perform skim without shared file system
  • one of the files has wrong registration in MWT2_UC: E84139A5-57DA-DC11-956D-000423D990FC (fdr08_run1.0003073.MinBias.merge.AOD.o1_r12_t1._0001.1) has no checksum (NULL) instead of ad:1b609040
  • Modify dq2_put (dq2_put.marco_dev) to work with adler32 and to work around this null checksum. This version is not general and ready to be submitted to Tadashi
  • Register DS user.MarcoMambelli.test.skim_2evAODs.080401._12
  • Pathena (inDS+shipInput) failing

[uct3-edge3] /home/marco/tmp1 > dq2_put.marco_dev -v -e -p PFC_2ev.xml user.MarcoMambelli.test.skim_2evAODs.080401._12
curl --user-agent "dqcurl" --silent --insecure --config 7e1d2b76-0325-4e0e-9c44-181e1b72c6ff http://uct2-grid1.uchicago.edu:8000/dq2/lrc/PoolFileCatalog
data="lfns=fdr08_run1.0003073.MinBias.merge.AOD.o1_r12_t1._0001.1+fdr08_run1.0003070.MinBias.merge.AOD.o1_r12_t1._0001.1"
(0, '\n\n\n\n\n  \n\n  \n\n  \n\n  \n\n  \n\n  \n    \n      \n    \n    \n      \n    \n    \n    \n    \n    \n    \n  \n\n  \n    \n      \n    \n    \n      \n    \n    \n    \n    \n    \n    \n  \n\n')
PAR:  True ['5E450E93-57DA-DC11-949A-000423D99C36', 'E84139A5-57DA-DC11-956D-000423D990FC'] ['40179453', '49597234'] ['ad:1b609040', 'ad:a546328c'] ['5E450E93-57DA-DC11-949A-000423D99C36', 'E84139A5-57DA-DC11-956D-000423D990FC'] ['40179453', '49597234'] ['ad:NULL', 'ad:a546328c']
2 files in dataset, 0 not in LRC
Registering 2 files as user.MarcoMambelli.test.skim_2evAODs.080401._12

[uct3-edge5] /ecache/marco/wdir_pathena/tag_pathena_mwt2 > pathena -v --inputFileList=filelist.txt --shipInput --inDS=user.MarcoMambelli.test.skim_2evAODs.080401._12 --outDS=user.MarcoMambelli.test.tag.080404._01 AnalysisSkeleton_topOptions.py
...
ANALY_BNL_ATLAS_1 holds 2 files
ERROR : No (new) files available in user.MarcoMambelli.test.skim_2evAODs.080401._12
        pathena runs on files which were failed or were not used in
        previous submissions if it runs with the same inDS and outDS