DataManageRwg

Only crap that works. Hostname left in some places for context.

Setups

   rwg@uct2-grid7:~$ source /share/wlcg-client/setup.sh 
   rwg@uct2-grid7:~$ source /share/dq2-client/setup.sh 
   rwg@uct2-grid7:~$ voms-proxy-init -voms atlas:/atlas/usatlas/Role=production -hours 300

  • Set the local site ID, eg. export DQ2_LOCAL_SITE_ID=MWT2_UC_GROUPDISK (GROUPDISK deprecated; see Tiers of ATLAS)
  • Use, eg., MWT2_UC_USERDISK

List all my datasets

dq2-ls -L MWT2_UC_USERDISK user.RobertWGardnerJr.*

List files in a specific dataset

rwg@uct3-edge5:~$ dq2-ls -L MWT2_UC -f user.RobertWGardnerJr.AnalyEx-B9

user.RobertWGardnerJr.AnalyEx-B9
[X]	user.RobertWGardnerJr.AnalyEx-B9.hist._00001.root	eaa6863b-4dae-41af-86a2-eacc6486a22a	md5:1f05e43c7ffb81808d91cc043bdbbb9b	234318
[X]	user.RobertWGardnerJr.AnalyEx-B9._12704374.log.tgz	459a7d77-f918-4238-a0a0-19821b98a9c7	md5:75f8d084cec71a3f5d527418f8756117	433152
total files: 2
local files: 2
total size: 667470
date: 2008-06-17 22:42:09
rwg@uct3-edge5:~$ 

List all datasets at a site

rwg@uct3-edge5:~$ dq2-list-dataset-site UCT3
fdr08_run2.0052280.physics_Egamma.merge.AOD.o3_f8_m10
fdr08_run2.0052280.physics_Minbias.merge.AOD.o3_f8_m10
fdr08_run2.0052280.physics_Muon.recon.ESD.o3_f8
fdr08_run2.0052283.physics_Egamma.merge.AOD.o3_f8_m10
fdr08_run2.0052280.physics_Jet.merge.AOD.o3_f8_m10
csc11.005061.Alpgen_JimmyQCD3ExclJets.evgen.EVNT.v11004206
fdr08_run2.0052280.physics_Bphys.merge.AOD.o3_f8_m10
fdr08_run2.0052280.physics_Jet.recon.ESD.o3_f8
fdr08_run2.0052280.physics_Muon.merge.AOD.o3_f8_m10
fdr08_run2.0052280.physics_Jet.merge.DPD_SUSY.o3_f8_m11
user.HironoriIto.Dq2Ping.0_00

List all subscriptions for MWT2_UC

dq2-list-subscription-site MWT2_UC

UCT3 endpoint operations

  • dq2-register-subscription fdr08_run2.0052280.physics_Minbias.merge.AOD.o3_f8_m10 UCT3
    • files appearing in /pnfs/uct3/atlas/fdr08_run2/AOD/fdr08_run2.0052280.physics_Minbias.merge.AOD.o3_f8_m10/
  • dq2-list-subscription-site UCT3
  • dq2-list-subscription-info fdr08_run2.0052280.physics_Minbias.merge.AOD.o3_f8_m10 UCT3

Dataset subscription request interface

Deleting datasets

Check for datasets at BNL

Example:

dq2-list-dataset-site BNL-OSG2_DATADISK

Where are the replicas?

Example:

dq2-ls -L MWT2_UC  -r ccrc08_run2.037104.physics_Bphys.merge.AOD.o0_r0_t0

Or, this works and doesn't need the -L parameter, eg:

dq2-list-dataset-replicas csc11.005200.T1_McAtNlo_Jimmy.recon.AOD.v11004204

References


-- RobGardner - 07 Aug 2008
Topic revision: r11 - 17 May 2011, RobGardner
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback