How-to ... DQ2 at UofC (Subscriptions, ...)

The one in this page is an example of how to use DQ2 at UC. It is not aimed to substitute official and more complete DQ2 documentation that will explain how to use DQ2 and how to have your own installation of the client tools.

The official DQ2 pages are here: https://twiki.cern.ch/twiki/bin/view/Atlas/DistributedDataManagement and this is a user guide: https://twiki.cern.ch/twiki/bin/view/Atlas/UsingDQ2

Setup DQ2

To use DQ2 you can use the installation on tier2-06.

Before using it you have to execute the setup:

ssh tier2-06.uchicago.edu
...

cd /local/inst/pjs
. ./setup.sh
cd ../dq2
. ./setup.sh

Then you have to initiate your grid proxy (necessary if you authenticate with a server) and you are ready to go

How to Use DQ2 client

Now you can use DQ2 and here is a list of commands:

• To see a list of available commands in DQ2 type dq2.
• To subscribe a dataset to UC dCache VOBox type dq2  registerDatasetSubscription UC_VOB
• To view current datasets at UC VOBox type dq2 listSubscriptionsInSite UC_VOB
• To view current subscriptions type dq2 listDatasetsInSite UC_VOB

How to Use DQ2 Utils (end-user-tools)

dq2 utils are also available:

• E.g. cd /local/workdir;  dq2_get -r -d ./ csc11.005010.J1_pythia_jetjet.merge.TAG.v11004104 csc11.005010.J1_pythia_jetjet.merge.TAG.v11004104._00003.pool.root

Example: make your files available

You must have your file in a directory accessible by the local DQ2 server (NFS, dCache, ...). Then follow the following steps:

• Setup DQ2 as above
• Choose a name for the dataset. Normal users should follow the naming convention for user datasets, i.e., user..
• Register your files using dq2_put (dq2_put -h for help)
  • If you have a poolfilecatalog you can use something like = dq2_put -p PoolFileCatalog.xml user.MarcoMambelli.dataset.12345.v2=
  • If you dont and your files are all in one directory use something like dq2_put -d /pnfs/usatlas.bnl.gov/data/mambelli/tmp/ds1 user.MarcoMambelli..dataset.12345.v1
• Subscribe your dataset to the destination SE using a command like dq2  registerDatasetSubscription  CERN

Datasets at UC useful for TAG building

• Dataset Browser General link
• Release 12 datasets on UC_VOB (dcache)
  • 5711 AOD
  • 5711 ESD
• csc11 datasets on UC
  • 5010 AOD

• To see the directories with these files, one can do 'ls -l /pnfs/uchicago.edu/data/usatlas/testIdeal_07/' on tier2-06.

-- MarcoMambelli - 10 Aug 2006 -- JackCranshaw - 10 Oct 2006