Meeting November 1

Meeting

Wednesday 11/1 at 2.00pm Chicago time

Phone conference ESNet Call: 510-665-5437

When: November 1, 2006, 02:00 PM America/Chicago

Meeting ID: 22333

Agenda

• Files and DB access
• TAGs produced by Jerry
• registering files
• review 10/10 action items
• set this week action items

Notes

Attending: Ed, Jack, Marco, Robert

Discussion

In this section I tried to report the discussion. Feel free to correct or add to this.The most important is the action section.

MM: There is a dq2_put equivalent that allows to register a file in the local catalogs at Chicago.

JC: At Chicago there are multiple file catalogs. In Athena, multiple DBs can be federated (this is performed in the sw that Athena uses, not in the DB themselves): - is this something that we can use? have generic to Pool DB that presents the content of multiple file catalogs? In Pool, we could use it - is it possible to set up a service?

EM: The Panda interface of the catalog gives correct results (files are in Chicago and BNL), but dq2_get finds the files only at BNL and retrieves them from BNL. This is inefficient.

Local test work: some files were missing (in unit of digit files) for the AOD files that we have TAGS and the DB content - missing some event becsuse there were same missing input digit files (3/50) when the AOD were created. 39850ev

Tag DB filled by Jack new wiki page

PNT Tag navigation tools on Friday.

Actions

• update dq2usertools (Tadashi's modifications) (Marco)
  • I have tried unsuccesfully to copy files stored in dcache on tier2-06 using the dq2_get command. Not sure if this is a problem with the setup.sh file alone or some other problem with UC_VOB. Will continue to investigate with Ed May. - Jerry
• ask Tadashi how to access remotely files in dCache (or not local in general) (Marco)
• local test (copy files, create local PFC, check that you get the full file content, try a query - using the files, no DB) (Jack - Jerry)
• setup condor on tier2-06 (Marco)
• test running on the WNs using Condor (Jerry)
  • I have successfully tested using condorG to submit simple "helloworld" jobs to various OSG sites. Staging an executable worked fine as well as transferring back both an output and error file. The next step wil lbe to test a grid job to create a TAG output file using ATLAS Release 12.0.3. - Jerry
• test accessing files remotely (Jerry - Marco)
• preliminary version of Rob's test - see if feasible and what is involved (Jack)

Followup

Condor has been installed on tier2-06 in /opt/condor and is available. It allows submissions to the prototype tier2 cluster.

Discussion with Tadashi about dCache access:

• there is no special access to dCache. The pilot is the original one.
• For TAG analysis, pilot doesn't copy files to a local disk.
• Athena running at BNL directly reads files in BNL dCache via dcap. Athena (ROOT) might read files using gsidcap from another site, but I have never tried it.
  • The pilot copies only TAG files which have references to AOD/ESD.
  • AOD/ESD are in dCache and Athena reads them like PoolAODInput = ["dcache:/pnfs/usatlas.bnl.gov/data/ketevi/ttbar/11.0.41/aod/mc11.004100.AOD.pool"] *LYON people may be using gsidcap (grid access to dCache).

When dCache is not mounted locally:, PoolAODInput is: PoolAODInput = ["dcap://dcgftp.usatlas.bnl.gov:22125/pnfs/usatlas.bnl.gov/data/ketevi/ttbar/11.0.41/aod/mc11.004100.AOD.pool"]

The references in TAG files are GUIDs. PoolFileCatalog.xml is creted and referencing the dcap files:

  <File ID="8474BB36-B213-DA11-87E4-000E0C4D3C64">
    <physical>
      <pfn filetype="ROOT_All"
name="dcache:/pnfs/usatlas.bnl.gov/atlas/offline/data/testfile/q02initialprod.0001.H_2e2mu.q02dig_1004.etac
ut.0001_extract.pool.root"/>
    </physical>
  </File>

If the file is on a normal disk, it could be e.g.,

  <File ID="8474BB36-B213-DA11-87E4-000E0C4D3C64">
    <physical>
      <pfn filetype="ROOT_All"
name="/scratch/atlas/offline/data/testfile/q02initialprod.0001.H_2e2mu.q02dig_1004.etacut.0001_extract.pool
.root"/>
    </physical>
  </File>

To summarize, a PFN is sn URL in one of the following formats:

• dcap://host/location - remote dCache
• dcache:/location - local dCache
• location - disk

-- MarcoMambelli - 01 Nov 2006