FY08-Q2 Effort Report of Marco Mambelli

This effort report covers the period of activity of Marco Mambelli from January-March, 2008.

Distributed Data Services and DSS

Work continues in the Tier2 Data Services activity, see further: http://twiki.mwt2.org/bin/view/DataServices/WebHome. This activity involves a joint group from the Midwest Tier2 Center and Argonne National Lab (Jack Cranshaw, David Malon, Rob Gardner).

The goal of the activity is to design, prototype and package services that:

• Host or provide access to ATLAS specific database services, such as TAG and possibly conditions (IOV and calibration) databases.
• Provide a dataset skimming service (DSS) for Tier2-resident datasets through either command line or web interfaces.
• Provide users the ability to access the Tier2's DQ2 server.

Work towards these goals included: group coordination (chairing meetings, taking minutes); planning, scope definition and initial organization; prototype data services machine initial deployment; testing. DSS has ben used successfully by friendly users to perform skims of FDR data using tags from the TAG DB.

DSS Milestones:

• The prototype of the Web interface is available at: http://tier2-06.uchicago.edu:8800/dss/
• Further development and integration of skim extraction jobs
• Further development and support of the Data Movement Utilities component (DMU) of DSS to move data reliably between local storage elements and worker nodes, including file registration in DQ2 site services. The DMU component is also in use by Panda.
• DSS v0.5. Integrated with Oracle TAG DB (ELSSI) and allows execution of local or Pathena jobs using streamtest data and ATLAS Rel 12.0.6 (15 Dec 07, 3 Jan 08, completed)
• TAG Session at the Atlas Software Workshop: skimming of Stremtest data with TAG DB (25 Feb 08, 25 Feb 08, completed)
• DSS v0.6. Integrated with Oracle TAG DB (ELSSI) and allows execution of local or Pathena jobs using FDR-1 data and ATLAS Rel 13.0.4 (unplanned, 15 Mar 08, completed)
• Demo and examples on FDR analysis using Pathena and TAG from the TAG DB during US ATLAS FDR Analysis Jamboree (18 Mar 08, 18-20 Mar 08, completed)
• Planned Functional Delivery of DSS 1.0 (30 Sep 07, 23 Jun 08, delayed/expanded)

User and Tier3 support

Prepare and document an exemplar installation at MWT2 of the software used by end users to transfer datasets (with the support of Charles Waldman - sysadmin). Support the existing installation, support sysadmin (UIUC Tir3), educate/support analysis user in moving files and registering datasets.

Data movement Milestones:

• troubleshooting and fixing a critical bug in globus-url-copy (the Globus team fixed the actual code) (unplanned, Jan 08, completed)
• new version of dq2_put improving reliability and performance and adding new features required by users: (unplanned, 11 Feb 08, completed)
  • allowed publication of new pythia generated events (Fred Luehering)
  • publication of user.MonicaDunford.mSugraGrid_25x25 dataset (Monica Dunford)
• presentation at the US ATLAS Transparent Distributed Facility workshop "ATLAS data and the Tier3" (4 Mar 08, 4 Mar 08, completed)

Panda Troubleshooting

Continued activity on PilotChecker (project started on FY07Q3), see further: http://twiki.mwt2.org/bin/view/DataServices/PandaSubmitHost. This activity emerged as support of my troubleshooting activity for Panda production. Troubleshooting needs and Site administrators requests prompted improved versions of the tool. The tool has been used also for official Site certification (http://www.usatlas.bnl.gov/twiki/bin/view/Admins/SiteCertificationP1) following the 2 steps procedure defined in http://www.usatlas.bnl.gov/twiki/bin/view/Admins/PilotCheckerP1.html. This activity includes maintenance of the server and development.

PilotChecker Milestones:

• Support and small UI improvements of PilotChecker v 0.3

Continued activity in support of Panda production, specially the pilot submission, pilot troubleshooting on USATLAS sites and help supporting ATLAS production at MWT2 and integration of new Tier3 (UIUC).

Readiness verification consists in:

• work closely with CE-admin
• support OSG installation
• verify that OSG components are installed correctly and functional
• coordinate ATLAS sw installation
• verify that ATLAS pilots and then ATLAS jobs execute correctly
• coordinate activation of the CE in Panda central server and monitoring infrastructure
• verify that the site is ready for production and notify production shift

Some accomplishments for Panda production:

• Rewriting of data movement utilities (used by pilot2/3) to make a real use of timeouts (using timed_command module contributed by Charles Waldman) (Jan 08, Jan 08, completed)
• Readiness verification of new sites like UIUC-HEP (unplanned, 31 Mar 08, continuing)

-- MarcoMambelli - 15 Apr 2008