FY08-Q4 Effort Report of Marco Mambelli

This effort report covers the period of activity of Marco Mambelli from July-September, 2008.

Distributed Data Services and DSS

Work continues in the Tier2 Data Services activity, see further: http://twiki.mwt2.org/bin/view/DataServices/WebHome. This activity involves a joint group from the Midwest Tier2 Center and Argonne National Lab (Jack Cranshaw, David Malon, Rob Gardner).

The goal of the activity is to design, prototype and package services that:

• Host or provide access to ATLAS specific database services, such as TAG and possibly conditions (IOV and calibration) databases.
• Provide a dataset skimming service (DSS) for Tier2-resident datasets through either command line or web interfaces.
• Provide users the ability to access the files on Tier2's SE and registered in the file catalog.

Work towards these goals included: group coordination (chairing meetings, taking minutes); planning, scope definition and initial organization; prototype data services machine initial deployment; testing.

DSS has ben used successfully by friendly users to perform skims of FDR data using tags from the TAG DB.

Package software to access data in US-ATLAS (WLCG-Client) and provide and update user documentation like:

• http://twiki.mwt2.org/bin/view/DataServices/InstallingDQ2AtUC
• http://www.usatlas.bnl.gov/twiki/bin/view/Admins/WlcgClient

DSS Milestones:

• The prototype of the Web interface is available at: http://tier2-06.uchicago.edu:8800/dss/
• Further development and integration of skim extraction jobs
• Further development and support of the Data Movement Utilities component (DMU) of DSS to move data reliably between local storage elements and worker nodes, including file registration in DQ2 site services. The DMU component is also in use by Panda.
• Demo and examples on FDR-2 analysis using Pathena and TAG from the TAG DB as part of the TAG demo during ATLAS offline software tutorial (18 Jul 08, 16-18 Jul 08, completed)
• Release of WLCG-Client v0.13 - (unplanned, 15 Aug 08, completed)
• Testing and troubleshooting of LFC-Client package from VDT (Aug 08, Aug 08, completed)
• Demo and examples on FDR-2 analysis using Pathena and TAG from the TAG DB as part of the TAG demo during ATLAS offline software tutorial (Aug 08, Aug 08, completed)
• Demo and examples on FDR-2/2c analysis using Pathena and TAG Analysis Jamboree for Readiness Exercise at ANL/BNL/LBNL (9 Sep 08, 9-12 Sept 08, completed)
• Planned Functional Delivery of DSS 1.0 (30 Nov 07, 23 Jun 08, delayed/expanded)

Related to data movement are the following roles:

• Responsible for US-ATLAS Facilities Clients: http://www.usatlas.bnl.gov/twiki/bin/view/Admins/USAtlasFacilityClients.html
• Member of the LFC Subcommittee to facilitate File Catalog migration (from LRC to LFC) of US-ATLAS sites: http://www.usatlas.bnl.gov/twiki/bin/view/Admins/FileCatalog.html

Data movement milestones

• Pre-release WlcgClient for testing (WLCG-Client 0.10): June 30
• First release of WlcgClient for wider adoption (WLCG-Client 0.12): July 25
• Report on the status of Storage Elements client access in US-ATLAS at the US-ATLAS weekly facilities meeting (13 Aug 08, 13 Aug 08, completed)
• First production release of WlcgClient (WLCG-Client 0.14): Sep 22 (Note1)
• New production release of WN-Client (including LFC support v1.10.1mwt2): Sep 22
• Facility-wide client throughput performance report: Sep 30

User and Tier3 support

Prepare and document an exemplar installation at MWT2 of the software used by end users to transfer datasets (with the support of Charles Waldman - sysadmin). Support the existing installation, support sysadmin (UIUC Tir3), educate/support analysis user in moving files and registering datasets.

Data movement Milestones:

• presentation at the ATLAS Tier-2 / Tier-3 Meeting "Tier3 Data Access and Management Issues" (22 Sep 08, 22 Sep 08, completed)

Panda Troubleshooting

Continued activity on PilotChecker (project started on FY07Q3), see further: http://twiki.mwt2.org/bin/view/DataServices/PandaSubmitHost. This activity emerged as support of my troubleshooting activity for Panda production. Troubleshooting needs and Site administrators requests prompted improved versions of the tool. The tool has been used also for official Site certification (http://www.usatlas.bnl.gov/twiki/bin/view/Admins/SiteCertificationP1) following the 2 steps procedure defined in http://www.usatlas.bnl.gov/twiki/bin/view/Admins/PilotCheckerP1.html. This activity includes maintenance of the server and development.

PilotChecker Milestones:

• Support and small UI improvements of PilotChecker v 0.3

Continued activity in support of Panda production, specially the pilot submission, pilot troubleshooting on USATLAS sites and help supporting ATLAS production at MWT2 and integration of new Tier3.

Member of the US-ATLAS production shift team (with Yuri Smirnov, Mark Sosebee, Wensheng Deng, Barry Spurlock), part of the world-wide ATLAS production team coordinated by Xavier Espinal and Kaushik De: https://twiki.cern.ch/twiki/bin/view/Atlas/ADCoS Shifters perform surveillance and troubleshooting of ATLAS production Mon-Sat, alternating with EU and Asian shifters in order to cover a 24h cycle. Each shifters covers 6days/mo (3 2days shifts).

Shift duties include:

• monitor production jobs (Panda, ARDA, ...)
• monitor data transfers (FTS, Panda, ARDA, ...)
• monitor ATLAS tasks
• submit tickets to RT, GGUS and Savannah
• report anomalies on the eLog (http://atlas003.uta.edu:8080/ADCoS/)
• troubleshoot problems
• update other shifters about open problems and investigations
• contact experts or Site's administrators
• report weekly (ADC meeting and Prodsys meeting)
• steer production (activating/deactivating sites depending on problems and planned downtimes)

Shifts are recorded in US Calendar (http://grid.uchicago.edu/cgi/plans.cgi?cal_id=1) and in ATLAS OTP: https://pptevm.cern.ch/mao/client/cern.ppt.mao.app.gwt.MaoClient/MaoClient.html#Ma0_Task_panel(N88)

Some accomplishments for Panda production:

• Support to autopilot: help in troubleshooting pilot and submit host problems
• Support to integrate new CE HU_ATLAS_Tier2 (started 8/8)

-- MarcoMambelli - 26 Jul 2008