NotesDec19

Attending

• Jack, Jerry, Rob, Marco, David

Agenda

• Marco test running on the whole sample (condor jobs in the cluster)
• Jerry's tests with TNT
• Jack's report from the SW week
• plans to move forward

Discussion

Marco and test on the whole DS: The test ran successfully the skimming of the whole dataset on the local condor cluster splitting the submission in 10 different job. Details and results are available in the page SkimTest061212

Jerry and TNT: TNT is a nice convenient way to set up the query to select the events. It is a python script to conveniently split the collection in sub-collections depending on the number of events that you want to process per job, good frontend to create the sub-collection. Anyway it is very tied in into LFC: it requires a complete dataset and it uses LFC and not the LRC catalog (e.g. it is using all the LFC tools coming from LCG). At the end (after shortcutting most of its elements) the test generated a JDL input for the jobs but we (in OSG) need Condor submit files to be able to use them with Condor-G and OSG. Result and details of Jerry's test are available in TagNavigatorToolOSG

RWG: Attention not to defocus paying too much attention to TNT

JC: We should point to have a working skim selector and a skim extractor, not 'THE' skim selector/extractor

RWG: the next target-motivator could be the next MW-physics analysis jamboree at ANL (Check with Tom). We could provide a little package of goodies that may be useful for the people at the analysis jamboree.

MM: which are the input that a user will provide to the Data Skimming service?

• DataSet
• Query: provide a where clause to the query to the DB
• name of the collection (DB that contains the TAGs, e.g. testIdeal): connect string for a collection (URL of DB or file)

We should check in TNT what people is providing to it to see what could be requited. At the moment there is no need for different jobOption files (a single one is OK) because we are doing only extraction; later we may swap elements and require different jobOptions.

From the SW week:

• DQ2 is moving to custom databases for the central catalogs. Will the move involve also file catalogs (LFC, LRC)?

At the moment LRC is OK. It is anyway missing file type information but files can all be safely classified as rootall type.

• Jack took the AMI tutorial and it seems to be much more useful than in the past. Has a better user interface than the DS browser at BNL and was returning the similar results in a more flexible way. Others should take a look the AMI tutorial at software week and form their impressions.

JG: Sometime tools have problems to deal with incomplete DS. How can we declare a DS complete? DS have to be closed before to become complete and only the owner can close them. A solution can be obtained by building a new DS with all and only the desired files that are present locally and close it and declare it complete.

DM: Get complete AOD from the Data Streaming Test (stream test): it begins with mixed channel data and Tag selection on it would be more interesting. Middle of January or February. J

Next steps

• Investigate TNT, which input is requiring from users, useful features to integrate
• prepare a toolset for the analysis jamboree
  • identify datasets which users want to access and have the TAG's built for them

-- MarcoMambelli - 19 Dec 2006