DeployXrootdProto

Software

• OSG Tier 3 Instructions are here: https://twiki.grid.iu.edu/bin/view/Tier3/XrootdInstall
• Installing from current OSG production cache: OSG 1.2.10
• OSG Release Documentation: https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/WebHome
• OSG 1.2.10 release notice: https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/OSG1210ReleaseNotice
• VDT 2.0.0 documentation
• VDT 2.0.0 release notes: http://vdt.cs.wisc.edu/releases/2.0.0/release-p17.html
• Xrootd 20100315-1007 (http://xrootd.slac.stanford.edu/)
• XrootdFS 3.0rc4 (http://wt2.slac.stanford.edu/xrootdfs/)
• Xrootd configuration: http://vdt.cs.wisc.edu/releases/2.0.0/man/configure_xrootd.html

Testbed machine inventory

The testbed we are using is the very old Tier 2 prototype cluster (admin page protected). Some of its features:

• tier2-mgt.uchicago.edu is the cluster management node (runs Puppet and Cobbler) -- http://tier2-mgt.uchicago.edu/cobbler_web/
• tier2-02 (alias tier2-osg) is the job management node (runs Condor) assign as a data serverr
• tier2-04 -- available: assign to be the redirector
• tier2-u2 - an interactive user node assign to run XrootdFS
• tier2-se2 (alias se2) is a storage node on the internal network; has two 1.8 TB partitions
• se5 serves the home directories & is the main user management node
• tier2-c021.local, tier2-c022.local: worker nodes, 32 bit Xeon 3.0 GHz dual CPU (single core) machines, 2GB RAM/core, all on a private network (but with outbound internet access)
• Servers running SL5.3 with a 2.6.18-164 kernel (with variations for both)

Accounts

• Service accounts have already been created: atlasadmin, condor, xrootd
• Working with Aaron to get these consistently setup.
• Aside: to re-run puppet (on node in question) /usr/sbin/puppetd --onetime --no-daemonize --logdest syslog
• We have some cleanup to do (/etc/passwd, /etc/group)

Checking firewalls

On both redirector and data server, eg:

# /etc/init.d/iptables status
Firewall is stopped.

Pacman

• tier2-02: /opt
• tier2-04: /opt
• tier2-u2: /opt

• # cd /opt
  # wget http://atlas.bu.edu/~youssef/pacman/sample_cache/tarballs/pacman-3.28.tar.gz
  # tar --no-same-owner -xzvf pacman-3.28.tar.gz
  # ln -s pacman-3.28 pacman
  # cd pacman
  # source setup.sh 
  

Redirector

• tier2-04
• I followed the instructions https://twiki.grid.iu.edu/bin/view/Tier3/XrootdInstall
• For simplicity I create directories, /xrd/path, /xrd/cache and bind mount the existing /export/ directory (which has 700 GB free) to /xrd: (mount --bind /export/ /xrd/).

Data server

• tier2-02 - seems weird to use this host, but its got some disk and this is a throw-away test.
• I followed the instructions https://twiki.grid.iu.edu/bin/view/Tier3/XrootdInstall
• Same thing regarding the bind mount trick, so we have the same directory structure on both machines (/xrd/path/, /xrd/cache/).

XrootdFS

• tier2-u2, a "user" machine
• We want to mount the xrootd file system implemented by tier2-4 and tier2-02

Benchmark jobs

• JoesROOTBenchmark
• DougsBenchmark

References

• ATLAS Tier 3 Xrootd working group page: https://twiki.cern.ch/twiki/bin/view/Atlas/XrootdTier3
• ATLAS Tier 3 Xrootd VDT-based instructions: https://twiki.cern.ch/twiki/bin/view/Atlas/XrootdVDT
• Notes from OSG Tier 3 group: https://twiki.grid.iu.edu/bin/view/Tier3/XrootdInstall
• MWT2 WAN Xrootd testing notes, ClusterUnification (protected)

-- RobGardner - 20 Jun 2010