DeployingOSG1d0

Internal MWT2 note: this PUBLIC VERSION of the DeployingOSG1dot0onUCT2P documented installation.

Background

These are installation notes for OSG 1.0 on tier2-osg.uchicago.edu (site name UC_ATLAS_MWT2). This is being installed before the official OSG 1.0 release date and therefore before the official documentation is available. For now:

• OSG CE install guide from ITB 0.9.1 documentation is here.
• Final release documentation will be here: https://twiki.grid.iu.edu/twiki/bin/view/ReleaseDocumentation/WebHome and the below instructions would be similar.

Host assignments for the UC_ATLAS_MWT2 "site plan":

• tier2-osg.uchicago.edu is the CE
• tier2-05.uchicago.edu is the $APP and wn-client host; exports /share and also /etc/grid-security.
• atlashome is the home server for the usatlas1 user
• se5 is the /home for all other users

Preliminaries

• Updated pacman to the recommended 3.25
• cd /opt
• wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-3.25.tar.gz
• tar --no-same-owner -xzvf pacman-3.25.tar.gz
• /opt/osg is a sym link that should always point to the current release. Previously, it pointed to /opt/osg-0.8.0 which had VDT 1.8.1h.
• cd /opt
• mkdir osg-1.0.0
• Shutdown the existing OSG services (vdt-control --off --force)
  • cf https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/ShuttingDownCE
• logout

Prepare

Consult: https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/PreparingComputeElement

• cd /opt/pacman; source setup.sh; cd /opt/osg-1.0.0
• export VDTSETUP_CONDOR_LOCATION=/opt/condor/
• export VDTSETUP_CONDOR_CONFIG=$VDTSETUP_CONDOR_LOCATION/etc/condor_config
• export VDT_GUMS_HOST=uct2-grid4.uchicago.edu
• export OLD_VDT_LOCATION=/opt/osg-0.8.0
• PATH=$PATH:/opt/condor/bin/

Install of the CE package

• pacman -get OSG:ce-1.0.0 (Note: this is a pre-release version. After 6/13/08, use pacman -get OSG:ce.)
• source setup.sh
• pacman -get OSG:Globus-Condor-Setup-1.0.0

Managed Fork

• I decide to skip the managed-fork in this installation.

Authorization mode: full privilege

• Consulting https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/FullPrivPreConfig
• Inspect $VDT_LOCATION/gums/config/gums-client.properties; no change needed
• cp /opt/osg-1.0.0/post-install/prima-authz.conf /etc/grid-security/
• cp /opt/osg-1.0.0/post-install/gsi-authz.conf /etc/grid-security/

Certificates

• No change

gums-host-cron

• vdt-control --enable gums-host-cron
• $VDT_LOCATION/gums/scripts/gums-host-cron
• logout

Configuring attributes

• Reference https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/ConfigureOSGAttributes
• cd /opt/osg-1.0.0/
• source setup.sh
• cd monitoring
• export OLD_VDT_LOCATION=/opt/osg-0.8.0/
• /configure-osg.py -e
• vi extracted-config.ini and modify for updates
  • disabled managed fork
  • changed these from /osg-0.8.0:
  • gridftp_log = /opt/osg-1.0.0/globus/var/gridftp.log
  • user_vo_map = /opt/osg-1.0.0/monitoring/grid3-user-vo-map.txt
  • osg = /opt/osg-1.0.0
• ./configure-osg.py -c -f ./extracted-config.ini

Turning on services

• Reference https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/StartingServices
• vdt-control --on
• vdt-control --off fetch-crl (getting these from another host)
• vdt-control --off update-certs (getting these from another host)
• vdt-control --off globus-ws (turning this guy off for now)

To disable services so they don't restart after reboot or full =vdt-control --on=:

• vdt-control --disable fetch-crl, etc.

Services:

[root@tier2-osg ~]# vdt-control --list
Service            | Type   | Desired State
-------------------+--------+--------------
fetch-crl          | cron   | do not enable
vdt-rotate-logs    | cron   | enable
vdt-update-certs   | cron   | do not enable
gris               | init   | do not enable
globus-gatekeeper  | inetd  | enable
gsiftp             | inetd  | enable
mysql              | init   | enable
globus-ws          | init   | do not enable
edg-mkgridmap      | cron   | do not enable
gums-host-cron     | cron   | enable
MLD                | init   | do not enable
condor-cron        | init   | enable
apache             | init   | enable
osg-rsv            | init   | do not enable
tomcat-55          | init   | enable
syslog-ng-sender   | init   | do not enable
gratia-condor      | cron   | enable

Installing worker node client

• On tier2-05. Also exports certificates and CRLs for the cluster.
• Update pacman
• cd /export/share/wn-client; source setup.sh
• Inspect it:

  
    root@tier2-05 wn-client]# vdt-control --list
    Service            | Type   | Desired State
    -------------------+--------+--------------
    fetch-crl          | cron   | enable
    vdt-rotate-logs    | cron   | enable
    globus-ws          | init   | do not enable

• vdt-control --off
• Move to wn-client-0.8.0.
• cd /export/share; mkdir wn-client; cd wn-client
• logout/login
• cd /share/wn-client Must invoke pacman from this directory (/export/share is bind mounted to /share)
• pacman -get OSG:wn-client-1.0.0
• Answers:
  • y to trusting caches
  • y to liscenses
  • y to logfile rotation
  • y to CRLs
  • y to certificates
  • r (root) - install into /etc/grid-security/certificates
• source setup.sh
• vdt-control --on
• Check crontab -l; should be something like:

[root@tier2-05 wn-client]# crontab -l
56 1 * * * /share/wn-client/fetch-crl/share/doc/fetch-crl-2.6.6/fetch-crl.cron
0 0 * * * /share/wn-client/vdt/bin/vdt-rotate-logs
29 * * * * /share/wn-client/vdt/sbin/vdt-update-certs-wrapper --vdt-install /share/wn-client --called-from-cron

Requesting a service certificate for RSV

• Reference https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/InstallAndConfigureRSV
• [root@tier2-osg ~]# cert-request -ou s -service rsv -host tier2-osg.uchicago.edu -label rsv-tier2-osg.uchicago.edu
• This produced two files: /root/rsv-tier2-osg.uchicago.edukey.pem and /root/rsv-tier2-osg.uchicago.edu.req.
• Went home
• Next morning, got an email from DOEGrids-CA-1@doegrids.org which said to do: # cert-retrieve -serialnum 24546, but this wont work because of the options used. Note - this has to be done from the directory where cert-request was invoked, as well.
• cert-retrieve -help
• [root@tier2-osg ~]# cert-retrieve -serialnum 24546  -dir ~/. -label rsv-tier2-osg.uchicago.edu
• [root@tier2-osg ~]# mv hostcert.pem /etc/grid-security/rsvcert.pem
• [root@tier2-osg ~]# mv hostkey.pem /etc/grid-security/rsvkey.pem

Create the rsvuser account

• Charles created rsvuser using standard Unix utility and propagating /etc/passwd, etc. to all nodes in the cluster.

GUMS configuration for RSV service certificate

• Reference here is https://twiki.grid.iu.edu/twiki/bin/view/MonitoringInformation/MapServiceCertToRsvuser
• Note: On uct2-grid4, the name of the rsv group is rsvgroup and the rsv user is rsvuser.
• Added rsvgroup to the UC hostToGroupMapping:

        <hostToGroupMapping
           groupToAccountMappings='rsvgroup,cernusatlas, cernatlas, cernusatlasSoft, cernusatlasProd, mis, osg, gums-test, localusers'
           description=''
           cn='*/?*.uchicago.edu'/>
  

• Added the DN /DC=org/DC=doegrids/OU=Services/CN=rsv/tier2-osg.uchicago.edu to the rsvgroup via the web interface, 'Manual User Group Members' -- SarahWilliams - 13 Jun 2008

Now, returning to RSV configuration

• Back to: https://twiki.grid.iu.edu/twiki/bin/view/Integration/ITB090/InstallAndConfigureRSV
• su -c "/bin/date" rsvuser works
• Here is the initial config.ini file w/ suggestions; here is he "extracted" config file currently, before RSV changes: extracted-config.ini.
• See also these pointers https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/ConfigurationFileHelp#Introduction_to_RSV_Section (from the official release docs).
• cd /opt/osg/monitoring/
• Edits to extracted-config.ini
• /configure-osg.py -c -f ./extracted-config.ini
• vdt-control --enable osg-rsv
• vdt-control --on condor-cron osg-rsv
• vdt-control --off apache; vdt-control --on apache
• Check
  • condor_cron_q
  • I found I had to do a full vdt-control --off; vdt-control --on to get condor_cron_q to show any jobs.
  • https://tier2-osg.uchicago.edu:8443/rsv/, visible only with a grid-cert-embedded browser.

Troubleshooting

• Changed /opt/osg/globus/certificates to point to /etc/grid-security/certificates (which is a sym link to the exported /share/certificates from tier2-05, which is being updated automatically by the updater that was installed with wn-client on that host. This makes the org.osg.certificates.crl-expiry RSV probe error go away.

Re-configuring RSV

• Sometimes, running configure_osg can make the rsv config disappear. To re-enable it:

 /opt/osg-1.0.0/vdt/setup/configure_osg_rsv --consumers --server y --init --grid-type OSG --user rsvuser --ce-probes --ce-uri tier2-osg.uchicago.edu --gridftp-probes --gridftp-uri tier2-osg.uchicago.edu --gridftp-dir /share/data/rsvdata/ --gratia --setup-for-apache --use-rsv-cert --rsv-cert-file /etc/grid-security/rsvcert.pem --rsv-key-file /etc/grid-security/rsvkey.pem --rsv-proxy-out-file /tmp/rsvproxy --verbose

--Main.SarahWilliams 2009 02 22

Update of Gratia probes to 1.02.1-5

• CharlesWaldman - 22 Feb 2009
• Updated Gratia probes at suggestion of Chris Green, to solve excessive CPU usage problem.
• cd /opt/osg; . setup.sh; pacman -update
• After this, RSV is not working. Fixed this by adding the following stanza to /opt/osg/apache/http.conf (this was clobbered by the update)

<Directory "/opt/osg/osg-rsv/output/html">
    Options None
    AllowOverride None

    Order allow,deny
    Allow from all
</Directory>
Alias /rsv /opt/osg/osg-rsv/output/html

-- RobGardner - 13 Jun 2008