VDTChimeraUpgrade < ITB

You are here: Foswiki>ITB Web>VDTChimeraUpgrade (12 Jul 2010, SuchandraThapa)Edit Attach

Introduction

Steps taken to upgrade from the vdt 2.4.8 dcache distribution to vdt 3.0.1 dcache distribution. The upgrade involved a change from using pnfs to chimera.

Nodes

itb-dcache1.uchicago.edu -- admin node
itb-dcache1.uchicago.edu -- pnfs and directory node / chimera and directory node
uct3-edge6.uchicago.edu -- pool node

Initial steps / backups

Downloaded files from VDT website to itb-dcache1, itb-dcache2, and uct3-edge6 (all in ~/dcache/3.0.1)
files located in /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/ or /root/dcache/3.0.1/vdt-dcache-SL4_32-3.0.1/ on all nodes

Stop services on systems

itb-dcache1
```
service dcache stop
```

itb-dcache2

service dcache stop
service pnfs stop
service postgresql stop

uct3-edge6
```
service dcache stop
service pool stop
```

Backup pnfs and dcache databases

itb-dcache1
```
tar cvzf /tmp/db.tar.gz /var/lib/pgsql/
```

itb-dcache2

tar cvzf /tmp/pnfs-backup.tar.gz /var/lib/pgsql

Installing/Upgrading

Edited siteinfo.conf (see attached file)

Need to add

DCACHE_PNFS="itb-dcache2.uchicago.edu"

and set

SETUP_CHIMERA="no"

Copy siteinfo.conf=to install directory (=/root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install or /root/dcache/3.0.1/vdt-dcache-SL4_32-3.0.1/install)
On itb-dcache2, start postgresql service
```
service postgresql start
```
(needed for install process)

Installation

itb-dcache1

[root@itb-dcache1:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# ./install.py --dryrun

VDT install of dCache on:
2010-07-12 06:25:27.403213
Will print actions but not perform them.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"
PostgreSQL is already installed and running. Nothing will be done on this node.
This is not the Chimera node.

Dryrun install of dCache.
Extracting dCache configuration files from /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm to /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache for dryrun.
122400 blocks
------------------------------------------------------------------------------------------------------
During DCache installation, the following will be done.
------------------------------------------------------------------------------------------------------
> cp /opt/d-cache/etc/dCacheSetup.template /opt/d-cache/config/dCacheSetup
> cp /opt/d-cache/etc/node_config.template /opt/d-cache/etc/node_config
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
These lines will be set in /opt/d-cache/config/dCacheSetup:
serviceLocatorHost=itb-dcache1.uchicago.edu
java=/usr/java/default/bin/java
permissionPolicyEnforcementPoint=PnfsManager
replicaManager=yes
srmIgnoreClientProtocolOrder=true
srmSpaceManagerEnabled=yes
srmImplicitSpaceManagerEnabled=no
SpaceManagerLinkGroupAuthorizationFileName=/opt/d-cache/etc/LinkGroupAuthorization.conf
remoteGsiftpMaxTransfers=2000
billingToDb=yes

These lines will be set in /opt/d-cache/etc/node_config:
SERVER_ID=uchicago.edu
NAMESPACE_NODE=itb-dcache2.uchicago.edu
NAMESPACE=chimera
NODE_TYPE=custom
lmDomain=yes
poolManager=yes
pnfsManager=no
dirDomain=no
adminDoor=yes
httpDomain=yes
utilityDomain=yes
gPlazmaService=yes
info=yes
replicaManager=yes
infoProvider=no
statistics=no
GSIDCAP=yes
DCAP=yes
GRIDFTP=yes
SRM=yes
XROOTD=no

Will create 1 dcap doors.
Will run /opt/d-cache/install/install.sh

WILL - change /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache/config/PoolManager.conf!! Will use default PoolManager.conf.
Will back up existing /opt/d-cache/config/PoolManager.conf.
Will create new /opt/d-cache/config/PoolManager.conf.
Will run dropdb -U srmdcache replicas
Will run createdb -U srmdcache replicas
Will run psql -U srmdcache -d replicas -f /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache/etc/psql_install_replicas.sql
Will run /opt/d-cache/install/upgrade_space_manager_schema.sh -jdbcUrl=jdbc:postgresql://localhost/dcache -jdbcDriver=org.postgresql.Driver -dbUser=srmdcache -dbPass=srmdcache

Removing temporary files for dryrun.
Finished Dryrun of dCache install.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"

Dryrun install of srmwatch.
Extracting dCache configuration files from /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm to /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache for dryrun.
122400 blocks
------------------------------------------------------------------------------------------------------
During srmwatch installation, the following will be done.
------------------------------------------------------------------------------------------------------
These lines will be set in /opt/d-cache/etc/srm_setup.env:
TOMCAT_HTTP_ENABLED=true

These lines will be set in /opt/d-cache/config/dCacheSetup:
srmJdbsMonitoringLogEnabled=true
srmDbLogEnabled=true

Will run /opt/install/deploy_srmv2.sh /opt/install/srm_setup.env

Updating previous srmwatch version srmwatch-1.1-0.
Will execute "rpm -Uvh --force /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/srmwatch-1.1-0.i386.rpm"
Removing temporary files for dryrun.
Finished Dryrun of srmwatch install.
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-common-1.06.16c-1.noarch.rpm"
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-extra-libs-1.06.16c-1.noarch.rpm"
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-extra-libs-arch-spec-1.06.16c-1.i386.rpm"
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-services-1.06.16c-1.noarch.rpm"
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-dCache-transfer-itb-1.06.16c-1.noarch.rpm"
Will execute "rpm -Uvh --nosignature /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/gratia-probe-dCache-storage-itb-1.06.16c-1.noarch.rpm"
Will change ProbeConfig for storage
Checking indexes on the billing database...
its important that all 4 indexes are present, else the gratia probe
will put significant load on your database. See the install log file
for help with creating indexes
billinginfo table does not has "dates_bi" index
login to the billing db and run "create index dates_bi on billinginfo(datestamp)"
billinginfo table does not has "initiator" index
login to the billing db and run "create index initiator on billinginfo(initiator);"
doorinfo table does not has "transaction" index
login to the billing db and run "create unique index transaction on doorinfo(transaction);"
doorinfo table does not has "dates_di" index
login to the billing db and run "create index dates_di on doorinfo(datestamp);"
Will execute ". /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/install_jython.sh;install_jython"
dCache Core Operations rpm is already unpacked.
dCache Chronicle rpm is already unpacked.

[root@itb-dcache1:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# ./install.py
VDT install of dCache on:
2010-07-12 06:27:12.900349
Unpacking dCache server software.
Checking services running on this node ...
PostgreSQL is already installed and running. Nothing will be done on this node.
This is not the Chimera node.
Installing dCache -
Transferring previous settings from dCacheSetup
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
Modifying dCache configuration files
Running dCache setup script.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
mount: mount to NFS server 'itb-dcache2.uchicago.edu' failed: RPC Error: Program not registered.
WARNING - changing /opt/d-cache/config/PoolManager.conf!! Using default PoolManager.conf.
Backing up previous /opt/d-cache/config/PoolManager.conf to /opt/d-cache/config/PoolManager.conf.100712_062812.old.
Dropping and recreating replica manager database.
Checking space manager schema, upgrading if necessary.
Finished installing dCache.
dCache rpm is already unpacked.
Installing srmwatch -
Modifying dCache configuration files
Running srm setup script.
Updating previous srmwatch version srmwatch-1.1-0.
Finished installing srmwatch.
Checking services running on this node ...
Unpacking gratia-probe-common-1.06.16c-1.noarch.rpm
Unpacking gratia-probe-extra-libs-1.06.16c-1.noarch.rpm
Unpacking gratia-probe-extra-libs-arch-spec-1.06.16c-1.i386.rpm
Unpacking gratia-probe-services-1.06.16c-1.noarch.rpm
Unpacking gratia-probe-dCache-transfer-itb-1.06.16c-1.noarch.rpm
Unpacking gratia-probe-dCache-storage-itb-1.06.16c-1.noarch.rpm
Gratia probes have been installed successfully.
To start transfer probe, start dCache and then execute:
/etc/init.d/gratia-dcache-transfer start
Check startup log in file /opt/d-cache/gratia/probe/dCache-transfer/nohup.out
The storage probe runs on its own as a cron job
Checking indexes on the billing database...
its important that all 4 indexes are present, else the gratia probe
will put significant load on your database. See the install log file
for help with creating indexes
billinginfo table does not has "dates_bi" index
login to the billing db and run "create index dates_bi on billinginfo(datestamp)"
billinginfo table does not has "initiator" index
login to the billing db and run "create index initiator on billinginfo(initiator);"
doorinfo table does not has "transaction" index
login to the billing db and run "create unique index transaction on doorinfo(transaction);"
doorinfo table does not has "dates_di" index
login to the billing db and run "create index dates_di on doorinfo(datestamp);"
Installing jython...
Welcome to Jython !
You are about to install Jython version 2.2.1
(at any time, answer c to cancel the installation)
For the installation process, the following languages are available: English, German
Please select your language [e/g] >>> e
Do you want to read the license agreement now ? [y/n] >>> n
Do you accept the license agreement ? [y/n] >>> y
The following installation types are available:
1. All (everything, including sources)
2. Standard (core, library modules, demos and examples, documentation)
3. Minimum (core)
9. Standalone (a single, executable .jar)
Please select the installation type [1/2/3/9] >>> 2
Do you want to install additional parts ? [y/n] >>> n
Do you want to exclude parts from the installation ? [y/n] >>> n
Please enter the target directory >>> /opt/jython
Directory /opt/jython is not empty - ok to overwrite contents ? [y/n] >>> y
Contents of directory /opt/jython will be deleted now! Are you sure to proceed ? [y/n] >>> y
Please enter the target java home directory (== for current) >>> ==
Your java version to start Jython is: Sun Microsystems Inc. / 1.6.0_20
Please press Enter to proceed >>>
Your operation system version is: Linux / 2.6.18-164.6.1.el5
Please press Enter to proceed >>>
Summary:
- mod: true
- demo: true
- doc: true
- src: false
- JRE: /usr/java/jdk1.6.0_20/jre
Please confirm copying of files to directory /opt/jython [y/n] >>> y
10 %
20 %
30 %
40 %
50 %
60 %
70 %
80 %
90 %
100 %
Generating start scripts ...
100 %
Do you want to show the contents of README ? [y/n] >>> n
Congratulations! You successfully installed Jython 2.2.1 to directory /opt/jython.
dCache Core Operations rpm is already unpacked.
dCache Chronicle rpm is already unpacked.

itb-dcache2

[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# ./install.py  --dryrun

VDT install of dCache on: 
2010-07-12 06:21:59.213204
Will print actions but not perform them.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"
PostgreSQL is already installed and running. Nothing will be done on this node.
You have chosen NOT to setup Chimera.

Dryrun install of dCache.
Extracting dCache configuration files from /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm to /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache for dryrun.
122400 blocks
------------------------------------------------------------------------------------------------------
During DCache installation, the following will be done.
------------------------------------------------------------------------------------------------------
> cp /opt/d-cache/etc/dCacheSetup.template /opt/d-cache/config/dCacheSetup
> cp /opt/d-cache/etc/node_config.template /opt/d-cache/etc/node_config
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
These lines will be set in /opt/d-cache/config/dCacheSetup:
serviceLocatorHost=itb-dcache1.uchicago.edu
java=/usr/java/default/bin/java
replicaManager=yes
srmIgnoreClientProtocolOrder=true
srmSpaceManagerEnabled=yes
srmImplicitSpaceManagerEnabled=no
SpaceManagerLinkGroupAuthorizationFileName=/opt/d-cache/etc/LinkGroupAuthorization.conf
remoteGsiftpMaxTransfers=2000
billingToDb=yes

These lines will be set in /opt/d-cache/etc/node_config:
SERVER_ID=uchicago.edu
NAMESPACE_NODE=itb-dcache2.uchicago.edu
NAMESPACE=chimera
NODE_TYPE=custom
lmDomain=no
poolManager=no
pnfsManager=yes
dirDomain=yes
adminDoor=no
httpDomain=no
utilityDomain=no
gPlazmaService=no
info=no
replicaManager=no
infoProvider=no
statistics=no
GSIDCAP=no
DCAP=no
GRIDFTP=no
SRM=no
XROOTD=no

Will run /opt/d-cache/install/install.sh

Removing temporary files for dryrun.
Finished Dryrun of dCache install.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"


[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# ./install.py
VDT install of dCache on: 
2010-07-12 06:22:24.765062
Unpacking dCache server software.
Checking services running on this node ...
PostgreSQL is already installed and running. Nothing will be done on this node.
You have chosen NOT to setup Chimera.
Installing dCache -
Transferring previous settings from dCacheSetup
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
Modifying dCache configuration files
Running dCache setup script.
mount: mount to NFS server 'localhost' failed: timed out (retrying).
mount: mount to NFS server 'localhost' failed: timed out (retrying).
mount: mount to NFS server 'localhost' failed: timed out (retrying).
mount: mount to NFS server 'localhost' failed: timed out (retrying).
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
mount: mount to NFS server 'localhost' failed: RPC Error: Program not registered.
Finished installing dCache.
dCache rpm is already unpacked.
Checking services running on this node ...

uct3-edge6

[root@uct3-edge6 install]# ./install.py  --dryrun

VDT install of dCache on: 
2010-07-12 06:27:33.252634
Will print actions but not perform them.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"
None of the services, which need postgres, are running on this node. No need to install postgres.
This is not the Chimera node.

Dryrun install of dCache.
Extracting dCache configuration files from /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm to /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/dryrun/d-cache for dryrun.
122400 blocks
------------------------------------------------------------------------------------------------------
During DCache installation, the following will be done.
------------------------------------------------------------------------------------------------------
> cp /opt/d-cache/etc/dCacheSetup.template /opt/d-cache/config/dCacheSetup
> cp /opt/d-cache/etc/node_config.template /opt/d-cache/etc/node_config
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
These lines will be set in /opt/d-cache/config/dCacheSetup:
serviceLocatorHost=itb-dcache1.uchicago.edu
java=/usr/java/default/bin/java
replicaManager=yes
srmIgnoreClientProtocolOrder=true
srmSpaceManagerEnabled=yes
srmImplicitSpaceManagerEnabled=no
SpaceManagerLinkGroupAuthorizationFileName=/opt/d-cache/etc/LinkGroupAuthorization.conf
remoteGsiftpMaxTransfers=2000
billingToDb=yes

These lines will be set in /opt/d-cache/etc/node_config:
SERVER_ID=uchicago.edu
NAMESPACE_NODE=itb-dcache2.uchicago.edu
NAMESPACE=chimera
NODE_TYPE=pool
lmDomain=no
poolManager=no
pnfsManager=no
dirDomain=no
adminDoor=no
httpDomain=no
utilityDomain=no
gPlazmaService=no
info=no
replicaManager=no
infoProvider=no
statistics=no
GSIDCAP=no
DCAP=no
GRIDFTP=no
SRM=no
XROOTD=no

Will run /opt/d-cache/install/install.sh

Creating and adding pools...
Will execute /opt/d-cache/bin/dcache pool create 870G /dcache
Will execute /opt/d-cache/bin/dcache pool add uct3-edge6_1 /dcache

Removing temporary files for dryrun.
Finished Dryrun of dCache install.
Unpacking dCache server software.
Will execute "rpm -Uvh /root/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install/../RPMS/dcache-server-1.9.5-19.noarch.rpm"


[root@uct3-edge6 install]# ./install.py

VDT install of dCache on: 
2010-07-12 06:28:21.281886
Unpacking dCache server software.
Checking services running on this node ...
None of the services, which need postgres, are running on this node. No need to install postgres.
This is not the Chimera node.
Installing dCache -
Transferring previous settings from dCacheSetup
DCACHE_REPLICA_MANAGER ... itb-dcache1.uchicago.edu
DCACHE_DOOR_SRM_IGNORE_ORDER ... true
SRM_SPACEMGR_ENABLED ... yes
SRM_IMPLICIT_SPACE_RES ... no
SRM_LINK_GROUP_AUTH_FILE ... /opt/d-cache/etc/LinkGroupAuthorization.conf
Modifying dCache configuration files
Running dCache setup script.
Creating and adding pools...
poolname: uct3-edge6_1
Finished installing dCache.
dCache rpm is already unpacked.
Checking services running on this node ...

The pnfs mounting errors are probably due to pnfs being stopped, didn't seem to affect the install

PNFS/Chimera migration

On itb-dcache2.uchicago.edu (the pnfs/chimera server):

[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# createdb -U postgres chimera
[root@itb-dcache2:~]# createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt chimera
Enter password for new role: 
Enter it again: 
[root@itb-dcache2:~]# psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/create.sql
psql:/opt/d-cache/libexec/chimera/sql/create.sql:23: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_inodes_pkey" for table "t_inodes"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:35: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_dirs_pkey" for table "t_dirs"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:45: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_inodes_data_pkey" for table "t_inodes_data"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:56: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_inodes_checksum_pkey" for table "t_inodes_checksum"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:73: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_1_pkey" for table "t_level_1"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:87: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_2_pkey" for table "t_level_2"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:101: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_3_pkey" for table "t_level_3"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:115: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_4_pkey" for table "t_level_4"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:129: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_5_pkey" for table "t_level_5"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:143: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_6_pkey" for table "t_level_6"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:157: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_level_7_pkey" for table "t_level_7"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:175: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_tags_inodes_pkey" for table "t_tags_inodes"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:189: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_tags_pkey" for table "t_tags"
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
CREATE INDEX
CREATE INDEX
psql:/opt/d-cache/libexec/chimera/sql/create.sql:256: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_storageinfo_pkey" for table "t_storageinfo"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:263: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_access_latency_pkey" for table "t_access_latency"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:270: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_retention_policy_pkey" for table "t_retention_policy"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:295: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_locationinfo_pkey" for table "t_locationinfo"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:311: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_locationinfo_trash_pkey" for table "t_locationinfo_trash"
CREATE TABLE
CREATE INDEX
psql:/opt/d-cache/libexec/chimera/sql/create.sql:332: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "t_acl_pkey" for table "t_acl"
CREATE TABLE
CREATE INDEX
[root@itb-dcache2:~]# createlang -U postgres plpgsql chimera
[root@itb-dcache2:~]# psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE TRIGGER
CREATE FUNCTION
CREATE TRIGGER
CREATE SEQUENCE
CREATE FUNCTION
CREATE TRIGGER
vi /opt/d-cache/config/chimera-config.xml

Changed the user to chimera and password to password given when creating chimera user for postgresql. Also changed log file to /var/log/chimera.log

[root@itb-dcache2:~]# vi /etc/exports

Added the following two entries / localhost(rw), /pnfs on separate lines

[root@itb-dcache2:~]# ln -s /opt/d-cache/libexec/chimera/chimera-nfs-run.sh /etc/init.d/chimera-nfs-run.sh
[root@itb-dcache2:~]# chkconfig --add chimera-nfs-run.sh
[root@itb-dcache2:~]# chkconfig chimera-nfs-run.sh on
[root@itb-dcache2:~]# chkconfig chimera-nfs-run.sh on
[root@itb-dcache2:~]# chkconfig --list chimera-nfs-run.sh
chimera-nfs-run.sh	0:off	1:off	2:on	3:on	4:on	5:on	6:off
[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh start
Starting Chimera-NFSv3 interface
Waiting for NFS server to register itself: ....
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/uchicago.edu
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/YOUR-DOMAIN/data
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/uchicago.edu/data
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/
fs/           ftpBase/      uchicago.edu  
[root@itb-dcache2:~]# /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/uchicago.edu/data/uchicago
[root@itb-dcache2:~]# echo "chimera" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/uchicago.edu/data sGroup
[root@itb-dcache2:~]# echo "StoreName sql" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/uchicago.edu//data OSMTemplate

Following done to support dcap using the mounted fs

root@itb-dcache2:~]# mount localhost:/ /mnt
[root@itb-dcache2:~]# mkdir /mnt/admin/etc/config/dCache
[root@itb-dcache2:~]# touch /mnt/admin/etc/config/dCache/dcache.conf
[root@itb-dcache2:~]# touch /mnt/admin/etc/config/dCache/'.(fset)(dcache.conf)(io)(on)'
[root@itb-dcache2:~]# echo "FQHN-dCap-Door:Port" > /mnt/admin/etc/config/dCache/dcache.conf
[root@itb-dcache2:~]# echo "itb-dcache1.uchicago.edu:22125" > /mnt/admin/etc/config/dCache/dcache.conf
[root@itb-dcache2:~]# umount /mnt/

PNFS migration

Following steps taken

[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh stop
Shutting down Chimera-NFSv3 interface
[root@itb-dcache2:~]# rpm -Uvh ~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/RPMS/pnfs-dump-1.0.21-1.i386.rpm 
Preparing...                ########################################### [100%]
	file /opt/pnfs/share/sql/prep-chimera-for-migration.sql from install of pnfs-dump-1.0.21-1.i386 conflicts with file from package pnfs-postgresql-3.1.14-5f.sl52.i386

Note you need to force the upgrade due to the file conflict

[root@itb-dcache2:~]# psql -U postgres -f /opt/pnfs/share/sql/prep-chimera-for-migration.sql chimera
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE FUNCTION

[root@itb-dcache2:~]# /etc/init.d/pnfs start
Starting dCache PNFS services... Starting pnfs services (PostgreSQL version): 
 Shmcom : Installed 8 Clients and 8 Servers
 Starting database server for admin (/opt/pnfsdb/pnfs/databases/admin) ... O.K.
 Starting database server for data1 (/opt/pnfsdb/pnfs/databases/data1) ... O.K.
 Waiting for dbservers to register ... Ready
 Starting Mountd : pmountd 
 Starting nfsd : pnfsd 
mount
[root@itb-dcache2:~]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
itb1:/certificates on /nfs/certificates type nfs4 (rw,noatime,bg,intr,addr=128.135.158.224)
itb1:/home on /nfs/home type nfs4 (rw,bg,intr,addr=128.135.158.224)
localhost:/fs on /pnfs/fs type nfs (rw,udp,intr,noac,hard,nfsvers=2,addr=127.0.0.1)

Get the pnfs directory id for the main directory

[root@itb-dcache2:~]# cd /pnfs/uchicago.edu/data/
[root@itb-dcache2:/pnfs/uchicago.edu/data]# cat '.(get)(cursor)'
dirID=000100000000000000001060
dirPerm=0000001400000000
mountID=000000000000000000001040
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/install]# /etc/init.d/pnfs stop

Get the chimera directory id for the main directory

[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh start
Starting Chimera-NFSv3 interface
Waiting for NFS server to register itself: 
..
[root@itb-dcache2:~]# mount localhost:/pnfs /pnfs
[root@itb-dcache2:~]# cat '/pnfs/uchicago.edu/.(id)(data)'
000023889E3CF7134EEEBB4746FFED0D65DF
[root@itb-dcache2:~]# umount /pnfs
[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh stop
Shutting down Chimera-NFSv3 interface

The chimera and pnfs directory ids need to be used where indicated

[root@itb-dcache2:~]# /etc/init.d/pnfs start
Starting dCache PNFS services... Starting pnfs services (PostgreSQL version): 
 Shmcom : Installed 8 Clients and 8 Servers
 Starting database server for admin (/opt/pnfsdb/pnfs/databases/admin) ... O.K.
 Starting database server for data1 (/opt/pnfsdb/pnfs/databases/data1) ... O.K.
 Waiting for dbservers to register ... Ready
 Starting Mountd : pmountd 
 Starting nfsd : pnfsd 

[root@itb-dcache2:~]# /opt/pnfs/tools/pnfsDump -r 000100000000000000001060 -vv -o chimera /tmp/pnfs.sql -2 -p
-bash: /opt/pnfs/tools/pnfsDump: No such file or directory

Needed to force rpm install in the previous step, doing so now

[root@itb-dcache2:~]# rpm --force -Uvh dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/RPMS/pnfs-dump-1.0.21-1.i386.rpm 
Preparing...                ########################################### [100%]
   1:pnfs-dump              ########################################### [100%]

The argument to -r needs to be the pnfs directory id and the argument to -p needs to be the chimera directory id

[root@itb-dcache2:~]# /opt/pnfs/tools/pnfsDump -r 000100000000000000001060 -vv -o chimera /tmp/pnfs.sql -2 -p 000023889E3CF7134EEEBB4746FFED0D65DF -o verify /tmp/verify.md5 -r -o files /tmp/files.lst -f | tee /tmp/pnfs-dump.log

Final stats:

inodes:
  nDir:            3
  nFile:           1
  nUnknown:        0
  nSkipped:        0
  -----------------
  Total:           4

Major DB errors: 0

Cache queries: 60 (hits: 30, misses: 0)

Time elapsed:    0.0s (00:00:00.0)
    dbserver:    0.0s (00:00:00.0)
    overhead:    0.0s (00:00:00.0)
    pnfsDump:    0.0s (00:00:00.0)

Average inode processing rate: (unreliable value)

[root@itb-dcache2:~]# psql -U postgres -f /tmp/pnfs.sql chimera
BEGIN
INSERT 0 1
psql:/tmp/pnfs.sql:17: NOTICE:  Tag OSMTemplates for 000023889E3CF7134EEEBB4746FFED0D65DFs exist, updating.
 update_tag 
------------
 
(1 row)

COMMIT
BEGIN
INSERT 0 1
psql:/tmp/pnfs.sql:24: NOTICE:  Tag sGroups for 000023889E3CF7134EEEBB4746FFED0D65DFs exist, updating.
 update_tag 
------------
 
(1 row)

COMMIT
BEGIN
INSERT 0 1
INSERT 0 1
UPDATE 1
INSERT 0 1
INSERT 0 1
COMMIT
BEGIN
INSERT 0 1
INSERT 0 1
UPDATE 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 1
COMMIT
BEGIN
INSERT 0 1
INSERT 0 1
UPDATE 1
INSERT 0 1
INSERT 0 1
COMMIT
BEGIN
INSERT 0 1
INSERT 0 1
UPDATE 1
INSERT 0 1
INSERT 0 1
COMMIT

[root@itb-dcache2:~]# /etc/init.d/pnfs stop
Stopping dCache PNFS services... Shutting down pnfs services (PostgreSQL version): 
 Stopping Heartbeat ....  Ready
 Killing pnfsd  Done
 Killing pmountd . Done
 Killing dbserver . Done
 Removing 8 Clients  0+ 1+ 2+ 3+ 4+ 5+ 6+ 7+
 Removing 8 Servers  0+ 1+ 2+ 3+ 4+ 5+ 6+ 7+
 Removing main switchboard ... O.K.

[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh start
Starting Chimera-NFSv3 interface
Waiting for NFS server to register itself: ..
[root@itb-dcache2:~]# mount localhost:/pnfs /pnfs
[root@itb-dcache2:~]# cd /pnfs/uchicago.edu/data/uchicago/
[root@itb-dcache2:/pnfs/uchicago.edu/data/uchicago]# md5sum -c /tmp/verify.md5 | grep -v :\ OK$
md5sum: .(tag)(OSMTemplate): No such file or directory
md5sum: .(tag)(sGroup): No such file or directory
md5sum: .(id)(rsv): No such file or directory
md5sum: rsv/.(tag)(OSMTemplate): No such file or directory
md5sum: rsv/.(tag)(sGroup): No such file or directory
md5sum: rsv/.(id)(1273105680-storage-probe-test-file-remote.28268): No such file or directory
md5sum: rsv/.(use)(2)(1273105680-storage-probe-test-file-remote.28268): No such file or directory
md5sum: .(id)(osg): No such file or directory
md5sum: osg/.(tag)(OSMTemplate): No such file or directory
md5sum: osg/.(tag)(sGroup): No such file or directory
md5sum: .(id)(osgedu): No such file or directory
md5sum: osgedu/.(tag)(OSMTemplate): No such file or directory
md5sum: osgedu/.(tag)(sGroup): No such file or directory
md5sum: WARNING: 13 of 13 listed files could not be read
.(tag)(OSMTemplate): FAILED open or read
.(tag)(sGroup): FAILED open or read
.(id)(rsv): FAILED open or read
rsv/.(tag)(OSMTemplate): FAILED open or read
rsv/.(tag)(sGroup): FAILED open or read
rsv/.(id)(1273105680-storage-probe-test-file-remote.28268): FAILED open or read
rsv/.(use)(2)(1273105680-storage-probe-test-file-remote.28268): FAILED open or read
.(id)(osg): FAILED open or read
osg/.(tag)(OSMTemplate): FAILED open or read
osg/.(tag)(sGroup): FAILED open or read
.(id)(osgedu): FAILED open or read
osgedu/.(tag)(OSMTemplate): FAILED open or read
osgedu/.(tag)(sGroup): FAILED open or read

[root@itb-dcache2:/pnfs/uchicago.edu/data]# cd 
[root@itb-dcache2:~]# umount /pnfs
[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh stop
Shutting down Chimera-NFSv3 interface

The verify should have been done in /pnfs/uchicago.edu/data and not in =/pnfs/uchicago.edu/data/uchicago, redoing verify

[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh start
Starting Chimera-NFSv3 interface
Waiting for NFS server to register itself: ..
[root@itb-dcache2:~]# mount localhost:/pnfs /pnfs
[root@itb-dcache2:~]# cd /pnfs/uchicago.edu/
[root@itb-dcache2:/pnfs/uchicago.edu]# ls
data
[root@itb-dcache2:/pnfs/uchicago.edu]# cd data/
[root@itb-dcache2:/pnfs/uchicago.edu/data]# ls
osg  osgedu  rsv  uchicago
[root@itb-dcache2:/pnfs/uchicago.edu/data]# md5sum -c /tmp/verify.md5 | grep -v :\ OK$
[root@itb-dcache2:/pnfs/uchicago.edu/data]# cd
[root@itb-dcache2:~]# umount /pnfs
[root@itb-dcache2:~]# /etc/init.d/chimera-nfs-run.sh stop
Shutting down Chimera-NFSv3 interface

Migrate companion database

[root@itb-dcache2:~]# pg_dump -U  postgres -t cacheinfo companion | psql -U postgres chimera
SET
SET
SET
SET
SET
SET
SET
SET
CREATE TABLE
ALTER TABLE
ALTER TABLE
CREATE INDEX
CREATE INDEX

[root@itb-dcache2:~]# psql -U postgres -f companion2chimera.sql chimera
companion2chimera.sql: No such file or directory

The companion2chimera.sql is in the tools directory located in the tarball, so you'll need to go to that directory.

[root@itb-dcache2:~]# cd dcache/3.0.1/
[root@itb-dcache2:~/dcache/3.0.1]# cd vdt-dcache-SL5_32-3.0.1
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1]# cd tools/
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# ls
companion2chimera.sql
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# psql -U postgres -f companion2chimera.sql chimera
CREATE FUNCTION
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# echo "select companion2chimera();" | psql -U postgres chimera
 companion2chimera 
-------------------
 
(1 row)

[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# echo "drop table cacheinfo;" |  psql -U postgres chimera
DROP TABLE

Remove pnfs service

[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# chkconfig --del pnfs
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# rm /etc/init.d/pnfs 
rm: remove regular file `/etc/init.d/pnfs'? y

Finishing up and starting services

According to the documentation /opt/d-cache/etc/node_config needs to be edited and the value of NAMESPACE needs to be changed from pnfs to chimera, this was already the case. Now run:

on itb-dcache1.uchicago.edu

root@itb-dcache1:~]# /etc/init.d/postgresql start
Starting postgresql service:                               [  OK  ]
[root@itb-dcache1:~]# /etc/init.d/dcache start
Starting dCache services... Mounting /pnfs
Starting lmDomain Done
Starting dCacheDomain Done
Starting adminDoorDomain 6 Done
Starting httpdDomain Done
Starting utilityDomain 6 Done
Starting gPlazma-itb-dcache1Domain Done
Starting infoDomain Done
Starting replicaDomain Done
Starting dcap-itb-dcache1Domain Done
Starting gridftp-itb-dcache1Domain Done
Starting gsidcap-itb-dcache1Domain Done
Using CATALINA_BASE:   /opt/d-cache/libexec/apache-tomcat-5.5.20
Using CATALINA_HOME:   /opt/d-cache/libexec/apache-tomcat-5.5.20
Using CATALINA_TMPDIR: /opt/d-cache/libexec/apache-tomcat-5.5.20/temp
Using JRE_HOME:       /usr/java/jdk1.6.0_20

Pinging srm server to wake it up, will take few seconds ...
VersionInfo : v2.2
backend_type:dCache

itb-dcache2

[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# /etc/init.d/chimera-nfs-run.sh start
Starting Chimera-NFSv3 interface
Waiting for NFS server to register itself: ..
[root@itb-dcache2:~/dcache/3.0.1/vdt-dcache-SL5_32-3.0.1/tools]# /etc/init.d/dcache start
Starting dCache services... Starting dirDomain Done
Starting chimeraDomain Done

uct3-edge6

[root@uct3-edge6 ~]# /etc/init.d/pool start
Starting Pools... Starting uct3-edge6Domain 6 Done

-- SuchandraThapa - 12 Jul 2010

I	Attachment	Action	Size	Date	Who	Comment
conf	siteinfo.conf	manage	14 K	12 Jul 2010 - 20:25	SuchandraThapa

Topic revision: r2 - 12 Jul 2010, SuchandraThapa

ITB

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback