Availability on x86_64

Introduction

wlcg-client + DQ2Clients is the current platform recommended to ATLAS users (see wlcg-client: http://www.usatlas.bnl.gov/twiki/bin/view/Admins/WlcgClient).

DQ2Clients is distributing also MySQLdb, MySQL python bindings (see http://sourceforge.net/projects/mysql-python, http://mysql-python.sourceforge.net/MySQLdb.html), compiled for 32 bit platforms. This breaks the package on 64bit platforms:

Possible Solutions

Distribute Python for x86 (32bit)

This is the solution adopted in gLite that includes Python 2.4

• pro: is not relying on the system python, can enforce its own version (if other sw has python version requirements)
• contra: may create incompatibilities, e.g. Pathena requires python>=2.5 and is incompatible with gLite setup
• contra: additional python libraries installed in the system become unavailable
• requires packaging and distribution in wlcg-client or VDT, testing on all platforms
• pro: easier for people that cannot install on host system (no root)

Distribute MySQLdb for both 32bit and 64bit in DQ2Client

DQ2Client developers would have to be convinced to put more effort and provide the additional package

• prepare an additional Pacman package with only MySQLdb would help them
• requires a dynamic loading of the correct MySQLdb (32 or 64 bit)

Distribute MySQLdb for both 32bit and 64bit in wlcg-client

DQ2Client developers would have to be convinced to drop the package:

• 32bit python dependency disappears
• MySQLdb dependency appears and MySQLdb is ont included in gLite
  • additional Pacman package with only MySQLdb would be useful
• requires a dynamic loading of the correct MySQLdb (32 or 64 bit)

Tests

It is possible to recognize the python architecture by executing:

python -c 'import platform; print platform.architecture()[0]'

This would allow to discriminate the directories to add to the

I was able to compile and make a self contained python installation using 32bit executable on a x86_64 OS. Building platform is uct2-dss (required the addition of glibc-devel.i386: yum install glibc-devel.i386). After some experimenting the recipe is simple:

wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tgz
CC="gcc -m32" CXX="c++ -m32" ./configure --prefix=/raid/tests/client/python-2.5
make
make install

Client installation in /raid/tests/client/ works fine:

bash-3.2$ export PATH=/raid/tests/client/python-2.5/bin:$PATH
bash-3.2$ . ./wlcg-client/setup.sh 
bash-3.2$ . ./dq2-client/setup.sh 
bash-3.2$ dq2-ls -L MWT2_UC -fp  fdr08_run1.0003073.StreamMuon.merge.AOD.o1_r24_t3
fdr08_run1.0003073.StreamMuon.merge.AOD.o1_r24_t3
[ ]	fdr08_run1.0003073.StreamMuon.merge.AOD.o1_r24_t3._0001.1	6AC91B51-451D-DD11-906D-0030487CB47A	ad:14be3fa9	620714510
total files: 1
total size: 620714510
date: 2008-05-13 12:08:50

WLGC-Client was installed in the 32bit version with local CA/...:

-bash-3.2$ export VDT_PRETEND_32=1
-bash-3.2$ mkdir wlcg-client
-bash-3.2$ cd wlcg-client
-bash-3.2$ pacman -get http://www.mwt2.org/caches:wlcg-client
...

After the installation certificates need to be installed UPDATE INSTRUCTIONS!

1: Edit the value of cacerts_url in the configuration file at
$VDT_LOCATION/vdt/etc/vdt-update-certs.conf
 
2: Run the following command:
. $VDT_LOCATION/vdt-questions.sh; $VDT_LOCATION/vdt/sbin/vdt-setup-ca-certificates

Python 2.4 with SSL suport

--++ Solution 1: All 32bit In order to do this is required:

• wn-client/wlcg-client 32bit installation
• python 32 bit

There are some requirements for python:

• LFC bindings in VDT's LFC-Client are compiler with Python C API 1012 (included in python 2.3.X and 2.4.X)
• secure connections are required (by pilot, for DQ2 connections), make sure that SSL support is included

To install wlcg-client forcing 32bit:

export VDT_PRETEND_32=1
mkdir wlcg-client; cd wlcg-client
pacman -get http://www.mwt2.org/caches:wlcg-client

A Python 32bit has been packaged and is available:

pacman -get http://www.mwt2.org/caches:Python32bit

To compile it:

mkdir python32bit-2.4
wget http://www.python.org/ftp/python/2.4.5/Python-2.4.5.tgz
tar -xzf Python-2.4.5.tgz
cd Python-2.4.5
CC="gcc -m32" CXX="c++ -m32" ./configure --prefix=../python32bit-2.4
make
make install
cd ..
tar -czf python32bit-2.4.5-mm.tar.gz python32bit-2.4

Make sure that SSL support has been compiled in (it should be by default):

• here they suggest how to force manual inclusion uncommenting lines in Modules/Setup.dist: http://paltman.com/2007/11/15/getting-ssl-support-in-python-251/
• here a test using https: http://www.noah.org/wiki/Python_HTTPS_and_SSL

To test if Python is 32 bit:

• python -c 'import platform; print platform.architecture()[0]' It should reply '32bit'

-- MarcoMambelli - 26 Sep 2008