NodesAllocation

Existing machines

Name	Primary Job	Notes
itb2.uchicago.edu		NFS server for home directories
itb3.uchicago.edu		MediaWiki
itb1.uchicago.edu
itb4.uchicago.edu		ITB Robot host

New machine node matrix

Node Matrix:

Name	VM	Job	RAM/Storage needed	Public IP	Provisioned	Notes
						Host OS for kvm on storage node, has 2 MD1200 attached, VMs for pool only need internal ips
						R610, initially used for io testing
						points to PBS scheduler
						points to Condor scheduler
						RSV / nagios central monitoring
						Home for ITB robot website / VTB cache / SVN webservices
						R610
						hosting pnfs for dcache
						Link to xrootd install
						Link to hdfs install
						(1)
						(1)
						(1)
						Initially used for KVM vs. VM vs. physical performance studies and KVM testing (physical vs. file vs direct i/o)
						Nimbus/VM shipping, setup as a PBS worker node as well
						PBS/condor overlay
						PBS/condor overlay
						PBS/condor overlay
						PBS/condor overlay -- priority to Condor, setup for htpc for condor

Notes:

• (1) Public IP is needed if we want to allow outside users direct access

How to create a KVM host with cobbler:

http://twiki.mwt2.org/bin/view/Admins/NewCobblerNode

New machines 1 (R610 with 2 MD1200)

Name	Primary Job	Cabling	Notes
itb-kvm1.uchicago.edu	KVM host	int/ext	hosts storage systems: dCache, Hadoop, xrootd

The available disk on the MD1200 boxes is allocated in 3 equal parts (dCache, Hadoop, xrootd). The R610 will use one drive to run the host OS (SL 5.5) and the other 5 drives (RAID configuration - LVS) will be used for the virtual machines. There are 9 VMs, 3 each for dCache, Hadoop, xrootd

Storage (virtual) nodes include:

• 3 VMs for dcache pools
• 3 VMs for xrootd pools
• 3 VMs for hdfs pools

New machines 2 (8 R610)

Name	Primary Job	Cabling	Notes
itb-esx3.uchicago.edu	ESXi host	int/ext	vmware ESXi to compare VM performance (and replace esx1 and 2 eventually)
itb-kvm2.uchicago.edu	KVM host	int/ext	hosts headnode services
itb-kvm3.uchicago.edu	KVM host	int	hosts worker nodes for the ITB clusters
itb-kvm4.uchicago.edu	KVM host	int	hosts worker nodes for the ITB clusters
itb-c001.uchicago.edu	worker node	int	worker node for the ITB clusters
itb-c002.uchicago.edu	worker node	int	worker node for the ITB clusters
itb-dyn.uchicago.edu	cloud	int/ext	test dynamic provisioning - on top of KVM? nimbus?
unnamed7	R610	int/ext	unused

These R610 servers will be installed with:

• VMware ESXi on itb-esx3 (installation CD image available from vmware.com)
• SL 5.5 and KVM in itb-kvmX: one drive to run the host OS (SL 5.5) and the other 5 drives (RAID configuration - LVS) will be used for the virtual machines
• SL 5.5 like Tier 2/3 worker nodes on itb-cXXX: probably one drive to run the OS (SL 5.5) and the other 5 drives (RAID configuration - LVS) for the scratch space
• for itb-dyn check what is needed for dynamic provisioning (create/destroy VM dynamically); check Nimbus requirements (http://www.nimbusproject.org/ )

Headnode services for ITB and the virtual tier 3 include:

• PBS scheduler
• Condor headnode (collector/negotiator)
• GUMS
• ITB CE Headnode
• VTB CE Headnode
• Tier 3 CE
• Tier 3 SE (server varies)
• Tier 3 GUMS
• Tier 3 UI
• VO frontend (for Glidein-WMS job submission)
• Engagement submit host (with OSG MM submission)
• Campus router

We have currently 12 services in the list above. We may consider allocating one of the 2 unallocated R610 to host headnode services as well (itb-kvm6.uchicago.edu, with int/ext cabling).

We discussed having one management/service node (possibly one of the previous ITB nodes) to use as:

• login node
• NFSv4 server for home directories
• Ganglia monitor

Services "borrowed" from the Tier 2/Tier 3 infrastructure:

• Cobbler
• OSG_APP with ATLAS releases

IP Reallocation

IPs from the grid school machines have been reallocated as follows:

Old hostname	New hostname	Changed?	IP	Subnet	Gateway	Notes
gs-gratia.uchicago.edu	itbv-web.uchicago.edu		128.135.158.177	255.255.255.128	128.135.158.129		edit
gsu1.uchicago.edu	itbv-ce-pbs.uchicago.edu		128.135.158.176	255.255.255.128	128.135.158.129		edit
gs-bdii.uchicago.edu	vtbv-ce.uchicago.edu		128.135.158.175	255.255.255.128	128.135.158.129	Name change requested	edit
gs-ress.uchicago.edu	itbv-ce-condor.uchicago.edu		128.135.158.140	255.255.255.128	128.135.158.129	Name change requested	edit
gs-mm.uchicago.edu	itbv-bmx.uchicago.edu		128.135.158.139	255.255.255.128	128.135.158.129	Name change requested	edit

Testing

Interesting tests that we discussed.

IO testing

• IO comparison:
  • local IO (on host OS) vs
  • direct IO from VM (aka raw disks, access to the HW partition) vs
  • virtual disks on VM

Worker nodes

• Compare real WN with virtual WN

Dynamic Provisioning

Dynamic allocation tests (http://www.nimbusproject.org/)

Virtual machine infrastructure

Compare different virtualization software frameworks, i.e. ESXi vs KVM.

-- MarcoMambelli - 20 Jan 2011