 |
|
You are here: Foswiki>Main Web>UChicagoBrookhavenNetworkTesting (19 Mar 2009, RobGardner)Edit Attach
UChicagoBrookhavenNetworkTesting
- Summary
- UC-BNL virtual circuit
- Baseline iperf tests nettest10g.usatlas.bnl.gov (1500 byte MTU @ BNL)
- Baseline tests to 130.199.3.8 (1 stream)
- Baseline tests to 130.199.3.7 (8 streams)
- Baseline tests to 130.199.3.7 (16 streams)
- Purely local tests
- Jumbo frame testing plan
- BNL -to- UC tests from John
- Jumbo frame tests to nettest10g.usatlas.gov
- Jumbo frame tests to 130.199.3.7 (newman.bnl.gov)
- MWT2_UC to MWT2_IU test
Summary
- 18 March 2009
- Before switch to jumbo frames, performance to nettest10g.usatlas.bnl.gov 3.2 Gbps (8 streams)
- Single stream performance ~ 1.2 Gbps; in general, multiple streams needed with 1500 frame size
- Local performance for these UC hosts: ~ 5 Gbps (10G -to- 10G hosts, both at UC). This should be higher - need to look at NIC settings on these hosts.
- We get 6 Gbps from IU to UC (as good as local performance), but 4 Gbps (single stream) (jumbo frames end-to-end)
- Jumbo_frame_tests.rtf: John Bigrow's before/after results on 3/18/09
UC-BNL virtual circuit
rwg@uct2-s1:~$ traceroute nettest10g.usatlas.bnl.gov traceroute to nettest10g.usatlas.bnl.gov (192.12.15.25), 30 hops max, 38 byte packets 1 b07-147-200-v624 (128.135.158.217) 0.237 ms 0.182 ms 0.170 ms 2 south-to-ri.uchicago.net (10.4.247.73) 0.283 ms 0.222 ms 0.214 ms 3 10.4.247.224 (10.4.247.224) 8.654 ms 0.517 ms 0.833 ms 4 128.135.247.129 (128.135.247.129) 24.786 ms 24.670 ms 25.179 ms 5 nettest10g.usatlas.bnl.gov (192.12.15.25) 24.701 ms 24.672 ms 24.673 ms rwg@uct2-s1:~$
- small version:
Baseline iperf tests nettest10g.usatlas.bnl.gov (1500 byte MTU @ BNL)
Tracepath - 1500 ethernet frame size set by BNL
rwg@uct2-s1:~$ tracepath nettest10g.usatlas.bnl.gov
1: uct2-s1.uchicago.edu (128.135.158.221) 0.091ms pmtu 9000
1: b07-147-200-v624.uchicago.edu (128.135.158.217) 0.983ms
2: south-to-ri.uchicago.net (10.4.247.73) 1.001ms
3: 10.4.247.224 (10.4.247.224) asymm 2 1.009ms
4: 128.135.247.129 (128.135.247.129) asymm 3 26.177ms
5: 128.135.247.129 (128.135.247.129) asymm 3 25.711ms pmtu 1500
6: nettest10g.usatlas.bnl.gov (192.12.15.25) asymm 4 24.789ms reached
Resume: pmtu 1500 hops 6 back 4
rwg@uct2-s1:~$
Single stream iperf. Result: ~150 MB/s
rwg@uct2-s1:~$ iperf --client nettest10g.usatlas.bnl.gov --format M --port 20000 --time 1000 -i 5 -w4M ------------------------------------------------------------ Client connecting to nettest10g.usatlas.bnl.gov, TCP port 20000 TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte) ------------------------------------------------------------ [ 3] local 128.135.158.221 port 41478 connected with 192.12.15.25 port 20000 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 463 MBytes 92.7 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 729 MBytes 146 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 788 MBytes 158 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 15.0-20.0 sec 762 MBytes 152 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 20.0-25.0 sec 789 MBytes 158 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-26.1 sec 3712 MBytes 142 MBytes/secMultistream iperf. Result: ~400 MB/s (3.2 Gbps)
rwg@uct2-s1:~$ iperf --client nettest10g.usatlas.bnl.gov --format M --port 20000 -P8 --time 1000 -i 5 -w4M ------------------------------------------------------------ Client connecting to nettest10g.usatlas.bnl.gov, TCP port 20000 TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte) ------------------------------------------------------------ [ 9] local 128.135.158.221 port 41562 connected with 192.12.15.25 port 20000 [ 3] local 128.135.158.221 port 41563 connected with 192.12.15.25 port 20000 [ 10] local 128.135.158.221 port 41568 connected with 192.12.15.25 port 20000 [ 8] local 128.135.158.221 port 41564 connected with 192.12.15.25 port 20000 [ 5] local 128.135.158.221 port 41566 connected with 192.12.15.25 port 20000 [ 4] local 128.135.158.221 port 41567 connected with 192.12.15.25 port 20000 [ 7] local 128.135.158.221 port 41561 connected with 192.12.15.25 port 20000 [ 6] local 128.135.158.221 port 41565 connected with 192.12.15.25 port 20000 [ ID] Interval Transfer Bandwidth [ 10] 0.0- 5.0 sec 286 MBytes 57.3 MBytes/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0- 5.0 sec 275 MBytes 55.0 MBytes/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0- 5.0 sec 294 MBytes 58.8 MBytes/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0- 5.0 sec 267 MBytes 53.4 MBytes/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0- 5.0 sec 314 MBytes 62.8 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 290 MBytes 58.0 MBytes/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0- 5.0 sec 111 MBytes 22.1 MBytes/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0- 5.0 sec 27.8 MBytes 5.55 MBytes/sec [SUM] 0.0- 5.0 sec 1864 MBytes 373 MBytes/sec [ ID] Interval Transfer Bandwidth [ 9] 5.0-10.0 sec 342 MBytes 68.4 MBytes/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 290 MBytes 58.1 MBytes/sec [ ID] Interval Transfer Bandwidth [ 10] 5.0-10.0 sec 322 MBytes 64.5 MBytes/sec [ ID] Interval Transfer Bandwidth [ 5] 5.0-10.0 sec 297 MBytes 59.4 MBytes/sec [ ID] Interval Transfer Bandwidth [ 4] 5.0-10.0 sec 323 MBytes 64.6 MBytes/sec [ ID] Interval Transfer Bandwidth [ 7] 5.0-10.0 sec 197 MBytes 39.5 MBytes/sec [ ID] Interval Transfer Bandwidth [ 6] 5.0-10.0 sec 356 MBytes 71.2 MBytes/sec [ ID] Interval Transfer Bandwidth [ 8] 5.0-10.0 sec 36.8 MBytes 7.36 MBytes/sec [SUM] 5.0-10.0 sec 2165 MBytes 433 MBytes/sec
Baseline tests to 130.199.3.8 (1 stream)
- > 300 Mbps reached
rwg@uct2-s1:~$ iperf --client 130.199.3.7 --port 5010 --time 1000 -i 5 -w8M ------------------------------------------------------------ Client connecting to 130.199.3.7, TCP port 5010 TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte) ------------------------------------------------------------ [ 3] local 128.135.158.221 port 42856 connected with 130.199.3.7 port 5010 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 145 MBytes 244 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 162 MBytes 271 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 162 MBytes 271 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-18.0 sec 564 MBytes 262 Mbits/sec
Baseline tests to 130.199.3.7 (8 streams)
Wed Mar 18 05:38:24 CDT 2009- 2.8 Gbps reached
rwg@uct2-s1:~$ iperf --client 130.199.3.7 --port 5010 -P8 --time 1000 -i 5 -w8M ------------------------------------------------------------ Client connecting to 130.199.3.7, TCP port 5010 TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte) ------------------------------------------------------------ [ 10] local 128.135.158.221 port 42706 connected with 130.199.3.7 port 5010 [ 3] local 128.135.158.221 port 42699 connected with 130.199.3.7 port 5010 [ 5] local 128.135.158.221 port 42701 connected with 130.199.3.7 port 5010 [ 4] local 128.135.158.221 port 42700 connected with 130.199.3.7 port 5010 [ 6] local 128.135.158.221 port 42702 connected with 130.199.3.7 port 5010 [ 7] local 128.135.158.221 port 42703 connected with 130.199.3.7 port 5010 [ 8] local 128.135.158.221 port 42704 connected with 130.199.3.7 port 5010 [ 9] local 128.135.158.221 port 42705 connected with 130.199.3.7 port 5010 [ ID] Interval Transfer Bandwidth [ 10] 0.0- 5.0 sec 114 MBytes 191 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0- 5.0 sec 261 MBytes 439 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0- 5.0 sec 223 MBytes 374 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0- 5.0 sec 227 MBytes 381 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0- 5.0 sec 224 MBytes 375 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0- 5.0 sec 210 MBytes 352 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0- 5.0 sec 247 MBytes 415 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 101 MBytes 170 Mbits/sec [SUM] 0.0- 5.0 sec 1.57 GBytes 2.70 Gbits/sec [ ID] Interval Transfer Bandwidth [ 10] 5.0-10.0 sec 189 MBytes 318 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 182 MBytes 305 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 5.0-10.0 sec 278 MBytes 467 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 5.0-10.0 sec 256 MBytes 430 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 5.0-10.0 sec 255 MBytes 428 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 5.0-10.0 sec 188 MBytes 316 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 5.0-10.0 sec 130 MBytes 219 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 5.0-10.0 sec 190 MBytes 319 Mbits/sec [SUM] 5.0-10.0 sec 1.63 GBytes 2.80 Gbits/sec [ ID] Interval Transfer Bandwidth [ 10] 10.0-15.0 sec 155 MBytes 260 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 190 MBytes 318 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 10.0-15.0 sec 239 MBytes 401 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 10.0-15.0 sec 214 MBytes 359 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 10.0-15.0 sec 268 MBytes 449 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 10.0-15.0 sec 237 MBytes 398 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 10.0-15.0 sec 188 MBytes 316 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 10.0-15.0 sec 194 MBytes 325 Mbits/sec [SUM] 10.0-15.0 sec 1.64 GBytes 2.82 Gbits/sec
Baseline tests to 130.199.3.7 (16 streams)
- 4.32 Gbps reached
rwg@uct2-s1:~$ iperf --client 130.199.3.7 --port 5010 -P16 --time 1000 -i 5 -w8M ------------------------------------------------------------ Client connecting to 130.199.3.7, TCP port 5010 TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte) ------------------------------------------------------------ [ 18] local 128.135.158.221 port 42959 connected with 130.199.3.7 port 5010 [ 3] local 128.135.158.221 port 42944 connected with 130.199.3.7 port 5010 [ 5] local 128.135.158.221 port 42946 connected with 130.199.3.7 port 5010 [ 6] local 128.135.158.221 port 42947 connected with 130.199.3.7 port 5010 [ 4] local 128.135.158.221 port 42945 connected with 130.199.3.7 port 5010 [ 7] local 128.135.158.221 port 42948 connected with 130.199.3.7 port 5010 [ 8] local 128.135.158.221 port 42949 connected with 130.199.3.7 port 5010 [ 9] local 128.135.158.221 port 42950 connected with 130.199.3.7 port 5010 [ 10] local 128.135.158.221 port 42951 connected with 130.199.3.7 port 5010 [ 11] local 128.135.158.221 port 42952 connected with 130.199.3.7 port 5010 [ 12] local 128.135.158.221 port 42953 connected with 130.199.3.7 port 5010 [ 14] local 128.135.158.221 port 42955 connected with 130.199.3.7 port 5010 [ 13] local 128.135.158.221 port 42954 connected with 130.199.3.7 port 5010 [ 15] local 128.135.158.221 port 42956 connected with 130.199.3.7 port 5010 [ 17] local 128.135.158.221 port 42958 connected with 130.199.3.7 port 5010 [ 16] local 128.135.158.221 port 42957 connected with 130.199.3.7 port 5010 [ ID] Interval Transfer Bandwidth [ 9] 0.0- 5.0 sec 127 MBytes 214 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 0.0- 5.0 sec 126 MBytes 211 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 0.0- 5.0 sec 131 MBytes 220 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 0.0- 5.0 sec 121 MBytes 203 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0- 5.0 sec 120 MBytes 201 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 164 MBytes 276 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 0.0- 5.0 sec 61.5 MBytes 103 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0- 5.0 sec 149 MBytes 250 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0- 5.0 sec 113 MBytes 190 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0- 5.0 sec 104 MBytes 175 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0- 5.0 sec 126 MBytes 211 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 0.0- 5.0 sec 106 MBytes 178 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0- 5.0 sec 120 MBytes 201 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 0.0- 5.0 sec 33.1 MBytes 55.6 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 0.0- 5.0 sec 129 MBytes 217 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 0.0- 5.0 sec 67.9 MBytes 114 Mbits/sec [SUM] 0.0- 5.0 sec 1.76 GBytes 3.02 Gbits/sec [ ID] Interval Transfer Bandwidth [ 18] 5.0-10.0 sec 221 MBytes 370 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 5.0-10.0 sec 130 MBytes 218 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 5.0-10.0 sec 138 MBytes 231 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 5.0-10.0 sec 191 MBytes 320 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 5.0-10.0 sec 235 MBytes 395 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 5.0-10.0 sec 173 MBytes 291 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 5.0-10.0 sec 219 MBytes 367 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 5.0-10.0 sec 183 MBytes 307 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 5.0-10.0 sec 68.4 MBytes 115 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 5.0-10.0 sec 176 MBytes 296 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 5.0-10.0 sec 63.1 MBytes 106 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 5.0-10.0 sec 155 MBytes 261 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 212 MBytes 356 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 5.0-10.0 sec 139 MBytes 234 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 5.0-10.0 sec 175 MBytes 294 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 5.0-10.0 sec 30.6 MBytes 51.4 Mbits/sec [SUM] 5.0-10.0 sec 2.45 GBytes 4.21 Gbits/sec
Purely local tests
uct2-s1.uchicago.edu and uct2-s3.uchicago.edu are 10G hosts. There is an iperf server running on uct2-s3. They are connected by a Dell Power Connect 6248 switch. Jumbo frames on this subnet (in the switch, MTU set to 9216).
rwg@uct2-s1:~$ tracepath uct2-s3.uchicago.edu
1: uct2-s1.uchicago.edu (128.135.158.221) 0.106ms pmtu 9000
1: uct2-s3.uchicago.edu (128.135.158.223) 0.115ms reached
Resume: pmtu 9000 hops 1 back 1
rwg@uct2-s1:~$
Single stream iperf: 5 Gbps
rwg@uct2-s1:~$ iperf --client uct2-s3.uchicago.edu --port 5001 --time 1000 -i 5 -w8M ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte) ------------------------------------------------------------ [ 3] local 128.135.158.221 port 42375 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 2.37 GBytes 4.07 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 2.93 GBytes 5.03 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 2.93 GBytes 5.03 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-18.0 sec 9.99 GBytes 4.77 Gbits/sec
Multiple streams(8): 5.7 Gbps
rwg@uct2-s1:~$ iperf --client uct2-s3.uchicago.edu --port 5001 -P8 --time 1000 -i 5 -w8M ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 16.0 MByte (WARNING: requested 8.00 MByte) ------------------------------------------------------------ [ 4] local 128.135.158.221 port 42440 connected with 128.135.158.223 port 5001 [ 5] local 128.135.158.221 port 42441 connected with 128.135.158.223 port 5001 [ 6] local 128.135.158.221 port 42442 connected with 128.135.158.223 port 5001 [ 8] local 128.135.158.221 port 42444 connected with 128.135.158.223 port 5001 [ 3] local 128.135.158.221 port 42439 connected with 128.135.158.223 port 5001 [ 9] local 128.135.158.221 port 42445 connected with 128.135.158.223 port 5001 [ 7] local 128.135.158.221 port 42443 connected with 128.135.158.223 port 5001 [ 10] local 128.135.158.221 port 42446 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 6] 0.0- 5.0 sec 628 MBytes 1.05 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 233 MBytes 390 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0- 5.0 sec 662 MBytes 1.11 Gbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0- 5.0 sec 228 MBytes 383 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0- 5.0 sec 544 MBytes 913 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0- 5.0 sec 616 MBytes 1.03 Gbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0- 5.0 sec 338 MBytes 566 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0- 5.0 sec 238 MBytes 399 Mbits/sec [SUM] 0.0- 5.0 sec 3.40 GBytes 5.85 Gbits/sec [ ID] Interval Transfer Bandwidth [ 6] 5.0-10.0 sec 694 MBytes 1.16 Gbits/sec [ ID] Interval Transfer Bandwidth [ 8] 5.0-10.0 sec 698 MBytes 1.17 Gbits/sec [ ID] Interval Transfer Bandwidth [ 9] 5.0-10.0 sec 197 MBytes 330 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 5.0-10.0 sec 703 MBytes 1.18 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 195 MBytes 327 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 5.0-10.0 sec 195 MBytes 328 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 5.0-10.0 sec 504 MBytes 845 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 5.0-10.0 sec 199 MBytes 335 Mbits/sec [SUM] 5.0-10.0 sec 3.31 GBytes 5.68 Gbits/sec [ ID] Interval Transfer Bandwidth [ 7] 10.0-15.0 sec 713 MBytes 1.20 Gbits/sec [ ID] Interval Transfer Bandwidth [ 10] 10.0-15.0 sec 188 MBytes 315 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 10.0-15.0 sec 706 MBytes 1.18 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 10.0-15.0 sec 511 MBytes 857 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 10.0-15.0 sec 707 MBytes 1.19 Gbits/sec [ ID] Interval Transfer Bandwidth [ 4] 10.0-15.0 sec 183 MBytes 307 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 10.0-15.0 sec 185 MBytes 310 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 193 MBytes 324 Mbits/sec [SUM] 10.0-15.0 sec 3.31 GBytes 5.68 Gbits/sec [ ID] Interval Transfer Bandwidth [ 8] 15.0-20.0 sec 710 MBytes 1.19 Gbits/sec [ ID] Interval Transfer Bandwidth [ 7] 15.0-20.0 sec 719 MBytes 1.21 Gbits/sec [ ID] Interval Transfer Bandwidth [ 4] 15.0-20.0 sec 186 MBytes 313 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 15.0-20.0 sec 701 MBytes 1.18 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 15.0-20.0 sec 181 MBytes 304 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 15.0-20.0 sec 511 MBytes 857 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 15.0-20.0 sec 184 MBytes 308 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 15.0-20.0 sec 189 MBytes 317 Mbits/sec [SUM] 15.0-20.0 sec 3.30 GBytes 5.67 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-21.2 sec 2.14 GBytes 868 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-21.2 sec 2.88 GBytes 1.17 Gbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-21.2 sec 842 MBytes 333 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-21.2 sec 2.86 GBytes 1.15 Gbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-21.3 sec 2.84 GBytes 1.15 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-21.3 sec 850 MBytes 335 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-21.3 sec 955 MBytes 376 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-21.4 sec 852 MBytes 335 Mbits/sec [SUM] 0.0-21.4 sec 14.1 GBytes 5.69 Gbits/sec
Jumbo frame testing plan
From: Bigrow, John F [mailto:big@bnl.gov] Sent: Friday, March 13, 2009 2:15 PM To: Ernst, Michael; Rob Gardner Cc: McKee, Shawn; Burstein, Frank; O' Connor, Michael P (ESNET); Hockett, Roy; Yongrui Tang; Anita Nikolich; Ron Rusnak; Yu, Dantong; O' Connor, Michael P (ESNET) Subject: RE: [Usatlas-grid-l] MTU setting at bnlsite-bnlmr1.es.net(198.124.216.178) Good Afternoon Everyone; Sorry I didn’t get a chance to reply earlier but I’m working With ES net on our near term site upgrades. Thanks for your comments and insights they were quite helpful. Can I propose we adopt the following methodology to test this capability? One morning next week, I will activate the jumbo frame features. I have to notify the BNL community about network changes. This is an internal policy. I’m thinking Wednesday morning at 09:00 EST. I’ll activate the jumbo MTU on both the 192.12.15.0/24 and U Chicago links. This will allow us to test access into this network here at BNL. In the interim, I have A 10 gig host at IP 130.199.3.7 (newmon.bnl.gov). This segment is the first entry into the BNL normal campus LAN. This host will keep it’s current 1500 byte MTU as will that segment. I currently have an iperf server running (iperf -s -p 5010) there on port 5010. Dantong Yu has a 10gig box on our atlas segment 192.12.15.25 (nettest10g.usatlas.bnl.gov) that we can also run iperf tests to. That iperf is port 20000. When I make the switch next week to support jumbo frames, he can set his platform to 9k mtu and we can test against these hosts again. This will give us a baseline set of measurements. At the end of the day, I can set the MTU back to 1500 if desired. If this plan sounds ok with everyone I’ll make an internal announcement informing the BNL community about the changes for Wednesday next week. I’d like to run the same tests to your systems from BNL. Also on at the University of Chicago if that’s available to. Can you email me back with the hosts and services on your side and if the day and time is ok with you? Disclaimer.: This is a production network so if this testing impacts or causes other performance issues I will have to undo these changes. Thanks Have a good weekend everyone John F. BigrowBNL -to- UC tests from John
Hi Rob From the last two days some random hits from me back to you!!! Keep in mind, I think this link is BW limited to either 1 or 2 GB.bigrow@newmon:~$ traceroute uct2-s3.uchicago.edu traceroute to uct2-s3.uchicago.edu (128.135.158.223), 30 hops max, 40 byte packets 1 amon (130.199.3.124) 0.247 ms 0.156 ms 0.147 ms 2 mutt (130.199.3.224) 0.229 ms 0.183 ms 0.178 ms 3 128.135.247.130 (128.135.247.130) 24.831 ms 24.711 ms 24.674 ms 4 * * * 5 uct2-s3.uchicago.edu (128.135.158.223) 24.581 ms 24.601 ms 24.575 ms bigrow@newmon:~$ bigrow@newmon:~$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 131072 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 10] local 130.199.3.7 port 41339 connected with 128.135.158.223 port 5001 [ 3] local 130.199.3.7 port 41332 connected with 128.135.158.223 port 5001 [ 9] local 130.199.3.7 port 41334 connected with 128.135.158.223 port 5001 [ 6] local 130.199.3.7 port 41335 connected with 128.135.158.223 port 5001 [ 8] local 130.199.3.7 port 41337 connected with 128.135.158.223 port 5001 [ 5] local 130.199.3.7 port 41338 connected with 128.135.158.223 port 5001 [ 4] local 130.199.3.7 port 41333 connected with 128.135.158.223 port 5001 [ 7] local 130.199.3.7 port 41336 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 8] 0.0-10.1 sec 663 MBytes 551 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-10.1 sec 434 MBytes 359 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.2 sec 457 MBytes 377 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.2 sec 278 MBytes 229 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.5 sec 455 MBytes 365 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-10.5 sec 457 MBytes 364 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-11.0 sec 257 MBytes 195 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.6 sec 281 MBytes 203 Mbits/sec [SUM] 0.0-11.6 sec 3.21 GBytes 2.37 Gbits/sec bigrow@newmon:~$ [test@nettest10g ~]$ traceroute uct2-s3.uchicago.edu traceroute to uct2-s3.uchicago.edu (128.135.158.223), 30 hops max, 40 byte packets 1 mutt.usatlas.bnl.gov (192.12.15.224) 0.357 ms 0.450 ms 0.436 ms 2 128.135.247.130 (128.135.247.130) 25.478 ms 25.611 ms 25.723 ms 3 uct2-s3.uchicago.edu (128.135.158.223) 24.639 ms 24.750 ms 24.735 ms [test@nettest10g ~]$ [test@nettest10g ~]$ traceroute uct2-s3.uchicago.edu traceroute to uct2-s3.uchicago.edu (128.135.158.223), 30 hops max, 40 byte packets 1 mutt.usatlas.bnl.gov (192.12.15.224) 0.357 ms 0.450 ms 0.436 ms 2 128.135.247.130 (128.135.247.130) 25.478 ms 25.611 ms 25.723 ms 3 uct2-s3.uchicago.edu (128.135.158.223) 24.639 ms 24.750 ms 24.735 ms [test@nettest10g ~]$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 13107 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 10] local 192.12.15.25 port 38458 connected with 128.135.158.223 port 5001 [ 4] local 192.12.15.25 port 38452 connected with 128.135.158.223 port 5001 [ 5] local 192.12.15.25 port 38453 connected with 128.135.158.223 port 5001 [ 6] local 192.12.15.25 port 38454 connected with 128.135.158.223 port 5001 [ 7] local 192.12.15.25 port 38455 connected with 128.135.158.223 port 5001 [ 8] local 192.12.15.25 port 38456 connected with 128.135.158.223 port 5001 [ 9] local 192.12.15.25 port 38457 connected with 128.135.158.223 port 5001 [ 3] local 192.12.15.25 port 38451 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.2 sec 191 MBytes 158 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.5 sec 256 MBytes 204 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.6 sec 256 MBytes 202 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-10.9 sec 299 MBytes 231 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-11.2 sec 212 MBytes 159 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-11.9 sec 147 MBytes 103 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-12.6 sec 148 MBytes 98.1 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-13.2 sec 236 MBytes 150 Mbits/sec [SUM] 0.0-13.2 sec 1.70 GBytes 1.11 Gbits/sec [test@nettest10g ~]$ [test@nettest10g ~]$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 13107 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 4] local 192.12.15.25 port 34509 connected with 128.135.158.223 port 5001 [ 5] local 192.12.15.25 port 34510 connected with 128.135.158.223 port 5001 [ 6] local 192.12.15.25 port 34511 connected with 128.135.158.223 port 5001 [ 7] local 192.12.15.25 port 34512 connected with 128.135.158.223 port 5001 [ 8] local 192.12.15.25 port 34513 connected with 128.135.158.223 port 5001 [ 10] local 192.12.15.25 port 34515 connected with 128.135.158.223 port 5001 [ 3] local 192.12.15.25 port 34508 connected with 128.135.158.223 port 5001 [ 9] local 192.12.15.25 port 34514 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.2 sec 191 MBytes 157 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-10.2 sec 191 MBytes 158 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.4 sec 212 MBytes 171 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.5 sec 191 MBytes 153 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-10.6 sec 233 MBytes 183 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-10.8 sec 190 MBytes 148 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.8 sec 191 MBytes 149 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-12.2 sec 191 MBytes 131 Mbits/sec [SUM] 0.0-12.2 sec 1.55 GBytes 1.09 Gbits/sec [test@nettest10g ~]$ bigrow@newmon:~$ traceroute uct2-s3.uchicago.edu traceroute to uct2-s3.uchicago.edu (128.135.158.223), 30 hops max, 40 byte packets 1 amon (130.199.3.124) 0.247 ms 0.156 ms 0.147 ms 2 mutt (130.199.3.224) 0.229 ms 0.183 ms 0.178 ms 3 128.135.247.130 (128.135.247.130) 24.831 ms 24.711 ms 24.674 ms 4 * * * 5 uct2-s3.uchicago.edu (128.135.158.223) 24.581 ms 24.601 ms 24.575 ms bigrow@newmon:~$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 131072 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 10] local 130.199.3.7 port 41339 connected with 128.135.158.223 port 5001 [ 3] local 130.199.3.7 port 41332 connected with 128.135.158.223 port 5001 [ 9] local 130.199.3.7 port 41334 connected with 128.135.158.223 port 5001 [ 6] local 130.199.3.7 port 41335 connected with 128.135.158.223 port 5001 [ 8] local 130.199.3.7 port 41337 connected with 128.135.158.223 port 5001 [ 5] local 130.199.3.7 port 41338 connected with 128.135.158.223 port 5001 [ 4] local 130.199.3.7 port 41333 connected with 128.135.158.223 port 5001 [ 7] local 130.199.3.7 port 41336 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 8] 0.0-10.1 sec 663 MBytes 551 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-10.1 sec 434 MBytes 359 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.2 sec 457 MBytes 377 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.2 sec 278 MBytes 229 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.5 sec 455 MBytes 365 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-10.5 sec 457 MBytes 364 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-11.0 sec 257 MBytes 195 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-11.6 sec 281 MBytes 203 Mbits/sec [SUM] 0.0-11.6 sec 3.21 GBytes 2.37 Gbits/sec bigrow@newmon:~$ bigrow@newmon:~$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 131072 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 10] local 130.199.3.7 port 51167 connected with 128.135.158.223 port 5001 [ 4] local 130.199.3.7 port 51161 connected with 128.135.158.223 port 5001 [ 3] local 130.199.3.7 port 51162 connected with 128.135.158.223 port 5001 [ 6] local 130.199.3.7 port 51163 connected with 128.135.158.223 port 5001 [ 7] local 130.199.3.7 port 51164 connected with 128.135.158.223 port 5001 [ 8] local 130.199.3.7 port 51165 connected with 128.135.158.223 port 5001 [ 5] local 130.199.3.7 port 51160 connected with 128.135.158.223 port 5001 [ 9] local 130.199.3.7 port 51166 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.4 sec 388 MBytes 314 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.4 sec 567 MBytes 456 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-10.5 sec 481 MBytes 386 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-10.5 sec 479 MBytes 383 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.7 sec 213 MBytes 167 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.8 sec 235 MBytes 182 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-11.2 sec 150 MBytes 111 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-11.4 sec 192 MBytes 140 Mbits/sec [SUM] 0.0-11.4 sec 2.64 GBytes 1.98 Gbits/sec bigrow@newmon:~$ bigrow@newmon:~$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 131072 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 10] local 130.199.3.7 port 36507 connected with 128.135.158.223 port 5001 [ 3] local 130.199.3.7 port 36500 connected with 128.135.158.223 port 5001 [ 5] local 130.199.3.7 port 36502 connected with 128.135.158.223 port 5001 [ 6] local 130.199.3.7 port 36503 connected with 128.135.158.223 port 5001 [ 8] local 130.199.3.7 port 36505 connected with 128.135.158.223 port 5001 [ 9] local 130.199.3.7 port 36506 connected with 128.135.158.223 port 5001 [ 4] local 130.199.3.7 port 36501 connected with 128.135.158.223 port 5001 [ 7] local 130.199.3.7 port 36504 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.1 sec 595 MBytes 493 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-10.2 sec 552 MBytes 456 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.2 sec 686 MBytes 563 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.4 sec 191 MBytes 154 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.4 sec 526 MBytes 424 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-10.5 sec 322 MBytes 256 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-11.5 sec 191 MBytes 139 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-11.5 sec 127 MBytes 92.2 Mbits/sec [SUM] 0.0-11.5 sec 3.12 GBytes 2.32 Gbits/sec bigrow@newmon:~$ iperf -c nettest10g.usatlas.bnl.gov -p 20000 -P 5 -w 16M -l 131072 [test@nettest10g ~]$ iperf -c uct2-s3.uchicago.edu -w 32M -P8 -l 13107 ------------------------------------------------------------ Client connecting to uct2-s3.uchicago.edu, TCP port 5001 TCP window size: 64.0 MByte (WARNING: requested 32.0 MByte) ------------------------------------------------------------ [ 4] local 192.12.15.25 port 33140 connected with 128.135.158.223 port 5001 [ 5] local 192.12.15.25 port 33141 connected with 128.135.158.223 port 5001 [ 6] local 192.12.15.25 port 33142 connected with 128.135.158.223 port 5001 [ 7] local 192.12.15.25 port 33143 connected with 128.135.158.223 port 5001 [ 8] local 192.12.15.25 port 33144 connected with 128.135.158.223 port 5001 [ 10] local 192.12.15.25 port 33146 connected with 128.135.158.223 port 5001 [ 3] local 192.12.15.25 port 33139 connected with 128.135.158.223 port 5001 [ 9] local 192.12.15.25 port 33145 connected with 128.135.158.223 port 5001 [ ID] Interval Transfer Bandwidth [ 10] 0.0-10.2 sec 170 MBytes 140 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.4 sec 148 MBytes 119 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.5 sec 147 MBytes 118 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-10.5 sec 211 MBytes 169 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-11.0 sec 169 MBytes 130 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-11.0 sec 169 MBytes 129 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-12.2 sec 148 MBytes 102 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-12.3 sec 169 MBytes 115 Mbits/sec [SUM] 0.0-12.3 sec 1.30 GBytes 905 Mbits/sec [test@nettest10g ~]$
Jumbo frame tests to nettest10g.usatlas.gov
Wed Mar 18 09:10:44 CDT 2009
rwg@uct2-s1:~$ tracepath nettest10g.usatlas.bnl.gov
1: uct2-s1.uchicago.edu (128.135.158.221) 0.076ms pmtu 9000
1: b07-147-200-v624.uchicago.edu (128.135.158.217) 0.910ms
2: south-to-ri.uchicago.net (10.4.247.73) 1.085ms
3: 10.4.247.224 (10.4.247.224) asymm 2 1.106ms
4: 128.135.247.129 (128.135.247.129) asymm 3 25.634ms
5: nettest10g.usatlas.bnl.gov (192.12.15.25) asymm 4 25.226ms reached
Resume: pmtu 9000 hops 5 back 4
rwg@uct2-s1:~$
Result: very poor, something's wrong:
rwg@uct2-s1:~$ iperf --client nettest10g.usatlas.bnl.gov --port 20000 -P16 --time 1000 -i 5 -w4M [SUM] 0.0-29.7 sec 206 MBytes 58.4 Mbits/sec
Jumbo frame tests to 130.199.3.7 (newman.bnl.gov)
rwg@uct2-s1:~$ date
Wed Mar 18 09:06:38 CDT 2009
rwg@uct2-s1:~$ tracepath newmon.bnl.gov
1: uct2-s1.uchicago.edu (128.135.158.221) 1.020ms pmtu 9000
1: b07-147-200-v624.uchicago.edu (128.135.158.217) 1.202ms
2: south-to-ri.uchicago.net (10.4.247.73) 1.675ms
3: 10.4.247.224 (10.4.247.224) asymm 2 0.999ms
4: 128.135.247.129 (128.135.247.129) asymm 3 25.660ms
5: 128.135.247.129 (128.135.247.129) asymm 3 25.673ms pmtu 1500
6: newmon.bnl.gov (130.199.3.7) asymm 5 24.661ms reached
Resume: pmtu 1500 hops 6 back 5
rwg@uct2-s1:~$
Are jumbo frames enabled in BNL's local network?
rwg@uct2-s1:~$ iperf --client 130.199.3.7 --port 5010 -P16 --time 1000 -i 5 -w8M -snip- [SUM] 30.0-35.0 sec 2.58 GBytes 4.43 Gbits/sec -snip- [SUM] 0.0-39.1 sec 14.0 GBytes 3.09 Gbits/secLooks about the same as before IT SHOULD BE.. frame size was unchanged.
MWT2_UC to MWT2_IU test
16 streams
- Result: we get 6 Gbps from IU to UC (as good as local performance)
rwg@uct2-s1:~$ tracepath iut2-s1.iu.edu 1: uct2-s1.uchicago.edu (128.135.158.221) 0.086ms pmtu 9000 1: b07-147-200-v624.uchicago.edu (128.135.158.217) 0.930ms 2: south-to-ri.uchicago.net (10.4.247.73) 0.980ms 3: 10.4.247.224 (10.4.247.224) asymm 2 1.045ms 4: omnipop-mcleod-10g-router.uchicago.edu (128.135.247.125) 1.248ms 5: 149.165.254.2 (149.165.254.2) asymm 4 5.705ms 6: 149.165.254.250 (149.165.254.250) asymm 4 6.507ms 7: no reply 8: no reply rwg@uct2-s1:~$ iperf --client iut2-s1.iu.edu --port 443 -P16 --time 1000 -i 5 -w4M ------------------------------------------------------------ Client connecting to iut2-s1.iu.edu, TCP port 443 TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte) ------------------------------------------------------------ [ 13] local 128.135.158.221 port 49772 connected with 149.165.225.225 port 443 [ 5] local 128.135.158.221 port 49757 connected with 149.165.225.225 port 443 [ 16] local 128.135.158.221 port 49758 connected with 149.165.225.225 port 443 [ 17] local 128.135.158.221 port 49759 connected with 149.165.225.225 port 443 [ 11] local 128.135.158.221 port 49760 connected with 149.165.225.225 port 443 [ 6] local 128.135.158.221 port 49761 connected with 149.165.225.225 port 443 [ 4] local 128.135.158.221 port 49762 connected with 149.165.225.225 port 443 [ 12] local 128.135.158.221 port 49765 connected with 149.165.225.225 port 443 [ 9] local 128.135.158.221 port 49764 connected with 149.165.225.225 port 443 [ 10] local 128.135.158.221 port 49766 connected with 149.165.225.225 port 443 [ 14] local 128.135.158.221 port 49767 connected with 149.165.225.225 port 443 [ 15] local 128.135.158.221 port 49768 connected with 149.165.225.225 port 443 [ 18] local 128.135.158.221 port 49769 connected with 149.165.225.225 port 443 [ 3] local 128.135.158.221 port 49771 connected with 149.165.225.225 port 443 [ 7] local 128.135.158.221 port 49770 connected with 149.165.225.225 port 443 [ 8] local 128.135.158.221 port 49763 connected with 149.165.225.225 port 443 [ ID] Interval Transfer Bandwidth [ 13] 0.0- 5.0 sec 186 MBytes 312 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0- 5.0 sec 286 MBytes 479 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0- 5.0 sec 265 MBytes 444 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0- 5.0 sec 339 MBytes 569 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 0.0- 5.0 sec 193 MBytes 324 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 0.0- 5.0 sec 165 MBytes 277 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0- 5.0 sec 233 MBytes 391 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0- 5.0 sec 293 MBytes 492 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 0.0- 5.0 sec 174 MBytes 292 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0- 5.0 sec 327 MBytes 548 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 0.0- 5.0 sec 124 MBytes 208 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 250 MBytes 420 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 0.0- 5.0 sec 198 MBytes 332 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0- 5.0 sec 203 MBytes 341 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 0.0- 5.0 sec 304 MBytes 511 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 0.0- 5.0 sec 79.4 MBytes 133 Mbits/sec [SUM] 0.0- 5.0 sec 3.54 GBytes 6.07 Gbits/sec [ ID] Interval Transfer Bandwidth [ 13] 5.0-10.0 sec 215 MBytes 360 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 5.0-10.0 sec 219 MBytes 368 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 5.0-10.0 sec 294 MBytes 494 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 5.0-10.0 sec 170 MBytes 285 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 5.0-10.0 sec 289 MBytes 485 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 5.0-10.0 sec 285 MBytes 479 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 5.0-10.0 sec 330 MBytes 553 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 235 MBytes 394 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 5.0-10.0 sec 253 MBytes 424 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 5.0-10.0 sec 144 MBytes 242 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 5.0-10.0 sec 236 MBytes 395 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 5.0-10.0 sec 298 MBytes 499 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 5.0-10.0 sec 250 MBytes 419 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 5.0-10.0 sec 301 MBytes 505 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 5.0-10.0 sec 223 MBytes 373 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 5.0-10.0 sec 134 MBytes 225 Mbits/sec [SUM] 5.0-10.0 sec 3.78 GBytes 6.50 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 10.0-15.0 sec 144 MBytes 241 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 10.0-15.0 sec 150 MBytes 251 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 10.0-15.0 sec 376 MBytes 630 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 10.0-15.0 sec 186 MBytes 312 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 10.0-15.0 sec 186 MBytes 312 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 10.0-15.0 sec 258 MBytes 433 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 10.0-15.0 sec 204 MBytes 342 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 307 MBytes 514 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 10.0-15.0 sec 274 MBytes 459 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 10.0-15.0 sec 170 MBytes 286 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 10.0-15.0 sec 236 MBytes 396 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 10.0-15.0 sec 182 MBytes 306 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 10.0-15.0 sec 251 MBytes 422 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 10.0-15.0 sec 371 MBytes 623 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 10.0-15.0 sec 183 MBytes 308 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 10.0-15.0 sec 323 MBytes 542 Mbits/sec [SUM] 10.0-15.0 sec 3.71 GBytes 6.38 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 15.0-20.0 sec 316 MBytes 530 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 15.0-20.0 sec 105 MBytes 177 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 15.0-20.0 sec 147 MBytes 247 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 15.0-20.0 sec 223 MBytes 375 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 15.0-20.0 sec 297 MBytes 499 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 15.0-20.0 sec 327 MBytes 549 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 15.0-20.0 sec 228 MBytes 383 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 15.0-20.0 sec 136 MBytes 227 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 15.0-20.0 sec 225 MBytes 378 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 15.0-20.0 sec 148 MBytes 249 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 15.0-20.0 sec 345 MBytes 579 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 15.0-20.0 sec 233 MBytes 391 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 15.0-20.0 sec 144 MBytes 241 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 15.0-20.0 sec 273 MBytes 458 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 15.0-20.0 sec 327 MBytes 548 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 15.0-20.0 sec 338 MBytes 568 Mbits/sec [SUM] 15.0-20.0 sec 3.72 GBytes 6.40 Gbits/sec [ ID] Interval Transfer Bandwidth [ 14] 20.0-25.0 sec 305 MBytes 512 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 20.0-25.0 sec 307 MBytes 514 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 20.0-25.0 sec 232 MBytes 390 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 20.0-25.0 sec 234 MBytes 392 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 20.0-25.0 sec 236 MBytes 395 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 20.0-25.0 sec 221 MBytes 371 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 20.0-25.0 sec 192 MBytes 322 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 20.0-25.0 sec 155 MBytes 260 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 20.0-25.0 sec 364 MBytes 611 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 20.0-25.0 sec 290 MBytes 486 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 20.0-25.0 sec 180 MBytes 303 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 20.0-25.0 sec 266 MBytes 446 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 20.0-25.0 sec 164 MBytes 276 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 20.0-25.0 sec 182 MBytes 306 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 20.0-25.0 sec 182 MBytes 306 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 20.0-25.0 sec 213 MBytes 357 Mbits/sec [SUM] 20.0-25.0 sec 3.64 GBytes 6.25 Gbits/sec [ ID] Interval Transfer Bandwidth [ 13] 25.0-30.0 sec 194 MBytes 326 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 25.0-30.0 sec 200 MBytes 335 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 25.0-30.0 sec 194 MBytes 326 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 25.0-30.0 sec 267 MBytes 448 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 25.0-30.0 sec 196 MBytes 328 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 25.0-30.0 sec 190 MBytes 318 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 25.0-30.0 sec 236 MBytes 395 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 25.0-30.0 sec 208 MBytes 349 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 25.0-30.0 sec 286 MBytes 479 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 25.0-30.0 sec 229 MBytes 383 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 25.0-30.0 sec 256 MBytes 430 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 25.0-30.0 sec 259 MBytes 435 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 25.0-30.0 sec 208 MBytes 349 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 25.0-30.0 sec 242 MBytes 405 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 25.0-30.0 sec 264 MBytes 443 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 25.0-30.0 sec 211 MBytes 354 Mbits/sec [SUM] 25.0-30.0 sec 3.55 GBytes 6.11 Gbits/sec [ ID] Interval Transfer Bandwidth [ 5] 30.0-35.0 sec 282 MBytes 473 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 30.0-35.0 sec 185 MBytes 311 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 30.0-35.0 sec 180 MBytes 302 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 30.0-35.0 sec 193 MBytes 324 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 30.0-35.0 sec 218 MBytes 366 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 30.0-35.0 sec 181 MBytes 303 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 30.0-35.0 sec 324 MBytes 543 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 30.0-35.0 sec 223 MBytes 373 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 30.0-35.0 sec 239 MBytes 401 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 30.0-35.0 sec 246 MBytes 413 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 30.0-35.0 sec 193 MBytes 324 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 30.0-35.0 sec 261 MBytes 439 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 30.0-35.0 sec 198 MBytes 333 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 30.0-35.0 sec 175 MBytes 294 Mbits/sec [ ID] Interval Transfer Bandwidth [ 13] 30.0-35.0 sec 196 MBytes 329 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 30.0-35.0 sec 198 MBytes 331 Mbits/sec [SUM] 30.0-35.0 sec 3.41 GBytes 5.86 Gbits/sec [ ID] Interval Transfer Bandwidth [ 13] 35.0-40.0 sec 195 MBytes 327 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 35.0-40.0 sec 282 MBytes 473 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 35.0-40.0 sec 296 MBytes 497 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 35.0-40.0 sec 198 MBytes 332 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 35.0-40.0 sec 272 MBytes 456 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 35.0-40.0 sec 295 MBytes 496 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 35.0-40.0 sec 239 MBytes 401 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 35.0-40.0 sec 174 MBytes 292 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 35.0-40.0 sec 174 MBytes 292 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 35.0-40.0 sec 210 MBytes 353 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 35.0-40.0 sec 247 MBytes 415 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 35.0-40.0 sec 237 MBytes 397 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 35.0-40.0 sec 187 MBytes 314 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 35.0-40.0 sec 215 MBytes 360 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 35.0-40.0 sec 237 MBytes 398 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 35.0-40.0 sec 302 MBytes 507 Mbits/sec [SUM] 35.0-40.0 sec 3.67 GBytes 6.31 Gbits/sec [ ID] Interval Transfer Bandwidth [ 13] 0.0-42.8 sec 1.83 GBytes 368 Mbits/sec [ ID] Interval Transfer Bandwidth [ 5] 0.0-42.8 sec 1.99 GBytes 399 Mbits/sec [ ID] Interval Transfer Bandwidth [ 17] 0.0-42.8 sec 1.62 GBytes 325 Mbits/sec [ ID] Interval Transfer Bandwidth [ 6] 0.0-42.8 sec 2.19 GBytes 439 Mbits/sec [ ID] Interval Transfer Bandwidth [ 4] 0.0-42.8 sec 1.88 GBytes 378 Mbits/sec [ ID] Interval Transfer Bandwidth [ 12] 0.0-42.8 sec 1.71 GBytes 343 Mbits/sec [ ID] Interval Transfer Bandwidth [ 10] 0.0-42.8 sec 1.98 GBytes 397 Mbits/sec [ ID] Interval Transfer Bandwidth [ 15] 0.0-42.8 sec 2.10 GBytes 421 Mbits/sec [ ID] Interval Transfer Bandwidth [ 18] 0.0-42.8 sec 1.86 GBytes 373 Mbits/sec [ ID] Interval Transfer Bandwidth [ 7] 0.0-42.8 sec 2.15 GBytes 431 Mbits/sec [ ID] Interval Transfer Bandwidth [ 8] 0.0-42.8 sec 1.83 GBytes 367 Mbits/sec [ ID] Interval Transfer Bandwidth [ 16] 0.0-42.8 sec 1.77 GBytes 355 Mbits/sec [ ID] Interval Transfer Bandwidth [ 11] 0.0-42.8 sec 1.91 GBytes 383 Mbits/sec [ ID] Interval Transfer Bandwidth [ 9] 0.0-42.8 sec 2.39 GBytes 481 Mbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-42.8 sec 1.85 GBytes 371 Mbits/sec [ ID] Interval Transfer Bandwidth [ 14] 0.0-42.8 sec 2.06 GBytes 414 Mbits/sec [SUM] 0.0-42.8 sec 31.1 GBytes 6.24 Gbits/sec rwg@uct2-s1:~$
Single stream
- Result: 4.4 Gbps
rwg@uct2-s1:~$ iperf --client iut2-s1.iu.edu --port 443 --time 1000 -i 5 -w4M ------------------------------------------------------------ Client connecting to iut2-s1.iu.edu, TCP port 443 TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte) ------------------------------------------------------------ [ 3] local 128.135.158.221 port 50612 connected with 149.165.225.225 port 443 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 5.0 sec 2.46 GBytes 4.22 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 5.0-10.0 sec 2.50 GBytes 4.30 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 10.0-15.0 sec 2.58 GBytes 4.44 Gbits/sec [ ID] Interval Transfer Bandwidth [ 3] 0.0-15.4 sec 7.70 GBytes 4.30 Gbits/sec rwg@uct2-s1:~$
-- RobGardner - 18 Mar 2009
| I | Attachment | Action | Size | Date | Who | Comment |
|---|---|---|---|---|---|---|
 jpg |
BNL-UChicago-fig.jpg | manage | 63 K | 18 Mar 2009 - 09:36 | RobGardner | small version |
 png |
BNL-UChicago.png | manage | 631 K | 18 Mar 2009 - 09:32 | RobGardner | BNL-UChicago network connectivity |
 rtf |
Jumbo_frame_tests.rtf | manage | 31 K | 19 Mar 2009 - 06:21 | RobGardner | John Bigrow's before/after results on 3/18/09 |
Edit | Attach | Print version | History: r3 < r2 < r1 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r3 - 19 Mar 2009, RobGardner
CycleSeeder
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding Foswiki? Send feedback