HPSS Benchmarks

A series of benchmarks were run between the PROBE HPSS system and the PDSF cluster. The purpose of these benchmarks was to study the performance of the TMS RAM-SAN disk. To mitigate network bottlenecks, the network connection between the PDSF cluster and PROBE used Jumbo GigE.

The lessons we learned from these benchmarks were:

• HSI and HPSS performed very well. Using multiple streams, we were able to achieve over 300 MB/sec between the PDSF cluster and the PROBE HPSS system. Higher performance could have been achieved with faster computer hardware on the probe system for two of the HPSS movers (the IBM H50 & F50). We believe that the hosts were limiting performance rather than the network or the TMS RAM-SAN. In other words, we did not reach the peak performance of the TMS.
• HPSS and HSI work well with multiple streams.
• The performance characteristics of the client file system (PDSF cluster) is very important and can affect HPSS transfer rates.

Details:

• For PROBE/HPSS we had four data movers. Two of the movers were on an IB H70 and the others were on an IBM H50 and an IBM F50. Multiple Classes of Service (COS) were configured so that specific network/host/fibre channel interfaces could be selected for each transfer. Each mover was connected to a different LUN on the TMS box over fiber (through a Brocade switch). We had to use the Brocade switch to connect to the TMS box because the TMS was not supported by our Fibre cards under AIX at the time of this testing. We also installed Brocade's Switch Management Software (HS-200WTOOL) for configuring zones between a specific IBM/AIX host and a TMS LUN.
• Unless specified, reads from HPSS were to /dev/null using HSI.
• Files that were written from PDSF to HPSS were done using the local PDSF file system. In some cases we used sparse files for faster access to the local file system.

Results & Analysis:

The first series of benchmarks characterized and benchmarked single read and write performance between PDSF and the PROBE HPSS system. Not all COS could be reached from all machines because of the configuration of our GigE switch.

As can be seen by the first chart there is a performance difference between the PDSF nodes and HPSS. Movers on the IBM H50 and F50 consistently performed less than the either of the two movers on an IBM H70.

As shown by the next graph, write performance was significantly lower (~50%) than reads. The next graph shows that a lot of this was due to the performance of the local disk systems. We benchmarks reads from it to /dev/null to get an idea of the impact of the local disk and this is shown in the next graph.

Multiple Streams 

We next tried double streams over each of the data paths from PDSF through HPSS to the TMS Ram-SAN. Again there is a performance disparity among HPSS movers We cannot explain why the two data paths through an IBM H70 have different speeds. For the IBM H50 and F50, the speeds are slightly better with two streams compared to one.

We then did our best to reach the maximum possible speed using multiple streams through all four mover paths simultaneously. With nine data streams across four mover paths, we reached just over 300 MB/sec reading from HPSS.

A similar test was performed for writes, reaching a peak speed of just under 200 MB/sec.

Comparison between Fiber Channel Devices 

The last benchmark compared performance between PDSF and two different Fibre Devices (the TMS box and our STK 9176 disk array). As you can see, there was little difference between them.