The Texas Memory RAM-SAN is ideal for data base applications. The tests below show that the Texas Memory RAM-SAN excelled in transactional operations where high transaction rates were needed with low latency.
In order to compare the transactional capabilities of the Texas Memory RAM-SAN 520 with other devices, we ran a database analysis program that was written to extract HPSS accounting data from Transarc Encina Structure File Server (SFS) files. The data for these tests was taken from the NERSC "archive" HPSS system and reconstructed in the PROBE environment, using TRB and LA backup files that were created using the normal SFS backup mechanism.The HPSS metadata consists of approximately 50 SFS files, of which two are used by the accounting program. These files are the "cnsobjects" file, which is a sequential relative-record file, and the "bitfile" file, which is a random btree file, using the HPSS "bitfile ID" as the primary key.
The Encina SFS server uses raw AIX logical volumes, imposing its own page-based structure on top of the logical volumes, so the AIX file system buffer cache had no effect on these tests.
The accounting program reads sequentially through the cnsobjects file, extracting Nameserver-specific information, and then uses the (random) bitfile ID to read the Bitfile-server information for the file. For each record, information is extracted on file sizes, creation and access times, and tallied by owner, group and account ID, while also creating a histogram of file sizes. Non-file objects, such as directories and symbolic links, are ignored. During the execution of the program, periodic checkpoints are written to allow for restarting in case of interruption. At the end of the run, summary reports are written to a file.
The database used for these tests was composed of 5.2 million namespace objects.
In all cases, the tests were run on a dedicated IBM H70 with 4 processors and 1GB of memory, running AIX 4.3.3. The tests used a single LP8000 fibrechannel HBA, connected to the device via a Brocade 8-port switch. The following devices were tested:
Each of the following test cases was run first on the RAM-SAN and later (by using the AIX logical volume copy/rename capabilities) on the STK 9176. When multiple instances were run, the start times were staggered by 30 seconds, in order to try to avoid the situation where data read by processn would be cached by SFS for process n+1. 30 seconds was chosen as a reasonable guesstimate for the time that it would take for the 8MB SFS buffers to be overwritten by new data for one process before another process would ask for the same data.
While the test cases were running, the AIX monitor utility was run in a separate process, to collect periodic snapshots of the system performance data, averaged over the interval. PERL scripts were later written to extract variables of interest and to format the data for importing into MS-Excel for analysis.
Results of the initial test cases are summarized in the charts below. The RAM-SAN 520 was significantly faster in all cases.
As the number of processes increased, the ratio of duration times decreased. We believe this was due to saturation of the H70, as is borne out by the CPU and I/O charts later on in this document.
Since the H70 that was used for these tests is a 4-processor system, ideally we would expect to see about the same duration for 1, 2 and 4 processor tests if we were able to keep all 4 CPUs busy 100% of the time. However, this was not the case. As shown in the above chart, and below, there was approximately .2-.3 (20-30%)increase in duration for each extra process that was added to the mix when using the RAM-SAN. The ratio of durations as extra processes were added to the mix for the STK 9176 showed a much smaller increase:
For the STK 9176 disk array, increasing the mix by adding more processed showed a much smaller increase in the ratio of durations as more processes were added to the mix.
The CPU profiles for these tests are shown in the following charts. As mentioned above, these tests were run on a dedicated 4-processor system, and therefore one processor represents 25% of the total available CPU resource. Charts that both include and exclude idle time are included, in order to more clearly show the CPU distribution for user, system, and I/O wait time in the 4-processor environment.
As can be seen, for the 1-process test the RAM-SAN was effectively able to saturate one CPU, with I./O wait time effectively zero for over 50% of the duration, and no idle time (if the 3 unused CPUs are ignored). At sample 138, the program began to exhibit radically different behavior, which is examined in more detail in the next section. Once this phase of the program started, the single CPU was still saturated, but the CPU wait time rose to approximately 5%, while the user time dropped by the same amount. System time remained approximately constant for the entire test.
For the STK 9176, the CPU profile for the 1-process test showed bursts of CPU saturation during the first part of the test. Once the second phase of the program began, however, the I./O wait time rose to about 21% (84% of one CPU), with user time dropping to about 4% (16% of one cpu), and system time dropping from 5% to about 1.5% for the remainder of the test.
As the number of processes increased, the CPU usage approached 100% of the entire 4-processor system for the entire duration of the test for the RAM-SAN, but only slightly more than 40% of the entire system for the STK 9176.
The second phase of the program accounted for approximately 2/3 of the total time for the test for theSTK 9176, but only about 1/3 of the time for the RAM-SAN.
The following charts show I/O activity during the accounting tests for 1, 4 and 8 processes. The amount of I/O performed per operation was small (4K bytes). For a single instance of the test, about 1100 I/Os per second were seen with the RAM-SAN, and about 300 I/Os per second were observed with the 9176 Disk Array. As more processes were added to the mix, the RAM-SAN I/Os per second scaled up nearly linearly, while the disk array performance decreased slightly.
The CPU and I/O graphs present an interesting question, namely, what is going on to cause the performance changes after the program has been running for about an hour?
One possibility that we considered was that the differences might be related to the internal buffering used by the Encina Structure File Server (SFS).The initial buffer value was 8MB (8192 1K pages).We reran the tests, using 1MB and 16MB SFS buffers, and observed that the behavior did not change.
After further analysis, we believe that this behavior can be explained as an artifact of the way that the HPSS databases were created during the conversion from previous archival storage systems that were used at NERSC. As mentioned earlier, the accounting program sequentially reads the Nameserver's database; for each file in the database, a Bitfile ID is used to randomly access Bitfile Server metadata that is used for accounting. The Bitfile metadata is contained in a random BTREE file; the primary key is the Bitfile ID, which in turn contains a UUID as a primary part of the ID. The UUID is based in large part upon a high-resolution time value.
When the conversion to HPSS was performed, all of the Bitfile IDs were generated within a short period of time (several hours), and, as a result, the Bitfile keys would tend to cluster, resulting in records that were not uniformly spread across the volume. Files that have been added since the conversion would have UUIDs with a timestamps that cause the records to be spread more randomly throughout the database.
If this analysis is correct, then the initial phase of the program is reading metadata that was created during the initial HPSS conversion, and the second phase is dealing with files that have since been added and are randomly distributed. The second phase of the test is a more definitive comparision of the capabilities of the devices in dealing with truly random accesses.
The Texas Memory RAM-SAN was clearly the superior device to use for this type of application. Its zero latency access times, and high transaction rates provide the ideal combination for dealing with random accesses to large volumes of data.