Amber on Biowulf: Benchmarks

Amber 10 Benchmarks: Factor IX

May 2008. The Factor IX constant pressure benchmark from the AMBER benchmark suite. A TIP3 water solvated system with 90,905 total atoms and PME used with a direct space cutoff of 8 Angstroms.

Bottom line:

If at all possible, run on Infinipath (ipath). This job scales up to about 16 processors on the Infinipath network, but only up to 4 processors on gige.
The next best is Myrinet. The job scales up to 8 processors on the Myrinet nodes. However, these nodes have slower processors so the walltime is 1.5 times slower than the Infinipath nodes.
Do not run on more than 4 processors on gige, unless you have run your own benchmarks and can demonstrate that the job scales to more than 4p.

No. Processors	Run Time in seconds (Efficiency %)
No. Processors	o2800 gige 2.8 GHz Opteron 1 Gb/s ethernet Pathscale compilers, 64-bit	o2800 Ipath 2.8 GHz Opteron 8 Gb/s Infinipath Pathscale compilers,64-bit	o2800dc gige 2.8 GHz dual-core Opteron 1 Gb/s ethernet Pathscale compilers, 64-bit
1	723 (100)	728 (100)	725 (100)
2	395 (92)	380 (96)	432 (84)
4	336 (54)	199 (91)	229 (79)
8	221 (41)	106 (85)	236 (38)
16	159 (28)	62 (73)	176 (26)
32	142 (16)	38 (59)	153 (15)

Amber 9 Benchmarks: Factor IX

May 2008. The Factor IX constant pressure benchmark from the AMBER benchmark suite. A TIP3 water solvated system with 90,905 total atoms and PME used with a direct space cutoff of 8 Angstroms.

Bottom line:

If at all possible, run on Infinipath (ipath). This job scales up to about 16 processors on the Infinipath network, but only up to 4 processors on gige.
The next best is Myrinet. The job scales up to 8 processors on the Myrinet nodes. However, these nodes have slower processors so the walltime is 1.5 times slower than the Infinipath nodes.
Do not run on more than 4 processors on gige, unless you have run your own benchmarks and can demonstrate that the job scales to more than 4p.

No.
Processors Run Time in seconds (Efficiency %)
o2800 gige
2.8 GHz Opteron
1 Gb/s ethernet
Portland Group compilers, 32-bit o2600 gige
2.6 GHz dual-core Opteron
1 Gb/s ethernet
Portland Group compilers, 32-bit o2200 myr2k
2.2 GHz Opteron
2 Gb/s Myrinet
Portland Group compilers, 32-bit o2800 Ipath
2.8 GHz Opteron
8 Gb/s Infinipath
Pathscale compilers,64-bit o2800 gige
2.8 GHz Opteron
1 Gb/s ethernet
Intel compilers, 64-bit o2800 gige
2.8 GHz Opteron
1 Gb/s ethernet
Pathscale compilers, 64-bit
1 794 (100) 844 (100) 1018 (100) 669 (100) 648 (100) 659 (100)
2 430 (92) 497 (85) 536 (95) 351 (95) 397 (81) 378 (87)
4 301 (66) 306 (69) 292 (87) 179 (93) 304 (53) 298 (55)
8 217 (46) 226 (47) 162 (79) 95 (88) 211 (38) 235 (35)
16

102 (62) 55 (77) 151 (27) 169 (24)
32

74 (43) 37 (57) 134 (15) 141 (15)

Benchmarks with older nodes (July 2006)

No. Processors	Run Time in seconds (Efficiency %)
No. Processors	p2800, gige 2.8 GHz Xeon, 1Gb/s ethernet	p2800, Myr (2.8 GHz Xeon, 2Gb/s Myrinet)	o2200, gige (2.2 GHz Opteron, 1Gb/s ethernet)	o2200, myr (2.2 GHz Opteron, 2Gb/s Myrinet)	o2800, gige (2.8 GHz Opteron, 1Gb/s ethernet)
1	1403 (100)	1414 (100)	1037 (100)	1018 (100)	794 (100)
2	845 (83)	838 (84)	562 (92)	536 (95)	430 (92)
4	577 (61)	452 (78)	369 (70)	292 (87)	301 (66)
8	384 (46)	252 (70)	250 (52)	162 (78)	217 (46)
16	263 (33)	155 (57)	172 (37)	102 (62)	146 (34)
32	220 (20)		151 (21)	74 (43)	132 (19)

Bottom line:

Not surprisingly, the job runs faster on faster processors :-)
Scales much better on Myrinet than on GigE.
To get 60% efficiency, the job should be run on no more than 8 o2200 gige processors (4 nodes), or 16 Myrinet processors (8 nodes).

The chart below shows the same benchmarks, reported in psec/day. (calculated as per Amber website: dt=0.0015, so ps-per-day = 129.6/(time-per-step).

No.
Processors ps-per-day (Speedup)
p2800, gige
2.8 GHz Xeon, 1Gb/s ethernet p2800, Myr
(2.8 GHz Xeon, 2Gb/s Myrinet) o2200, gige
(2.2 GHz Opteron, 1Gb/s ethernet) o2200, myr
(2.2 GHz Opteron, 2Gb/s Myrinet) o2800, gige
(2.8 GHz Opteron, 1Gb/s ethernet)
1 46.17 45.81 62.46 63.63 81.53
2 76.61 (1.66) 77.32 (1.69) 115.12 (1.85) 120.89 (1.90) 150.55 (1.85)
4 112.27 (2.43) 143.33 (3.13) 175.19 (2.82) 221.33 (3.48) 214.99 (2.64)
8 168.65 (3.65) 257.12 (5.61) 259.01 (4.17) 399.48 (6.28) 298.14 (3.66)
16 246.07 (5.33) 417.26 (9.11) 374.59 (6.03) 633.37 (9.95) 441.33 (5.41)
32 293.25 (6.35) 427.53 (6.88) 872.14 (13.71) 488.39 (5.99)

No. Processors	ps-per-day (Speedup)
No. Processors	p2800, gige 2.8 GHz Xeon, 1Gb/s ethernet	p2800, Myr (2.8 GHz Xeon, 2Gb/s Myrinet)	o2200, gige (2.2 GHz Opteron, 1Gb/s ethernet)	o2200, myr (2.2 GHz Opteron, 2Gb/s Myrinet)	o2800, gige (2.8 GHz Opteron, 1Gb/s ethernet)
1	46.17	45.81	62.46	63.63	81.53
2	76.61 (1.66)	77.32 (1.69)	115.12 (1.85)	120.89 (1.90)	150.55 (1.85)
4	112.27 (2.43)	143.33 (3.13)	175.19 (2.82)	221.33 (3.48)	214.99 (2.64)
8	168.65 (3.65)	257.12 (5.61)	259.01 (4.17)	399.48 (6.28)	298.14 (3.66)
16	246.07 (5.33)	417.26 (9.11)	374.59 (6.03)	633.37 (9.95)	441.33 (5.41)
32	293.25 (6.35)		427.53 (6.88)	872.14 (13.71)	488.39 (5.99)