Date: | 2012-11-07 13:27:53 |
---|
Abstract
Simple bench mark tests for the Mozillians API Bench result of MozilliansAPI.test_happy_path: Access 20 times the main url
Table of contents
The test MozilliansAPI.test_happy_path contains:
The bench contains:
The number of Successful Tests Per Second (STPS) over Concurrent Users (CUs).
CUs STPS TOTAL SUCCESS ERROR 1 0.250 1 1 0.00%
The number of Successful Pages Per Second (SPPS) over Concurrent Users (CUs). Note that an XML RPC call count like a page.
CUs Apdex* Rating SPPS maxSPPS TOTAL SUCCESS ERROR MIN AVG MAX P10 MED P90 P95 1 1.000 Excellent 5.000 9.000 20 20 0.00% 0.116 0.198 1.116 0.118 0.123 0.618 1.116 50 1.000 Excellent 63.250 70.000 253 253 0.00% 0.399 0.791 1.226 0.572 0.772 1.025 1.091 100 0.704 FAIR 69.750 64.000 279 279 0.00% 0.844 1.532 2.186 1.048 1.556 1.919 1.995 200 0.500 POOR 70.250 64.000 281 281 0.00% 1.525 3.468 4.852 2.249 3.639 4.391 4.586 * Apdex 1.5
The number of Requests Per Second (RPS) successful or not over Concurrent Users (CUs).
CUs Apdex* Rating* RPS maxRPS TOTAL SUCCESS ERROR MIN AVG MAX P10 MED P90 P95 1 1.000 Excellent 5.000 9.000 20 20 0.00% 0.116 0.198 1.116 0.118 0.123 0.618 1.116 50 1.000 Excellent 63.250 70.000 253 253 0.00% 0.399 0.791 1.226 0.572 0.772 1.025 1.091 100 0.704 FAIR 69.750 64.000 279 279 0.00% 0.844 1.532 2.186 1.048 1.556 1.919 1.995 200 0.500 POOR 70.250 64.000 281 281 0.00% 1.525 3.468 4.852 2.249 3.639 4.391 4.586 * Apdex 1.5
The 5 slowest average response time during the best cycle with 50 CUs:
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
1.116
1.116
1.116
1.116
1.116
1.116
1.116
50
1.000
Excellent
23
23
0.00%
0.639
0.798
1.223
0.688
0.770
0.945
1.086
100
0.614
POOR
35
35
0.00%
1.302
1.695
2.072
1.391
1.642
2.042
2.071
200
0.500
POOR
20
20
0.00%
3.161
3.800
4.708
3.282
3.803
4.523
4.708
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.618
0.618
0.618
0.618
0.618
0.618
0.618
50
1.000
Excellent
38
38
0.00%
0.489
0.816
1.149
0.663
0.811
1.012
1.126
100
0.585
POOR
71
71
0.00%
1.257
1.672
2.120
1.465
1.666
1.917
1.993
200
0.500
POOR
31
31
0.00%
1.962
3.736
4.786
2.814
3.807
4.587
4.752
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.123
0.123
0.123
0.123
0.123
0.123
0.123
50
1.000
Excellent
48
48
0.00%
0.399
0.712
1.152
0.491
0.701
0.934
0.940
100
0.738
FAIR
82
82
0.00%
0.844
1.496
2.043
1.041
1.521
1.873
1.943
200
0.500
POOR
29
29
0.00%
1.823
3.347
4.709
1.886
3.648
4.315
4.696
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.125
0.125
0.125
0.125
0.125
0.125
0.125
50
1.000
Excellent
50
50
0.00%
0.433
0.734
1.025
0.560
0.686
0.977
1.010
100
0.824
FAIR
51
51
0.00%
0.875
1.398
2.186
1.040
1.360
1.898
1.963
200
0.500
POOR
31
31
0.00%
1.940
3.264
4.160
2.218
3.411
4.031
4.123
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.159
0.159
0.159
0.159
0.159
0.159
0.159
50
1.000
Excellent
49
49
0.00%
0.444
0.842
1.173
0.582
0.889
1.091
1.100
100
0.712
FAIR
26
26
0.00%
0.939
1.473
2.027
1.017
1.564
1.956
2.016
200
0.500
POOR
43
43
0.00%
1.932
3.757
4.776
2.655
3.908
4.524
4.586
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.123
0.123
0.123
0.123
0.123
0.123
0.123
50
1.000
Excellent
29
29
0.00%
0.560
0.843
1.226
0.575
0.857
1.098
1.106
100
0.875
Good
12
12
0.00%
0.887
1.270
1.856
1.005
1.240
1.727
1.856
200
0.500
POOR
60
60
0.00%
1.525
3.430
4.840
2.184
3.609
4.312
4.531
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.118
0.118
0.118
0.118
0.118
0.118
0.118
50
1.000
Excellent
12
12
0.00%
0.524
0.919
1.068
0.709
0.973
1.035
1.068
100
1.000
Excellent
2
2
0.00%
0.852
0.929
1.005
0.852
1.005
1.005
1.005
200
0.500
POOR
33
33
0.00%
1.858
3.404
4.852
2.038
3.426
4.523
4.807
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.119
0.119
0.119
0.119
0.119
0.119
0.119
50
1.000
Excellent
4
4
0.00%
0.535
0.786
1.120
0.535
0.833
1.120
1.120
200
0.500
POOR
19
19
0.00%
1.860
3.093
4.516
1.998
3.048
3.927
4.516
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.121
0.121
0.121
0.121
0.121
0.121
0.121
200
0.500
POOR
12
12
0.00%
2.093
3.156
4.683
2.299
3.453
3.785
4.683
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.127
0.127
0.127
0.127
0.127
0.127
0.127
200
0.500
POOR
3
3
0.00%
1.942
2.735
3.566
1.942
2.697
3.566
3.566
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.120
0.120
0.120
0.120
0.120
0.120
0.120
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.119
0.119
0.119
0.119
0.119
0.119
0.119
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.118
0.118
0.118
0.118
0.118
0.118
0.118
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.122
0.122
0.122
0.122
0.122
0.122
0.122
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.116
0.116
0.116
0.116
0.116
0.116
0.116
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.123
0.123
0.123
0.123
0.123
0.123
0.123
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.125
0.125
0.125
0.125
0.125
0.125
0.125
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.117
0.117
0.117
0.117
0.117
0.117
0.117
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.124
0.124
0.124
0.124
0.124
0.124
0.124
* Apdex 1.5
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
1
1
0.00%
0.126
0.126
0.126
0.126
0.126
0.126
0.126
* Apdex 1.5
CUs: Concurrent users or number of concurrent threads executing tests.
Request: a single GET/POST/redirect/xmlrpc request.
Page: a request with redirects and resource links (image, css, js) for an html page.
STPS: Successful tests per second.
SPPS: Successful pages per second.
RPS: Requests per second, successful or not.
maxSPPS: Maximum SPPS during the cycle.
maxRPS: Maximum RPS during the cycle.
MIN: Minimum response time for a page or request.
AVG: Average response time for a page or request.
MAX: Maximmum response time for a page or request.
P10: 10th percentile, response time where 10 percent of pages or requests are delivered.
MED: Median or 50th percentile, response time where half of pages or requests are delivered.
P90: 90th percentile, response time where 90 percent of pages or requests are delivered.
P95: 95th percentile, response time where 95 percent of pages or requests are delivered.
Apdex T: Application Performance Index, this is a numerical measure of user satisfaction, it is based on three zones of application responsiveness:
Satisfied: The user is fully productive. This represents the time value (T seconds) below which users are not impeded by application response time.
Tolerating: The user notices performance lagging within responses greater than T, but continues the process.
Frustrated: Performance with a response time greater than 4*T seconds is unacceptable, and users may abandon the process.
By default T is set to 1.5s this means that response time between 0 and 1.5s the user is fully productive, between 1.5 and 6s the responsivness is tolerating and above 6s the user is frustrated.
The Apdex score converts many measurements into one number on a uniform scale of 0-to-1 (0 = no users satisfied, 1 = all users satisfied).
Visit http://www.apdex.org/ for more information.
Rating: To ease interpretation the Apdex score is also represented as a rating:
Report generated with FunkLoad 1.16.1, more information available on the FunkLoad site.