Date: | 2012-11-07 13:49:43 |
---|
Abstract
Simple bench mark tests for the Mozillians API Bench result of MozilliansAPI.test_happy_path: Access 20 times the main url
Table of contents
The test MozilliansAPI.test_happy_path contains:
The bench contains:
The number of Successful Tests Per Second (STPS) over Concurrent Users (CUs).
CUs STPS TOTAL SUCCESS ERROR 1 7.600 76 76 0.00% 50 58.900 589 589 0.00% 100 57.300 573 573 0.00% 200 52.100 521 521 0.00%
The number of Successful Pages Per Second (SPPS) over Concurrent Users (CUs). Note that an XML RPC call count like a page.
CUs Apdex* Rating SPPS maxSPPS TOTAL SUCCESS ERROR MIN AVG MAX P10 MED P90 P95 1 1.000 Excellent 7.600 9.000 76 76 0.00% 0.114 0.131 0.670 0.116 0.121 0.134 0.139 50 0.968 Excellent 58.900 89.000 589 589 0.00% 0.289 0.830 2.460 0.504 0.759 1.160 1.683 100 0.634 POOR 57.300 78.000 573 573 0.00% 0.839 1.684 2.889 1.283 1.683 2.036 2.195 200 0.500 POOR 52.600 55.000 526 526 0.00% 1.576 3.445 4.468 2.778 3.521 4.001 4.130 * Apdex 1.5
The number of Requests Per Second (RPS) successful or not over Concurrent Users (CUs).
CUs Apdex* Rating* RPS maxRPS TOTAL SUCCESS ERROR MIN AVG MAX P10 MED P90 P95 1 1.000 Excellent 7.600 9.000 76 76 0.00% 0.114 0.131 0.670 0.116 0.121 0.134 0.139 50 0.968 Excellent 58.900 89.000 589 589 0.00% 0.289 0.830 2.460 0.504 0.759 1.160 1.683 100 0.634 POOR 57.300 78.000 573 573 0.00% 0.839 1.684 2.889 1.283 1.683 2.036 2.195 200 0.500 POOR 52.600 55.000 526 526 0.00% 1.576 3.445 4.468 2.778 3.521 4.001 4.130 * Apdex 1.5
The 5 slowest average response time during the best cycle with 50 CUs:
Req: 001, get, url /api/v1/users/?app_name=mozapp&app_key=test&format=json
CUs
Apdex*
Rating
TOTAL
SUCCESS
ERROR
MIN
AVG
MAX
P10
MED
P90
P95
1
1.000
Excellent
76
76
0.00%
0.114
0.131
0.670
0.116
0.121
0.134
0.139
50
0.968
Excellent
589
589
0.00%
0.289
0.830
2.460
0.504
0.759
1.160
1.683
100
0.634
POOR
573
573
0.00%
0.839
1.684
2.889
1.283
1.683
2.036
2.195
200
0.500
POOR
526
526
0.00%
1.576
3.445
4.468
2.778
3.521
4.001
4.130
* Apdex 1.5
CUs: Concurrent users or number of concurrent threads executing tests.
Request: a single GET/POST/redirect/xmlrpc request.
Page: a request with redirects and resource links (image, css, js) for an html page.
STPS: Successful tests per second.
SPPS: Successful pages per second.
RPS: Requests per second, successful or not.
maxSPPS: Maximum SPPS during the cycle.
maxRPS: Maximum RPS during the cycle.
MIN: Minimum response time for a page or request.
AVG: Average response time for a page or request.
MAX: Maximmum response time for a page or request.
P10: 10th percentile, response time where 10 percent of pages or requests are delivered.
MED: Median or 50th percentile, response time where half of pages or requests are delivered.
P90: 90th percentile, response time where 90 percent of pages or requests are delivered.
P95: 95th percentile, response time where 95 percent of pages or requests are delivered.
Apdex T: Application Performance Index, this is a numerical measure of user satisfaction, it is based on three zones of application responsiveness:
Satisfied: The user is fully productive. This represents the time value (T seconds) below which users are not impeded by application response time.
Tolerating: The user notices performance lagging within responses greater than T, but continues the process.
Frustrated: Performance with a response time greater than 4*T seconds is unacceptable, and users may abandon the process.
By default T is set to 1.5s this means that response time between 0 and 1.5s the user is fully productive, between 1.5 and 6s the responsivness is tolerating and above 6s the user is frustrated.
The Apdex score converts many measurements into one number on a uniform scale of 0-to-1 (0 = no users satisfied, 1 = all users satisfied).
Visit http://www.apdex.org/ for more information.
Rating: To ease interpretation the Apdex score is also represented as a rating:
Report generated with FunkLoad 1.16.1, more information available on the FunkLoad site.