Migration Deployment Scenarios - High Throughput or Low Latency
HigH tHRougHput oR loW latenCY Servers deployed for high throughput are configured very differently than low-latency application servers. It is important to separate applications that require low latency from those that require high throughput. The configuration of timeouts, max threads, and other basic options are at the opposite end of the spectrum from each other and will not mix well. Red Hat Consulting Strategic Migration Planning Guide10 www.jboss.com When targeting high throughput, it is the overall performance of the system that is prioritized — client requests are not given individual attention. As an example, for a system with only four processor cores, one might determine that 200 concurrent requests mark the optimal point of parallel processing with an average processing time of one second. Anything below this number would be under-utilizing the system, while a higher number of concurrent requests results in wasteful context switching and hurts the overall throughput. Configuration based on throughput would therefore limit concurrent requests to 200. In the event of 300 concurrent requests, we would queue up the remaining concurrent requests and process them as a batch consecutive to the first group. In this oversimplified analysis, requests that are part of the second batch will take almost twice as long to process as the first batch. This runs counter to prioritizing low latency and the requests from the second batch may well be violating our service-level agreement (SLA). Should low latency be our priority, we could configure the system to accept 300 concurrent requests. In our hypothetical example, it is conceivable that average processing time would then grow to 1.7 seconds. Requests from the second batch would complete in 1.7 seconds instead of 2 seconds and would therefore meet the SLA. The throughput, however, has gone down from 200 tps to 176.5 tps and even with a constant flow of 300 concurrent requests, the system will process fewer requests in the long-term. LOW LATENCY HIGH THROUGHPUT JBoss provides a robust platform for building out large-scale server farms that can support tens of thousands of concurrent requests.