Client side load balancer using cloud in Java

Web applications' traffic demand fluctuates widely and unpredictably. The common practice of provisioning a fixed capacity would either result in unsatisfied customers (underprovision) or waste valuable capital investment (overprovision). By leveraging an infrastructure cloud's on-demand, pay-per-use capabilities, we finally can match the capacity with the demand in real time. This paper investigates how we can build a large-scale web server farm in the cloud. Our performance study shows that using existing cloud components and optimization techniques, we cannot achieve high scalability. Instead, we propose a client-side load balancing architecture, which can scale and handle failure on a milli-second time scale. We experimentally show that our architecture achieves high throughput in a cloud environment while meeting QoS requirements.