Physical Sciences and Mathematics Commons^™

Dynamic Load Balancing For I/O-Intensive Applications On Clusters, Xiao Qin, Hong Jiang, Adam Manzanares, Xiaojun Ruan, Shu Yin

School of Computing: Faculty Publications

Load balancing for clusters has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, previous CPU- or memory-centric load balancing schemes suffer significant performance drop under I/O-intensive workloads due to the imbalance of I/O load. To solve this problem, we propose two simple yet effective I/O-aware load-balancing schemes for two types of clusters: (1) homogeneous clusters where nodes are identical and (2) heterogeneous clusters, which are comprised of a variety of nodes with different performance characteristics in computing power, memory capacity, and disk speed. In addition to assigning I/O-intensive sequential and parallel jobs …