Grid Resource Management and Scheduling for Data Streaming Applications

Abstract

Data streaming applications bring new challenges to resource management and scheduling for grid computing. Since real-time data streaming is required as data processing is going on, integrated grid resource management becomes essential among processing, storage and networking resources. Traditional scheduling approaches may not be sufficient for such applications, since usually only one aspect of grid resource scheduling is focused. In this work, an integrated resource scheduling approach is proposed and coordinated resource allocation of CPU cycles, storage capability and network bandwidth is implemented. Resource allocation is performed periodically with updated information on resources and applications and heuristic search for optimal solutions is used to assign various resources for running applications simultaneously. Performance metrics considered in this work include data throughput and utilization of processors, storage, and bandwidth, which are actually tightly coupled with each other when applied for grid data streaming applications. Experimental results show dramatic improvement of performance and scalability using our implementation.