Thermal Aware and DVFS Enabled Big Data Task Scheduling for DataCenters Hadoop Bigdata

Big data has received considerable attentions in recent years because of massive data volumes in multifarious fields. Considering various “V” features, big data tasks are usually highly complex and computational intensive. These tasks are generally performed in parallel in data centers resulting in massive energy consumption and Green House Gases emissions. Therefore, efficient resource allocation considering the synergy of the performance and energy efficiency is one of the crucial challenges today. In this paper, we aim to achieve maximum energy efficiency by combining thermal-aware and dynamic voltage and frequency scaling (DVFS) techniques. This paper proposes: (a) a thermal-aware and power-aware hybrid energy consumption model synchronously considering the computing, cooling, and migration energy consumption; (b) a tensor-based task allocation and frequency assignment model for representing the relationship among different tasks, nodes, time slots, and frequencies; and (c) a big data Task Scheduling algorithm based on Thermal-aware and DVFS-enabled techniques (TSTD) to minimize the total energy consumption of data centers. The experimental results demonstrate that the proposed TSTD algorithm significantly outperforms the state-of-the-art energy efficient algorithms from total, computing, and cooling energy consumption perspectives, as well as cooling energy consumption proportion and total energy consumption savings.