Energy-Aware and Proactive Host Load Detection in Virtual Machine Consolidation

VM consolidation technique on the physical server improves the energy consumption and resource efficiency of the physical server, and thus improves the quality of service (QoS). In this article, a server threshold prediction method is proposed that focuses on the server overload and server underload detection to improve server utilization and to reduce the number of VM migrations, which consequently improves the VM's QoS. Since the VM integration problem is very complex, the exponential smoothing technique is utilized for predicting server utilization. The results of the experiments show that the proposed method goes beyond existing methods in terms of power efficiency and the number of VM migrations.


Introduction
Cloud computing has been raised as one of the most crucial exciting technological developments, which is growing rapidly to provide almost unlimited process, storage, and network resources for its users [13]. The growing interest in cloud resources has led to the creation of enormous cloud data centers, which need a lot of energy, and cause a large amount of operational costs. Worldwide data center power utilization information shows a nonlinear increase over the past ten years and a similar trend is predicted for the future [28]. Cloud providers utilize a lot of data centers around the world [27]. Energy is one of the significant factors of the entire cost of managing a data center and its facilities [18]. Several methods have been proposed in the literature to enhance resource utilization in cloud environments to lead to energy efficiency.
As some examples, Load balancing and resource utilization were improved in [6] using a combination of reinforcement learning (RL) and coral reefs optimization. Multiple RL-based agents were utilized in [7] for online scheduling of dependent tasks of the cloud's workflows to improve resource utilization. Cooperative RL agents managed the cloud resources in [8] for multiple online scientific workflows. SARSA RL agents and genetic algorithm were used in [9] for cloud resource utilization.
VM consolidation is a common and useful method to enhance resource efficiency and power effectiveness in the data centers, which has been extensively investigated in recent years. For example, server consolidation frameworks for cloud data centers have been reviewed in [2]. Server consolidation techniques in virtualized data centers have been also reviewed in [33]. A joint VM and container consolidation approach was utilized in [17] for energy-aware resource management in cloud data centers. EMC2 was proposed in [20] as an energy-efficient and multi-resource fairness VM consolidation in cloud data centers. Another energy-efficient VM consolidation algorithm in cloud data centers was also proposed in [38]. VM consolidation leverages by using virtualization as the main technique for achieving better performance and improving energy consumption. Virtualization techniques enable the data centers to deploy and run multiple instances of the VMs on a single physical server, and during the VM lifetime, it is possible to move VMs among physical servers without downtime. To reduce the number of active servers, VMs should be regularly relocated among the consolidation process using live migration according to the resource demands, and to reduce the energy consumption, the idle servers should be switched to sleep or power-off mode [11]. Due to the dynamic resource utilization model of the most current applications because of their highly variable workloads, consolidating VMs in the cloud data centers is complex. Unrestricted consolidation of the VMs may cause performance degradation by increasing the requests of applications for resources. If the requested resource or program of an application is not provisioned, its response time is increased and causes degrading quality of service (QoS) of that application. The QoS is defined for the services of the cloud service provider for the end-users.
The main contributions of this paper are summarized as follows: _ Proposing a server threshold prediction method that focuses on detecting overloaded and underloaded servers. _ Improving server utilization, reducing the number of VM migrations, and enhancing the QoS. _ Utilizing an exponential smoothing technique for predicting server utilization to overcome the complexity of the VM integration problem.
The remainder of the paper is composed as follows: Section 2 discusses some recent related works. Section 3 explains the system model and defines the problem. The proposed method of this paper to predict server utilization is also described in Section 3. Section 4 presents the results of simulation and finally, Section 5 concludes the work.

Related Works
There are 4 sub-problems in the VM consolidation procedure which need to be solved [25]. The first sub-problem is to decide when a host is overloaded. The next sub-problem is to select and identify the proper VMs for migrating from an overloaded host. VM placement is the third sub-problem, in which, a proper host for the selected VM would be picked. The last sub-problem is to identify the underloaded servers to consolidate the VMs of these servers on a smaller number of active servers [36]. Depending on the specific problems of the VM consolidation and the perspective of the research, a variety of strategies have been suggested in this area of the literature. A systematic review of challenges and open isuues for server consolidation in virtualized data centers has been proposed in [1].
The data center's workload changes continuously, which affects resource efficiency and energy utilization of the physical servers [5]. Most existing studies determine the server's upper threshold for overloaded hosts according to the host resource utilization. By applying VM consolidation on the overloaded and underloaded hosts, the energy consumption is reduced, and the QoS is improved [19,35,39].
VM placement was carried out under optimal conditions in cloud data centers in [16]. A VM placement method, called MBFD, has been proposed in [12] that reduced the number of running hosts based on a bin-packing heuristic algorithm, called BFD (bestfit decreasing) [29]. Another VM placement method was also proposed in [22] for OpenStack Neat consolidation. A fuzzy threshold-based adaptive algorithm has been proposed in [31] for identifying the overloaded and underloaded hosts. The fuzzy inference engine utilized the information of the host (resource utilization) to estimate the numerical threshold of CPU utilization. Witanto et al. [34] focused on accomplishing a method to tradeoff between the QoS and the power consumption based on the system preferences. The concept of dynamic selection was also proposed in [34] by searching into a set of consolidation methods and comparing various impacts on service level agreement (SLA) and energy to accommodate the data center with the desired preferences.
The issue of cooperation among memory and processor was focused in [4], and a multi-input and multi-output controller was introduced for VM consolidation from resource capacity perspectives, while respecting SLA constraints. Dynamic modification of the capacity of the resources of the collection of VMs was considered for serving a multi-tier application. While the proposed method had a higher level of resource consolidation, its computational overhead was also high. The K-means clustering-based adaptive three-threshold framework of [37] divided the hosts into four categories, according to the amount of their loads. The number of VM migrations and the effects of these migrations on the QoS were decreased in the proposed method of [22] by consolidating VMs based on evaluating the possibility of change in the conditions of the resources into the overload state.
A comprehensive research on VM consolidation has been conducted in [10]. Two VM selection policies, namely MMT (minimum migration time) policy and MC (maximum correlation) policy, were proposed in [10]. The MMT policy selected the VM for migration that has the shortest required migration time, and the MC policy selected the VM with the highest correlation of CPU usage with the other VMs for relocation to decrease the probability of overloading the host. Ant colony and extreme learning were utilized in [23] to VM consolidation in cloud data centers. Pearson correlation was used for dynamic VM consolidation in [24]. Another energy-efficient method for dynamic VM consolidation was also proposed in [21]. Some applications of VM consolidation in cloud computing were provided in [40].
The proposed method of this paper exploited a server threshold prediction method to detect overloaded and underloaded servers. This method attempts to overcome the complexity of the VM integration problem by utilizing an exponential smoothing technique for predicting server utilization. Server utilization is improved, and the number of VM migrations is reduced using this method, and thus the QoS will be enhanced.

The Proposed Method
Predicting the appropriate times of relocating the VMs is a key factor for improving the utilization of the VM consolidation, which leads to consolidate the VMs on fewer hosts. Various techniques are used in statistical and computational methods for this prediction. Due to the nature of the VM consolidation problem, we leverage the host extended exponential smoothing method [14][15]. Exponential smoothing is a general guideline procedure that uses the exponential window function for smoothing time series data. Although the effects of distant periods are not significant in the predictions, it is nevertheless appropriate that all previous periods are involved in the predictions.
Simple exponential smoothing has been extended to empower the forecasting of data with a trend. This technique includes level and trend smoothing equations (l t and b t ), used by a forecast equation [3,[14][15] (3) where ℓt-1 is the estimate of the level in time t-1, bt-1 is the estimate of the growth rate in time t-1, l t and b t are respectively the estimates of the level and the trend in time t, α (0≤α≤1) and β (0≤β≤1) are respectively the smoothing parameters for the level and the trend, and yt is the observation of series in time t. The initial values of the parameters can be based on past statistics or initial knowledge and experience [3]. The parameters α and β can be adjusted by analyzing the results for different values to decrease the difference between real and predicted values.
Using aggressive VM migration leads to extra overhead and energy consumption. The proposed method of this paper aims to prevent unnecessary VM migrations by predicting the status of host resources to improve the QoS and energy consumption. For example, assume 5 VMs with the same specifications are running on a host. The actual efficiency of different VMs at different times is shown in Table 1, and the amount of CPU uti-lization predicted by the proposed method is shown in Table 2. As shown in Table 2, the predictions are very close to the real numbers, and they will be very helpful in identifying the hosts that are overloaded and underloaded. It is clear that the predictions are not the same as what is happening in reality, and a percentage of errors in predictions is tolerable. The prediction with an acceptable difference is above the real threshold, and the hosts' threshold will not rise above that. To show that this approach can be useful for consolidating the VMs, consider this example in a data center.
Suppose the host threshold is 27%. The host threshold can have different values, and this percentage is just an example to illustrate how the proposed solution works. According to this threshold, at the time 3, the host is considered as an overloaded host, and consequently, some of its VMs must be selected and migrated to the other appropriate hosts. However, if the system can be aware or guess what is going to happen to this host in a short time, it may be able to make a better decision. According to the predictions in Table 2, the resources will be reduced at the time 5. Therefore, the proposed method avoids choosing this host as an overloaded host, and allows it to continue without indicating as an overloaded host. Because the proposed method predicts that the host will return to the normal state in a short time. In other words, by predicting the future state of the host from the current state, the proposed method prevents unwanted migrations in the current state, and thus prevents the reduction in the QoS that occurs with the VM migrations.
This method can be also utilized to determine the underloaded hosts. Suppose that all data center hosts are in a normal state and their VMs are fully running without resource shortage. There is only one underloaded host, the utilization of which is lower than the specified threshold. The proposed method tries to relocate VMs from underloaded hosts to the other hosts, and makes the host go sleep state or shutdown to reduce energy consumption and improve efficiency. Whenever the data center needs resources, the proposed method powers on or wakes up the host. Switching on and off or sleeping and waking up the hosts is costly and time-consuming. When the method makes a host go sleep state or shutdown, and a few minutes later, due to the growing demand for VMs resources, the host should be restarted, some overhead costs, such as the reboot time, are imposed to the host. This causes the system to experience a shortage of resources and faces a level of service violation. In this situation, predicting the data center utilization (DCU) can help the method to identify the underloaded hosts, more accurately.
For example, consider a data center that has 5 hosts, which the utilizations of its hosts at different time intervals are shown in Table 3. The utilizations of the hosts are different and change during the time. At the time 3. as shown in Table 3, the utilization of the host 5 is lower than the threshold (30%), and thus the host should be considered as an underloaded host. However, prediction of the future of processor utilization is used for final decision. Since the overall data efficiency of the data center is still above the threshold in predictions, the method ignores them, and lets them continue because there is a possibility that the resources will be needed again. Hence, the host is not determined as an underloaded host. Consider time 7, when the utilization of the host 5 is lower than the threshold, and with looking at the predictions, a relative decrease in the total DCU can be considered. Therefore, this server is specified as an underloaded server if the other hosts have sufficient resources to accept the VMs of this host.

Simulation Result
The proposed method is executed on the CloudSim simulation toolkit [26]. The CloudSim toolkit is used to simulate the cloud data centers and their infrastructures. The experiments are carried out under following conditions: 1 The real data from the CoMon project (a monitoring system for Planetlab) are used for the experiments. The data includes the processor utilization data from more than 1,000 VMs located on hosts in more than 500 locations around the world. The data were collected on 10 random days in March and April 2011 [30]. 2 The simulation environment includes 300 servers of common types available in data centers, includ- Table 3  Data centers' utilization and their prediction   Time 1  Time 2  Time 3  Time 4  Time 5  Time 6  Time 7  Time 8  Time 9   Host1  84  74  84  84  68  55  43  80  75   Host2  85  35  79  82  75  60  15  75  35   Host3  75  75  45  75  60  75  85  60  60   Host4  60  78  60  77  55  60  75  65  ing Hp ProLiant DL360 G7, Hp ProLiant DL360 Gen9, and Hp ProLiant Ml110 G5. A random model is used to model the amount of memory and bandwidth, and the processor used the CoMon project workload model. Table 4 shows the specifications of the intended servers. Amazon EC2 VMs are also utilized (from https://aws.amazon.com/ec2/instance-types/), which include 5 different types of VMs. The details of these VMs are summarized in the Table 5.
The proposed method is implemented and its performance is evaluated compared to the other VM consolidation techniques, inter quartile range (IQR) and local regression (LR). These algorithms are evaluated in combination with MMT and MC. IQR applies statistical analysis for dynamic adaptation in the utilization threshold. LR estimates the future resources utilization to adjust the threshold accordingly. The first algorithm, known as LrMmt, which uses the LR algorithm with parameter 1.2 to detect the server overflow, and employs the MMT algorithm to select the VMs. The second algorithm, known as IqrMc, which uses the IQR algorithm with secure parameter 1.5 to detect the finite element, and exploits the MC algorithm to select the VMs. This comparison is based on different criteria including energy consumption, the number of VM migrations, SLA violations, and the number of host shutdowns. Table 6 shows the results obtained in one of the days.     Energy consumption (kWh)

Figure 3
SLA violation Fig. 1 shows the comparison of the LrMmt, the IqrMc, and the proposed method based on the number of VM migrations. The number of the migrations in the proposed method is 3154 migrations, which is 30%, and 31% lower than the number of the migrations in LrMmt and IqrMc, respectively. In the proposed method, the number of migrations has significantly diminished by eliminating relocations from the hosts that are not underloaded. When the method has to decide whether a host is overloaded or not, it predicts the state of the host's resources in the next point based on the amount of resource consumption in the past.
Simulation output for day 03/03/2011 Number of VM migrations

Figure 3
SLA violation

Conclusions
One of the main concerns of cloud data centers is to improve energy efficiency and to increase the profitability of these data centers, while keeping the quality of service at an acceptable level. In the proposed method of this paper, the problem of virtual machine (VM) consolidation was focused by considering host overload and host underload detection. The proposed method mapped the future state of the data center by predicting the hosts' workloads, and then based on these predictions, decided to specify a host as an overloaded or underloaded host. In the proposed method, an exponential smoothing technique was also leveraged to predict host utilization to make a better decision. Therefore, improper migrations were avoided, and the energy consumption was reduced. On the other hand, more accurate recognition of the underloaded hosts led to prevent continuous shutdowns of the hosts, and thus prevented a lot of VM relocations. Fig. 2 shows the comparison of the LrMmt, the IqrMc, and the proposed methods based on their energy consumptions. The proposed method is 5% and 20% better than LrMmt and IqrMc, respectively. workload prediction in the proposed method prevents turning on the new hosts, and thus reduce the energy consumption. When the number of allocated resources of a host exceeds the threshold, one or more VM(s) should be migrated to the other hosts. If a method cannot find a proper host for the migrated VM(s) a new host must be turned on for the VM(s) that cause increasing the energy consumption of the data center. The proposed method predicts the future status of the resources of the host, and if it detects that it will return to the normal state within a short time, it will not consider it as an overloaded host. Therefore, the proposed method prevents VM migrations to avoid further costs.

Figure 2
Energy consumption (kWh) Fig. 3 shows the comparison of LrMmt, IqrMc, and the proposed method in terms of the SLA. The proposed method performs better than the both LrMmt and IqrMc methods. The proposed method performs better in terms of the SLA parameter than the LrMmt, because of improving the number of VM migrations. Number of VM migrations

Conclusions
One of the main concerns of cloud data centers is to improve energy efficiency and to increase the profitability of these data centers, while The proposed method detects the host's threshold by predicting its threshold based on the host's current conditions and its history to prevent unwanted migrations, and thus addresses the performance degradation that occurs due to the VM migrations. The proposed method offers a higher level of SLA than the IqrMc, caused by hosting more VMs. Consequently, more VMs tend to compete for resources which in some cases cause a shortage of resources.

Conclusions
One of the main concerns of cloud data centers is to improve energy efficiency and to increase the profit-ability of these data centers, while keeping the quality of service at an acceptable level. In the proposed method of this paper, the problem of virtual machine (VM) consolidation was focused by considering host overload and host underload detection. The proposed method mapped the future state of the data center by predicting the hosts' workloads, and then based on these predictions, decided to specify a host as an overloaded or underloaded host. In the proposed method, an exponential smoothing technique was also leveraged to predict host utilization to make a better decision. Therefore, improper migrations were avoided, and the energy consumption was reduced. On the other hand, more accurate recognition of the underloaded hosts led to prevent continuous shutdowns of the hosts, and thus prevented a lot of VM relocations.