Task Group Allocation in Clustering-Based Multiple Workflows Scheduling

This section evaluates the proposed task group allocation methods for clustering-based multiple workflow scheduling, including enhanced best-fit task group allocation, adjustable task group allocation, and adaptive distributed task group allocation. The proposed methods are compared with the pure best-fit approach [13], PCH [21][22], and the distributed gap search approach [18].

Figures 6.16, 6.17, and 6.18 compare the continuous task group allocation methods under different CCR values. The experiments evaluate the proposed approaches on a 30-resource homogeneous system running 100 online workflows. The results indicate that in general our approaches, enhanced best-fit and adjustable best-fit, outperform previous methods and the best σ value varies under different CCR conditions. Experimental results show that σ=0.8 or 0.9 leads to the best performance improvement when CCR is 1 and σ=0.3&0.4 achieves the most obvious performance improvement when CCR is 10. When CCR is 0.1, different σ values make negligible performance difference. The results also indicate that larger CCR values lead to more significant performance improvement since larger CCR values imply longer idle time slots to accommodate task groups under continuous task group allocation, thus allowing different allocation decisions. On the other hand, when CCR is small, there are few idle time slots to accommodate entire task groups under continuous task group allocation. Therefore, different task group allocation methods make little difference. Looking at the performance of the adjustable best-fit approach under different CCR values, another insight is that the effect of the

time slot fitness is more important than EFT when CCR is medium, since higher σ values lead to better performance as shown in Figure 6.17. This is because idle time slots large enough for task groups are limited under such CCR values, and thus efficient utilization of those time slots is crucial. On the other hand, when CCR is high there are plenty of large idle time slots.

Therefore, the effect of EFT becomes more important as shown in Figure 6.18 where smaller σ values achieve better performance.

Figure 6.16:100 workflows on 30-resource homogeneous system with continuous task group allocation (CCR=0.1)

Figure 6.17:100 workflows on 30-resource homogeneous system with continuous task

group allocation (CCR=1)

Figure 6.18:100 workflows on 30-resource homogeneous system with continuous task

group allocation (CCR=10)

under different CCR values. The results indicate that our adaptive distributed task group allocation outperforms the original distributed task group allocation method [18] significantly under various CCR conditions. Similar to the experiments for continuous task group allocation, when CCR is medium the effect of time slot fitness is crucial, as shown in Figure 6.20, and the effect of EFT becomes more important when CCR increases, as shown in Figure 6.21.

Figure 6.19: 100 workflows on 30-resource homogeneous system for different distributed task group allocation methods (CCR=0.1)

Figure 6.20: 100 workflows on 30-resource homogeneous system for different distributed

task group allocation methods (CCR=1)

Figure 6.21: 100 workflows on 30-resource homogeneous system for different distributed

task group allocation methods (CCR=10) The following presents the evaluation of the proposed task group allocation methods in a speed-heterogeneous parallel system. Figures 6.22, 6.23, and 6.24 evaluate the proposed enhanced best-fit and adjustable best-fit approaches. Figures 6.25, 6.26, 6.27 compare the

proposed adaptive distributed task group allocation with the original distributed task group allocation method [17]. Similar to the results for homogeneous systems, our approaches, in general, outperform existing task group allocation methods in terms of average makespan.

One thing to be noted is that the pure best-fit approach proposed in [13] performs poorly for larger CCR. When CCR is 0.1, PCH [21][22] and the pure best-fit approach achieve almost the same performance since in such case there are very few idle time slots large enough for allocating task groups. As CCR increases to 1, the pure best-fit approach outperforms PCH slightly, demonstrating the benefits of best-fit allocation through raising resource utilization rates. However, when CCR becomes even larger, 10 in this case, the pure-best approach performs poorly, compared to PCH. This is because for large CCR there are plenty of idle time slots large enough for allocating task groups. In such cases, best-fit and first-fit allocation would lead to similar resource utilization rate, but the first-fit principle in PCH can achieve better performance since the pure best-fit approach might delay tasks’ start time and in turn degrade the performance of entire workflow due to skipping some earlier available time slots to find the fittest one. On the other hand, out enhanced best-fit and adjustable best-fit approaches can deliver consistently better performance with all CCR values.

Figure 6.23: 100 workflows on 30-resource heterogeneous system for continuous task

group allocation (CCR=1)

Figure 6.24: 100 workflows on 30-resource heterogeneous system for continuous task

group allocation (CCR=10)

Figure 6.25: 100 workflows on 30-resource heterogeneous system for distributed task group allocation (CCR=0.1)

Figure 6.26: 100 workflows on 30-resource heterogeneous system for distributed task

group allocation (CCR=1)

Figure 6.27: 100 workflows on 30-resource heterogeneous system for distributed task

group allocation (CCR=10)

Figure 6.28 shows the performance evaluation of the proposed adaptive distributed task group allocation, in terms of average makespan reduction, compared to the original distributed task group allocation method, with different numbers of online workflows. The experimental results indicate that the performance improvement increases as the system load rises, implied by more DAG’s. This is because distributed task group allocation is an auxiliary method which will be applied only when continuous task group allocation fails. As system load rises, the idle time slots become smaller and the occurrences of distributed task group allocation increase, leading to larger accumulated performance improvement.

Figure 6.28: Effect of different numbers of DAG’s (CCR=10)

In summary, our task group allocation methods for clustering-based multiple workflow scheduling outperform existing approaches, achieving up to 15.5% performance improvement.