Performance Results

Figures 4.1 to 4.4 compare the three service selection approaches across different SLA

service under the four scenarios, respectively. The results indicate that our two approaches, CD-ICA and OSSBNP, outperform the previous approach, DD-ICA, in [7] significantly in all the four scenarios.

Figure 4.1: Costs under scenario 1

Figure 4.2: Costs under scenario 2

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605 DD-ICA 849.8 849.8 849.8 849.8 849.8 849.2 CD-ICA 650.3084 650.3632 650.4987 650.8288 651.5415 652.9037 OSSBNP 650.3042 650.3316 650.3992 650.5645 650.9208 651.6017

0

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 DD-ICA 1264.78 1246.215 1239.978 1179.896 1111.776 1037.257 CD-ICA 651.0651 651.4815 652.901 655.8483 660.9628 668.7253 OSSBNP 650.6825 650.8907 651.6005 653.0738 655.6314 659.5129

0

Figure 4.3: Costs under scenario 3

Figure 4.4: Costs under scenario 4

Figures 4.5 to 4.8 compare the three service selection approaches in terms of service response time under the four scenarios, respectively. The penalty is two dollars per time units beyond the SLA constraint. The results reveal that in all scenarios, our two approaches lead to longer service response time compared to DD-ICA. This implies that our approaches could effectively sacrifice service response time under the SLA constraints for using cheaper services to reduce the total costs of providing composite cloud services.

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 DD-ICA 853.1 861.3 975.5463 1181.91 1295.238 1314.635 CD-ICA 650.6073 651.9323 656.6969 661.4979 659.6243 661.7523 OSSBNP 650.3037 650.3161 650.408 650.8228 652.205 655.4916

2000

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605 DD-ICA 1271.8 1271.8 1271.802 1271.829 1271.877 1272.042 CD-ICA 652.0443 652.901 654.1315 655.8483 658.1101 660.9628 OSSBNP 651.1721 651.6005 652.2155 653.0738 654.2049 655.6314

0

Figure 4.5: Service response time under scenario 1

Figure 4.6: Service response time under scenario 2

Figure 4.7: Service response time under scenario 3

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605 DD-ICA 578.744 578.711 578.589 578.592 578.72 578.553 CD-ICA 650.3084 650.3632 650.4987 650.8288 651.5415 652.9037 OSSBNP 650.3042 650.3316 650.3992 650.5645 650.9208 651.6017

540

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 OSSBNP 522.3165 522.3165 522.3165 522.3165 522.3165 522.3165 CD-ICA 522.3165 522.3165 522.3165 522.3165 522.3165 522.3165 DD-ICA 520.4901 520.6006 520.314 519.8586 519.6464 519.1677

0

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 DD-ICA 519.411 519.6995 519.2639 519.3159 519.5141 519.6567 CD-ICA 521.5493 521.5501 521.5165 521.4835 521.5002 521.5471 OSSBNP 521.5567 521.5567 521.5567 521.5567 521.5567 521.5567

518

Figure 4.8: Service response time under scenario 4

Figures 4.9 to 4.12 compare the three service selection approaches in terms of the total amount of time exceeding the SLA constraint, which is directly proportional to the penalty costs, under the four scenarios, respectively. The results indicate that compared to DD-ICA, our approaches are more capable of achieving a global optimization of total costs, not only trying to minimizing penalty costs.

Figure 4.9: SLA violation time under scenario 1

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605 DD-ICA 540.7761 540.7586 540.7536 540.7596 540.7408 540.7747 CD-ICA 602.3161 602.3161 602.3161 602.3161 602.3161 602.3161 OSSBNP 602.3161 602.3161 602.3161 602.3161 602.3161 602.3161

500520 540560 580600 620

response time

scenario4

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605

DD-ICA 0 0 0 0 0 0

CD-ICA 2.11853 15.81903 49.65759 132.1982 310.3793 650.9174 OSSBNP 2.11853 15.81903 49.65759 132.1982 310.3793 650.9174

1000 200300 400500 600700

SLA violation time

scenario1

Figure 4.10: SLA violation time under scenario 2

Figure 4.11: SLA violation time under scenario 3

Figure 4.12: SLA violation time under scenario 4

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 DD-ICA 19.99445 28.68512 144.585 374.1164 1044.199 2264.344 CD-ICA 191.282 295.3859 650.2747 1387.012 2665.653 4606.285 OSSBNP 191.282 295.3859 650.2747 1387.012 2665.653 4606.285

0

SLA=539 SLA=537 SLA=533 SLA=529 SLA=525 SLA=521 DD-ICA 0 0 11.57117 27.41956 259.5523 1074.383 CD-ICA 1.83252 8.058655 49.20526 249.4568 931.033 2587.958 OSSBNP 1.83252 8.058655 53.98444 261.335 952.4529 2595.633

0

SLA=615 SLA=613 SLA=611 SLA=609 SLA=607 SLA=605 DD-ICA 0 0 0.588013 7.26825 19.26825 35.40527 CD-ICA 436.0644 650.2748 957.8309 1387.012 1952.467 2665.653 OSSBNP 436.0644 650.2748 957.8309 1387.012 1952.467 2665.653

0

Figures 13 to 16 compare the three service selection approaches across different penalties per time unit in terms of total costs under the four scenarios, respectively. The results show that our two approaches still outperform DD-ICA in such situations.

Figure 4.13: Comparison across different penalties under scenario 1

Figure 4.14: Comparison across different penalties under scenario 2

PEN=2 PEN=12 PEN=22 PEN=32 PEN=42 PEN=52 DD-ICA 849.8 849.8 849.8 849.8 849.8 849.8 CD-ICA 650.3084 650.3509 650.3932 650.4355 650.478 650.5203 OSSBNP 650.3042 650.3254 650.3466 650.3678 650.389 650.4101

0

PEN=2 PEN=12 PEN=22 PEN=32 PEN=42 PEN=52 DD-ICA 1265.484 1270.633 1271.071 1290.107 1278.63 1305.93 CD-ICA 651.4815 657.3894 663.2969 669.2046 675.1125 681.0201 OSSBNP 650.8907 653.8447 653.8447 659.7524 662.7061 665.6599

2000

Figure 4.15: Comparison across different penalties under scenario 3

Figure 4.16: Comparison across different penalties under scenario 4

Figures 17 to 20 compare the three service selection approaches across three situations which differ in the smallest price difference between services. The results show that our approaches perform well under all situations.

PEN=2 PEN=12 PEN=22 PEN=32 PEN=42 PEN=52 DD-ICA 984.1252 979.8658 978.8 972.8 989.8 985.1033 CD-ICA 657.6113 680.9586 681.0753 686.5549 689.5368 680.3121 OSSBNP 650.408 650.9478 652.0275 652.5674 653.1072 650.408

0

PEN=2 PEN=12 PEN=22 PEN=32 PEN=42 PEN=52 DD-ICA 1271.741 1273.15 1273.972 1274.665 1275.465 1275.03 CD-ICA 660.9628 714.2761 767.5887 820.9018 874.215 927.5281 OSSBNP 655.6314 682.288 708.9443 735.6011 762.2574 788.9138

0

Figure 4.17: Comparison across different price configurations under scenario 1

Figure 4.18: Comparison across different price configurations under scenario 2

gap:100 gap:200 gap:300

Figure 4.20: Comparison across different price configurations under scenario 4

Figures 21 to 24 compare the three service selection approaches across different situations where services have different ranges of standard deviation of service response time. The results show that our approaches consistently outperform DD-ICA in all situations.

Figure 4.21: Comparison across different ranges of standard deviation of service response time under scenario 1 DD-ICA 849.8 1379.484 1389.17 1411.974 1451.553 1476.904 CD-ICA 650.3084 671.83575 709.5031 749.6134 790.3768 831.6997 OSSBNP 650.3042 661.0681 770.3431 789.1599 808.8853 972.1349

0

Figure 4.22: Comparison across different ranges of standard deviation of service response time under scenario 2

Figure 4.23: Comparison across different ranges of standard deviation of service response time under scenario 3

σ=1~5 σ=11~15 σ=21~25 σ=31~35 σ=41~45 σ=51~55 DD-ICA 1265.484 1127.2 1185.634 1301.161 1387.356 1459.904 CD-ICA 651.4815 679.1949 717.498 757.567 798.5086 840.4199 OSSBNP 650.8907 664.7473 683.8994 703.9338 724.4042 745.36

0 DD-ICA 984.1252 1339.675 1418.845 1507.45 1535.531 1584.256 CD-ICA 657.6113 678.804 716.8978 757.1491 798.1361 840.2138 OSSBNP 650.408 663.8412 683.3677 703.6299 724.2174 745.2067

0

Figure 4.24: Comparison across different ranges of standard deviation of service response time under scenario 4

Although in previous experiments, the performance of OSSBNP is very close to CD-ICA, OSSBNP has a particular advantage over the other approaches which is the capability of enforcing an upper bound on QoS violation while minimizing the costs. This advantage can be useful since in many cases the SLA might contain the enforcement of QoS violation ratio over a specific period of time or a specific number of service requests. In such cases, we can establish such enforcement in OSSBNP, while the other two approaches cannot do that. The experimental results, shown in Figure 25 where SLA contains a 5% violation limit, indicate that OSSBNP can effectively limit the SLA violation ratio under such constraint, less than 50 violations among 1000 repeated experiments, while the other two approaches cannot provide such guarantee. Therefore, OSSBNP has a unique advantage over the other approaches although its required execution time is about 10 times longer than CD-ICA, as shown in Figure 4.26.

σ=1~5 σ=11~15 σ=21~25 σ=31~35 σ=41~45 σ=51~55 DD-ICA 1271.741 1242.386 1215.926 1276.347 1354.913 1401.183 CD-ICA 660.9628 701.0797 742.4259 783.9233 825.4772 867.5408 OSSBNP 655.6314 675.6897 696.3633 717.1118 1015.636 982.7726

2000 400600 1000800 12001400 1600

COST

scenario4

Figure 4.25: Evaluation of the capability of SLA enforcement

Figure 4.26: Comparison of required execution time of different approaches

scenario1 scenario2 scenario3 scenario4

DD-ICA 235 63 6 215

CD-ICA 575 214 64 624

OSSBNP 2 19 1 0

0 100 200 300 400 500 600 700

violations

0 10 20 30 40 50 60

DD-ICA CD-ICA OSSBNP

execution time

DD-ICA CD-ICA OSSBNP