Minimum Test Cost Problem in Collaborative Testing

Under Internet environment, there are a large number of free and experienced folk human resources. Collaborative testing (or called Beta test) is usually used in online game and open source software to reduce test cost in software development stage. Based on the idea of beta test, we use folk testers in Internet to help us finding and reporting bugs. There are some constraints while applying collaborative testing. First, folk testers may not start to test at the

0 200 400 600

Tester A Tester B Tester C

0 100 200 300 400 Tester A

Tester B

Tester C

33

same time and they may delay to start the next test job after finishing current jobs. Second, we only consider those folk testers contributing themselves to collaborative testing. Therefore, each folk tester solves one sub-problem at least. Third, due to different complexity of each sub-problem, each sub-problem has distinct support threshold for completion. And then, the quality of folk testers (trustworthy) is different in Internet; therefore, it needs to consider the accumulation of the testers‘ trustworthy, rather than the number of testing. Final, each folk tester has different workload.

Before presenting our problem formulation, we first introduce the notations in Table 6.

The variables i represents the ith folk tester, the variable j represents jth sub-problem, and the variable k represents k_th assignment of the sub-problems. We use the binary variable. σ_ijk equals to 1 represents that the jth sub-problem is assigned to the ith folk tester in the kth

assignment. Otherwise, σ_ijk equals to 0. T_De(i, j) stands for the delay time of the i_th folk tester start to solve the jth sub-problem, TEx(I, j) stands for the jth sub-problem execution time of the i_th folk tester, and T (i, j) stands for the j_th sub-problem completion time of i_th folk tester. W_i represents the trustworthy of ith folk tester and Hi represents the workload of the ith folk tester.

ST_j represents the support threshold of j_th sub-problem.

34

Table 6. Notations of MTCP in collaborative testing i = ith folk tester

j = j_th sub-problem

k= kth assignment of the sub-problems

σijk= i_th folk tester does the j_th sub-problem in the k_th assignment TDe(i, j)= the jth sub-problem delay time of the ith folk tester T_Ex(i, j)= the j_th sub-problem execution time of the i_th folk tester T (i, j)= the jth sub-problem completion time of ith folk tester W_i= the trustworthy weight of i_th folk tester

Hi= the available time limit of the ith folk tester ST_j=support threshold of the j_th sub-problem

4.2.1 Problem formulation

The Minimum Test Cost Problem (MTCP) in collaborative testing can be formulated as IP-formulation. The objective function is the minimum of due time on different assignment, when due time is the maximum of the sum of sub-problem‘s time of each folk tester. The constraint 6 is the complete condition that each sub-problem need to be tested at least support threshold. The formulation of MTCP is presented below:

35

Definition 7: Minimum Test Cost Problem (MTCP) definition Objective function:

36

4.2.2 NP-Complete problem

In the section, we introduce an NP-complete problem, Job Assignment Problem (JAP) [30]. Then, JAP can be reduced in polynomial time to MTCP to complete the proof as followed.

The corresponding decision problem of Minimum Test Cost Problem (MTCP):

MTCP = {<D, J, U, ST, W, H, T, t>}

D= (S, δ, S0) is a directed tree.

J is a set of sub-problems in D.

U is a set of folk testers.

ST is a function form JR⁺ W is a function form U(0, 1]

H is a function form UR⁺ T is a function form UJR⁺

And there is an assignment with due time at most t

Theorem 1

Minimum Test Cost Problem is NP-Complete.

Proof:

First, we show that MTCP belongs to NP. Given an instance of the problem, the verification algorithm checks that sum of trustworthy W_i of assigned sub-problems of every folk tester i of each sub-problem j exceeds the support threshold STj, the sum of the completion time of assigned sub-problems of each folk tester i does not exceed H_i, the assignment of the sub-problems differs from the other assignment of the sub-problems, and checks whether the maximum of the sum of the completion time of assigned sub-problems of each folk tester is at most t. This process can certainly be done in polynomial time.

37

Second, to prove that MTCP is NP-Hard, we show that JAP_p MTCP. Let G=(V‘,E‘), J‘={Pi|i=1,…,n}, U‘={Ui|i=1,…,m}, d(i, j)=0, the confirm function S(k) ∀k∈Pi, the trustworthy function w by w(i, j) where i ∈U‘and j ∈J‘, the human resource function H by H(i)

∀i∈U‘, the execution time function T by T(Pi, j) ∀Pi∈J‘, j∈U‘, and the maximum total cost at most t of JAP. We construct an instance of MTCP as follows. We form the tree D= (S, δ, S₀) where S=Pi, δ=0, S0=Pi and we define the test sub-problem set J=J‘, the folk tester U=U‘, the support threshold function ST by ST_j=1 ∀j∈J, the trustworthy function W by W_i=w(i,j) ∀i∈U, the available time function H by Hi=H(i) ∀i∈U, the completion time function T by T(i, j)=T(Pi, j) ∀i∈U, ∀j∈J.

The instance of MTCP is then <D, J, U, ST, W, H, T, t>, where is easily formed in polynomial time.

We now show that graph G‘ has an assignment δ of the maximum total cost at t if and only if the tree D‘ has an assignmentσ‗ of the maximum due time at most t. Suppose there is an assignmentδwith maximum cost at most t. Therefore, there exists an assignment σ‘ such that _ijk 1 if _ij 1, the support threshold ST_j=1 ∀j∈J , the trustworthy weight of folk tester W_i=w(i,j) ∀i∈U, the available time Hi=H(i), the completion time T(i, j)=T(P_i,j) ∀i∈U,

∀j∈J. Thus, the assignment σ‘ is feasible solution and the maximum due time is t. Conversely, suppose that there is an assignment σ‘ with the maximum due time is t. Then, there existed an assignment a such that_ij _ijk, the trustworthy w(i, j)=Wi ∀i∈U‘ and j∈J‘, the human resource H(i)=Hi ∀i∈U‘, the execution time T(Pi, j)=T(i,j) ∀Pi∈J‘, ∀j∈U‘. Thus, the assignment is a feasible solution and the maximum total cost is t. Hence, MTCP is NP-Complete #

38