Several systematic and exhaustive dynamic testing techniques for concurrent programs have been developed to deal with the nondeterministic behavior of concurrent programs [30,31,32,33,34,35,36,37]. These techniques are model-checking free in that they do not need to employ a model checker to explore the feasible interleavings of the execution of the concurrent program. They explore interleavings of a concurrent program by systematically switching threads at synchronization points. However, the common assumption of these techniques is that the execution of the target concurrent program has a finite number of feasible SYN-sequences. Problems are encountered
when applying dynamic testing to terminating 2 concurrent programs where synchronization events are located in iterative statements, such as while loops, which might make the number of feasible SYN-sequences infinite. The testing process cannot stop because the testing tool tries to enumerate all the feasible SYN-sequences.
a is a shared variable whose initial value is 0.
Process P0 Process P1
a is a shared variable whose initial value is 0.
Process P0 Process P1
Figure 1. Two concurrent programs with infinite SYN-sequences
We give two examples to illustrate that terminating concurrent programs with synchronization events in iterative statements may have an infinite number of SYN-sequences. Note that the synchronization operations we consider in the two examples are shared variable read and write operations. In the first example, the number of SYN-sequences of the concurrent program is infinite under an unfair schedule. In Figure 1A, consider only statements S0,0, S0,1, and S1,0. One of the shortest execution sequences of this concurrent program is {S1,0, S0,0, S0,1}, and a longer one is {S0,0, S0,1, S1,0, S0,0, S0,1}. It is obvious that the execution of S1,0 is the condition required to allow the busy-waiting loop of statements S0,0 and S0,1 to exit. It is trivial that the loop within statements S0,0 and S0,1 is not infinite since S1,0 of process P1 will eventually be executed in the normal task scheduling of the operating system. In order for dynamic testing to
2 The execution of a terminating concurrent program is expected to terminate. In this thesis, we will not be concerned with nonterminating programs such as reactive systems which need to interact with their environment frequently and often do not terminate.
6
exhaustively execute all of the feasible SYN-sequences, the loop for statements S0,0 and S0,1 must be iterated an infinite number of times. That is, it should test execution sequences {(S0,0, S0,1,)i S1,0, S0,0, S0,1}, where 0≤i≤∞ (note that (S0,0, S0,1,)i means that
“S0,0, S0,1,” repeats i times). It is obviously impossible to exhaust all the feasible SYN-sequences since there is an infinite number of them. Note that the schedule is unfair if process P1 is unable to make progress for a long time.
In the second example, the infinity of SYN-sequences is caused by busy-waiting loops of two processes. For the concurrent program shown in Figure 1B, one of the shortest execution sequences of this program is {S1,0, S0,0, S0,1, S1,1}. However, we can easily see that the feasible execution sequences contain {(S0,0, S1,0, S0,1, S1,1,)i S1,0, S0,0, S0,1, S1,1}, where 0≤i≤∞. This makes the exhaustive exploration of all feasible SYN-sequences impossible.
This thesis would propose a systematic and model-checking-free scheme for performing dynamic testing on terminating concurrent programs with an infinite number of SYN-sequences. The proposed scheme can be applied to terminating concurrent programs that have cyclic state spaces due to the use of iterative statements such as busy-waiting loops. Our scheme is called dynamic effective testing, and it can support the testing of concurrent programs where the synchronization events are located in any type of iterative statement. The scheme can also be applied to programs where the repetitive issues of synchronization events are caused by recursive calls. The synchronization operations modeled in this thesis are shared memory read and write operations. In general, modeling all process synchronizations in a concurrent program as operations on shared data (or shared memory) is not restrictive, since many
synchronization primitives can be reduced to operations on shared data [38]. To demonstrate the portability of our scheme, we also report experiences and experimental results obtained in porting the proposed testing scheme to testing semaphore-based concurrent programs and service-oriented architecture applications.
The basic idea of dynamic effective testing is to apply a subset of the feasible tests, since there is usually an infinite number of feasible SYN-sequences. For a concurrent program P with input X, if there is a finite number of feasible execution states, it can be shown that the execution of P with X can be represented by a deterministic finite automaton3 (DFA) F in which the finite sets of states are the feasible execution states and the transitions are labeled with events. The SYN-sequences of P with X are usually infinite if there exists a cycle in F. This makes the exhaustive testing of P with X impossible.
Also, it is usually impossible to traverse all the states in F in a single test of P with X.
The effective test set of P with X consists of a finite set of tests ET of P with X. The associated SYN-sequences of ET traverse all the states and transitions in F. Dynamic effective testing of P with input X involves dynamic testing of an effective test set of P with input X. Actually, dynamic effective testing is a scheme that controls the execution of the target program to cover some effective test set so that we can conduct a state- and transition-cover testing for the concurrent program with a given input.
The method of dynamic effective testing proposed in this thesis only has to analyze the SYN-sequences that are collected during the dynamic testing of the concurrent program – static analysis of syntax and semantics of the target concurrent program is unnecessary. The analysis of SYN-sequences has two goals. First, we have to perform
3 Note that the DFA is not equivalent to the Kripke structure. We define our DFA in Chapter 5.
8
race analysis to explore interleavings which are used to guide the tested program to explore different states. Second, we can obtain a sequence of execution states from SYN-sequences so that we can identify reiterated states from them. According to these identified reiterated states, we may either truncate some events from the SYN-sequence or abandon the entire SYN-sequence, so as to achieve state-cover testing from an infinite number of SYN-sequences. Note that we define states traversed in a SYN-sequence S as states which can be reached by all the possible totally ordered sequences4 of S. Also, the operation of dynamic effective testing for a concurrent program P with input X does not need to construct the associated DFA of P with X. That is, we do not need to build up the model or Kripke structure of the target concurrent program before or during performing dynamic effective testing; therefore, it is model-checking free. As long as the SYN-sequences have an identical format, dynamic effective testing can be applied to different programming languages using a single implementation of the SYN-sequence analyzer; thus, it is language independent.
To the best of our knowledge, our scheme is the first systematic and model-checking-free dynamic testing scheme that can apply state- and transition-cover dynamic testing on a nondeterministic concurrent program containing an infinite number of SYN-sequences without requiring static analysis of the source code or the assistance of model checking. In addition to performing dynamic testing on concurrent programs, the effective test set collected during dynamic effective testing of a concurrent program can be used to construct the DFA that represents the execution of the program. The DFA can then be used to support debugging of the program.
4 Refer to Section 5.4.2 for the totally ordered sequences of a SYN-sequence.
This thesis is organized as follows. Chapter 2 surveys previous work on dynamic testing for concurrent programs. In Chapter 3 we present a problem called boundless state reiteration that occurs when a concurrent testing tool attempts to explore interleavings for a concurrent program with an infinite number of SYN-sequences or interleavings.
The effective test set and dynamic effective testing are formally defined in Chapter 4.
The framework and related algorithms for implementing dynamic effective testing are presented in Chapter 5. Chapter 6 shows how to test a single-port-multi-receiver message-passing program based on the extension of race table algorithm. The implementation details and experimental results are presented in Chapter 7, and conclusions are drawn in Chapter 8.
10