Testing-Coverage Dependent Software Reliability Growth Modeling and Its Applications
6
0
0
全文
(2) cally, Fujiwara and Yamada [9] and Malaiya et al. [10] have proposed a software reliability growth model with the testing-coverage, respectively, and Pham and Zhang [11] also proposed an NHPP model and software cost models with the testing-coverage. However, our approach is di erent from the researches above in terms of the relationship between the testingcoverage and the software reliability growth process. First we propose an alternative testingcoverage function to describe time-dependent behavior of a testing-coverage maturity process. Then, we formulate the relationship between the attained testing-coverage and the number of detected faults. Estimation methods for unknown parameters of the alternative testing-coverage function and our SRGM are also discussed, respectively. Finally, we derive several software reliability assessment measures which are useful metrics for quantitative software reliability assessment, and show numerical examples of software reliability analysis based on the proposed model by using an actual data set.. 1. Introduction. Software reliability assessment is one of the important issues to produce reliable software systems. A software reliability growth model (abbreviated as SRGM) [1{4] has been utilized as one of the fundamental techniques for quantitative assessment of software reliability. The SRGM describes the software fault-detection phenomenon or the software failure-occurrence phenomenon by applying stochastic and statistical theories. Especially, it is well-known that a nonhomogeneous Poisson process model (abbreviated as NHPP model) can characterize the software reliability growth process simply by supposing a suitable mean value function of an NHPP. Accordingly, the NHPP model has been utilized for software reliability assessment in many software houses and computer manufacturers from high applicability and simplicity of the model structure of an NHPP point of view [5]. Up to now, several speci
(3) c NHPP models have been also proposed such as imperfect debugging SRGM's. 2. Testing-Coverage. Testing-coverage is one of the important measures to evaluate the quality of testing and tested software products. The typical testing-coverage measures are classi
(4) ed into several types in terms of control ow testing as follows: statement coverage, branch coverage, path coverage. For example, the statement coverage is measured on the basis of the statement-paths that have been executed at least once by the test-cases. This is 938.
(5) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. equation by solving the di erential equation in Eq.(1): ;bsta t (4) C (t) = 1(1+ ;z ee;bsta t) ; where z = (1 ; r)=r. We call Eq.(4) an alternative testing-coverage function with testing-skill in this paper. The in ection point of the alternative testing-coverage function in Eq.(4) is derived as t = log z : (5). called C0 testing-coverage measure. Most SRGM's are ordinally developed by characterizing the relationship between the testingtime and the number of detected software faults. Accordingly, we need to characterize the relationship between the testing-time and the testingcoverage to develop an SRGM considering with testing-coverage maturity process
(6) rst. In this section we discuss basic concepts to describe the time-dependent behavior of the testing-coverage maturity process.. bsta. 2.1. Formulation. Then, we have. First we propose a basic equation to describe the time-dependent behavior of the testingcoverage maturity process, which is called as an alternative testing-coverage function in this paper. For developing the function, we assume that the testing-coverage maturity rate at any time is proportional to the di erence between the target value and the current one of testing-coverage. Letting C (t) be the ratio of testing-coverage attained by arbitrary testing time t, we can derive the following di erential equation from the assumption:. dC (t) =
(7) (t)[ ; C (t)] dt ( > 0;
(8) (t) > 0);. 3. Software Reliability Modeling 3.1. NHPP model. (1). Time-dependent behavior of a software faultdetection process or a failure-occurrence phenomenon, i.e., a software reliability growth process, has been formulated by using a counting process ordinarily. An NHPP which is one of the counting processes is widely used for the software reliability growth modeling. A counting process fN (t); t 0g is said to be an NHPP with mean value function H (t) if N (t) obeys the following distribution: 8 fH (t)gn exp f;H (t)g > > Pr f N ( t ) = n g = > < n! Z t (n = 0; 1; 2; ); > > > : H (t) = h( )d ; 0 (7) where h( ) is the intensity function representing the instantaneous fault-detection rate. The timedependent behavior of the fault-detection process is characterized by the mean value function H (t) which means the expected number of faults detected in the time-interval (0; t]. In this paper we assume that the expected number of faults detected at testing time t is proportional to the expected current fault content. Accordingly, we can obtain the following di erential equation: dH (t) = b(t)[a ; H (t)]; (8). 2.2. Formulation with testing-skill. The testing-coverage discussed above helps software development managers to evaluate whether the test-cases have been designed to detect faults e ectively. Accordingly, the timedependent behavior of the testing-coverage depends on a testing-skill of test-case designers. In this paper we assume that the testing-skill of testcase designers increases as the ratio of testing progress goes on. Supposing that the testing-skill factor for the test-case designers is given as sta. (2). we extend
(9) (t) in Eq.(1) as follows:.
(10) (t) B (C (t)) = bsta fr + (1 ; r) C (t) g;. . (6) 2 z In Eq.(4), C (t) indicates an exponential growth curve if r = 1 for the case that the internal program structure is simple, and indicates an Sshaped one called a logistic curve if r ! 0 for the case that the internal program structure is complex and the testing-e ort increases more and more as the testing time goes on.. where indicates the target value of testingcoverage to be attained, and
(11) (t) the testingcoverage maturity ratio at arbitrary testing time t. We can easily obtain the alternative testingcoverage function by solving the above di erential equation with respect to C (t).. r = bbini ;. . C (t ) = 1 ; 1 :. (3). dt where a represents the expected initial fault content in the software system, and b(t) the faultdetection rate per fault at testing time t. The. where bini represents the initial testing-skill factor of the test-case designers, bsta the steady-state one, and r the in ection coecient. Substituting Eq.(3) into Eq.(1), we can obtain the following 939.
(12) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. transformed into an integrable di erence equation. Concretely speaking,
(13) rst we derive the following integrable di erence equation via using Hirota's bilinearization methods [14] from the differential equation in Eq.(1) with
(14) (t) in Eq.(3):. mean value function can be derived as follows by solving the di erential equation in Eq.(8): . H (t) = a 1 ; exp[;. Z. 0. t. . b( )d ] :. (9). Cn+1 ; Cn = r bsta + bsta (12 ; 2r) [Cn + Cn+1 ] ; bsta (1 ; r) Cn Cn+1 :. Most NHPP models can be characterized by b(t) in Eq.(8) or Eq.(9), i.e., Eq.(9) is so-called a GoelOkumoto SRGM [13] when b(t) b.. 3.2. Modeling with testing-coverage. (13). In this section we propose a software reliability growth model considering with the testingcoverage maturity process based on the NHPP. First we formulate the relationship between the testing-coverage muturity process and the expected number of detected faults. Supposing that the expected number of faults detected at testing time t is proportional to the expected current fault content and the attained testing-coverage at testing time t, we can formulate the relationship as the following equation by using Eq.(4):. dHC (t) dt. ,. dC (t) = s[a ; H (t)]; C dt. Solving the above di erence equation yields an exact solution of Cn representing the testingcoverage attained by nth testing-period as h. (10). where s is the fault-detection rate per attained testing-coverage and per fault. That is, b(t) in Eq.(8) or (9) is given as b(t) bC (t) = s c(t) in which c(t) dC (t)=dt. Then, we can obtain the following solution by solving the di erential equation in Eq.(10) with respect to HC (t):. HC (t) = a [1 ; exp f;s C (t)g] :. . n i. 1; 2 bsta 1 ; 1+ 1 b 1; 12 b sta n Cn = 1 + z 1+ 122 bsta sta (z > 0; 0 r 1) ; (14) where represents the constant time-interval, that is, t = n. We should note that Eq.(13) conserves the characteristics of the di erential equation in Eq.(1) with
(15) (t) in Eq.(3). That is, the di erence equation in Eq.(13) has an exact solution, and, as ! 0, Eqs.(13) and (14) converge on the original di erential equation in Eq.(1) with
(16) (t) in Eq.(3) and the exact solution in Eq.(4) 1. of the di erential equation, respectively. These properties above are features of a integrable difference equation derived by using the Hirota's bilinearization methods. From Eq.(13), a regression equation to get parameter estimates can be derived as the following: Yn = A + B1 Kn + B2 Ln ; (15) where 8 Yn = Cn+1 ; Cn > > > > K =C +C > > < L n = Cn C n+1 n n n+1 (16) > A = rbsta > > > B = bsta (1 ; 2r) 2 > > : B1 = ;b 2 sta (1 ; r ) :. (11). We de
(17) ne the NHPP model with the mean value function in Eq.(11) as an SRGM with the testingcoverage. It is noted that the mean value function with the testing-coverage in Eq.(11) has the following property: lim HC (t) = a (1 ; exp[;s ]) ; (12) t!1. which implies that HC (t) in Eq.(11) does not converge on the initial fault content a in the software system even if t ! 1. Therefore, fa ; HC (1)g represents the expected total fault content to be detected on the other testing-coverage factors.. Using Eq.(15), we can estimate Ab, Bc1 , and Bc2 by using the observed testing-coverage data, which are the estimates of A, B1 , and B2 , respectively. Therefore, we can obtain the parameter estimates b, bd b from Eq.(16) as follows: sta , and r. 4. Parameter Estimation. We discuss methods of parameter estimation for the alternative testing-coverage function in Eq.(4) and the SRGM in Eq.(11), respectively. We suppose that K data pairs (tk ; xk ; yk )(k = 0; 1; 2; ; K ) with respect to the total number of detected faults, yk , and the total attained testingcoverage, xk , during the time-interval (0; tk ] are observed. We
(18) rst discuss an estimation method for the alternative testing-coverage function in Eq.(4). The method of least-squares is applied to Eq.(1). 8 .q 2 > c b c c b > B ; A B ; B. b = A 1 2 1 > > > q < . c 2 bc (17) bd sta = 2 B1 ; AB2 > q . > > > > rb = 1 ; B c1 = B c1 2 ; AbB c2 : 2:. Yn , Kn , and Ln in Eq.(15) are independent of because is not used in calculating Yn , Kn , and Ln in Eq.(15). Hence, we can obtain the 940.
(19) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. same parameter estimates b, bd b, respecsta , and r tively, when we choose any value of . It is said that this method can get more accurate parameter estimates than the ordinary method of leastsquares [15]. Second we discuss an estimation method for the mean value function in Eq.(11). We use the method of maximum-likelihood to get the parameter estimates, ba and sb, in the mean value function. Then, the logarithmic likelihood function is given as ln L =. K X k. =1. 1 0.9. C0 Testing Coverage. 0.8. K X k. =1. 0.5 0.4 0.3 0.2. ln[yk ; yk;1 ];. Actual Fitted. 0 0. 2. 4. 6. 8. 10. 12. 14. 16. 18. 20. 22. 24. Testing Time (number of weeks). Fig. 1： The estimated alternative testingcoverage function (on the C0 testingcoverage measure).. (18). Cumulative Number of Detected Faults. from the properties of the NHPP. Furthermore, we can derive the following simultaneous equations by partially di erentiating the logarithmic likelihood function ln L with respect to parameters a and s: @ ln L = @ ln L = 0: (19). @a. 0.6. 0.1. (yk ; yk;1 ) ln[HC (tk ) ; HC (tk;1 )]. ; HC (tK ) ;. 0.7. @s. By solving the above simultaneous equations numerically, we can estimate ba and sb which are the estimates of a and s, respectively.. 5. Assessment Measures. 320 300 280 260 240 220 200 180 160 140 120 100 80 60 40 20 0. Actual Upper Limit Lower Limit Fitted 0. 2. 4. 6. 8. 10. 12. 14. 16. 18. 20. 22. 24. Testing Time (number of weeks). In this section we derive several software reliability assessment measures which are useful for quantitative assessment of software reliability and the progress of the software testing process. Speci
(20) cally, we derive the software reliability function, the instantaneous and cumulative MTBF's (MTBF : mean time between software failures).. Fig. 2： The estimated mean value function with its 95% con
(21) dence limits.. 5.3. Cumulative MTBF. The cumulative MTBF is also the substitution for the MTBF. The cumulative MTBF is approximately derived as. 5.1. Software reliability function. MTBFC (t) = Ht(t) :. Given that the testing or the user operation has been going up to time t, the probability that a software failure does not occur in the timeinterval (t; t + x](x 0; t 0) is derived as R(x j t) = exp[;fH (t + x) ; H (t)g]; (20) from Eq.(7). R(x j t) in Eq.(20) is called a software reliability function. We can estimate the software reliability by using this equation.. (22). If the instantaneous MTBF in Eq.(21) and the cumulative MTBF in Eq.(22) take on a large value, respectively, then we decide that the software system becomes more reliable.. 6. Numerical Examples. For evaluating the performance of our model, we show numerical examples by using C0 testingcoverage measure data recorded along with fault count data collected from an actual testing of an embedded software system. There are totally 296 faults detected and 90.6% of the C0 testingcoverage measure attained within 24 weeks. Figure 1 shows the estimated alternative testing-coverage function Cb(t) of Eq.(4) in which. 5.2. Instantaneous MTBF. We discuss the instantaneous MTBF which has been used as one of the substitution for MTBF. An instantaneous MTBF is approximately given by (21) MTBFI (t) = h(1t) : 941.
(22) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. 1. 3 2.7 Mean Time Between Software Failures. 0.9. Software Reliability. 0.8 0.7 0.6 0.5 0.4. 2.4 2.1 1.8 1.5 1.2 0.9 0.6 0.3. 0.3. 0 0. 1. 2. 3. 4. 5. 6. 0. 2. 4. Operation Time (number of weeks). 6. 8. 10. 12. 14. 16. 18. 20. 22. 24. Testing Time (number of weeks). Fig. 3： The estimated software reliability.. Fig. 4： The estimated instantaneous MTBF.. the parameter estimates are obtained as b = 90:796; bd b = 52:338. From Figsta = 0:388, and z ure 1, we can see that the estimated alternative testing-coverage function
(23) ts well to the actual C0 testing-coverage measure data. Next, Figure d 2 shows the estimated mean value function H C (t) of Eq.(11) and its 95% con
(24) dence limits, where d the parameter estimates of H C (t) are obtained as ba = 919:9 and sb = 0:4282. The 100 % con
(25) dence d limits for H C (t) are derived as. faults and the attained testing-coverage has been discussed. And we have also discussed the parameter estimation methods of our models. In particular, we have obtained the parameter estimates of the alternative testing-coverage function by using the regression analysis based on the integrable di erence equation derived from the original di erential equation. Then, we derive several software reliability assessment measures such as the software reliability function, the instantaneous and cumulative MTBF's. Finally, we have shown numerical examples of the estimated alternative testing-coverage function and the estimated SRGM by using C0 testing-coverage measure data recorded along with fault count data collected in an actual testing phase. Especially, for embedded software systems, the proposed alternative testing-coverage function and the SRGM enable us to see the maturity process of the testing-coverage, and simultaneously estimate the number of faults detected in the testing phase. Further studies are needed that we have to discuss more about the useful software reliability assessment measures and the performance of the proposed model by using several actual data measured on the other testing-coverage criteria.. q. d d H C (t); C (t) K H. (23). where K indicates the 100(1+ )=2 percent point of the standard normal distribution. And then, we conduct the Kolmogorov-Smirnov (abbreviated as K-S) goodness-of-
(26) t test [2, 3] to evald uate whether H C (t)
(27) ts statistically to the observed data. This statistical testing is considered to be ecient even if the sample size of data set d is small [2]. We can verify that H C (t)
(28) ts to the observed data with the 5% level of signi
(29) cance from the result of the K-S goodness-of-
(30) t testing. Furthermore, Figure 3 depicts the estimated software reliability function at the termination of the testing in Eq.(20). If we assume that the developed software system is used in the operation phase which has the same environment as the testing phase, we can estimate the software reliability after 3 weeks from the termination time of the testing, R(3 24), to be about 0 554. And Figure 4 shows the estimated instantaneous MTBF. We can estimate the instantaneous MTBF at the termination time of the testing, MTBFI (24), to be about 3 024 (weeks). . \. Acknowledgments. This research was partially supported by the Grant-in-Aid for Scienti
(31) c Research (C)(2) from the Ministry of Education, Culture, Sports, Science, and Technology of Japan under Grant No. 15510129. And the
(32) rst author was
(33) nancially supported by the Sasakawa Scienti
(34) c Research Grant from the Japan Science Society, Grant No. 16-064.. j. . 7. Concluding Remarks. References. We have discussed software reliability growth modeling based on the testing-coverage which is one of the key factors related to the software reliability growth process in this paper. Speci
(35) cally, the relationship between the number of detected. [1] J.D. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application, McGraw-Hill, New York, 1987.. 942.
(36) Int. Computer Symposium, Dec. 15-17, 2004, Taipei, Taiwan.. [2] S. Yamada, Software Reliability Models: Fundamentals and Applications (in Japanese), JUSE Press, Tokyo, 1994. [3] H. Pham, Software Reliability, Springer-Verlag, Singapore, 2000. [4] S. Yamada, \Software Reliability Models," in Stochastic Models in Reliability and Maintenance, (S. Osaki, ed.), Springer-Verlag, Berlin, 2002, pp. 253{280. [5] K. Kanoun, M. Kaaniche, and J.C. Laprie, \Qualitative and quantitative reliability assessment," IEEE Software, Vo. 14, No. 2, pp. 77{87, 1987. [6] P.K. Kapur and S. Younes, \Modelling an imperfect debugging phenomenon in software reliability," Microelectronics and Reliability, Vol. 36, No. 5, pp. 645{650, 1996. [7] P. Zeephongsekul, \Reliability growth of a software model under imperfect debugging and generation of errors," Microelectronics and Reliability, Vol. 36, No. 10, pp. 1475{1482, 1996. [8] S. Yamada, H. Ohtera, and M. Ohba, \Software reliability growth model with testing-domain and comparisons of goodness-of-
(37) t," Computers & Mathematics with Applications, Vol. 24, No. 1/2, pp. 79{86, 1992. [9] T. Fujiwara and S. Yamada, \C0 coveragemeasure and testing-domain metrics based on a software reliability growth model," International Journal of Reliability, Quality and Safety Engineering, Vol. 9, No. 4, pp. 329{340, 2002. [10] Y.K. Malaiya, M.N. Li, J.M. Bieman, and R. Karcich, \Software reliability growth with test coverage," IEEE Transactions on Reliability, Vol. 51, No. 4, pp. 420{426, 2002 [11] H. Pham and X. Zhang, \NHPP software reliability and cost models with testing coverage," European Journal of Operational Research, Vol. 145, No. 2, pp. 443{454, 2003. [12] S. Yamada, M. Kimura, and M. Takahashi, Statistical Quality Control for TQM, Corona, Tokyo, 1998. [13] A.L. Goel and K. Okumoto, \Time-dependent error-detection rate model for software reliability and other performance measures," IEEE Transactions on Reliability, Vol. R-28, No. 3, pp. 206{211, 1979. [14] R. Hirota, \Nonlinear partial di erence equations. V. Nonlinear equations reducible to linear equations," Journal of Physical Society of Japan, Vol. 46, No. 1, pp. 312{319, 1979. [15] S. Yamada, S. Inoue, and D. Satoh, \Statistical data analysis modeling based on di erence equations for software reliability assessment," (in. Japanese), Transactions of the Japan Society for Industrial and Applied Mathematics, Vol. 12, No. 2, pp. 155{168, 2002.. 943.
(38)
數據
相關文件
Promote project learning, mathematical modeling, and problem-based learning to strengthen the ability to integrate and apply knowledge and skills, and make. calculated
stating clearly the important learning concepts to strengthen the coverage of knowledge, so as to build a solid knowledge base for students; reorganising and
Wang, Solving pseudomonotone variational inequalities and pseudocon- vex optimization problems using the projection neural network, IEEE Transactions on Neural Networks 17
Define instead the imaginary.. potential, magnetic field, lattice…) Dirac-BdG Hamiltonian:. with small, and matrix
These are quite light states with masses in the 10 GeV to 20 GeV range and they have very small Yukawa couplings (implying that higgs to higgs pair chain decays are probable)..
Overview of a variety of business software, graphics and multimedia software, and home/personal/educational software Web applications and application software for
Microphone and 600 ohm line conduits shall be mechanically and electrically connected to receptacle boxes and electrically grounded to the audio system ground point.. Lines in
The Insurer agrees to provide coverage in respect of any Claim made against the principal appointed on acting basis and approved by the Education Bureau of the Government of the
