Analysis of Incorporating Logistic Testing-Effort
Function Into Software Reliability Modeling
Chin-Yu Huang and Sy-Yen Kuo, Fellow, IEEE
Abstract—This paper investigates a SRGM (software
reli-ability growth model) based on the NHPP (nonhomogeneous Poisson process) which incorporates a logistic testing-effort function. SRGM proposed in the literature consider the amount of testing-effort spent on software testing which can be depicted as an exponential curve, a Rayleigh curve, or a Weibull curve. However, it might not be appropriate to represent the consumption curve for testing-effort by one of those curves in some software development environments. Therefore, this paper shows that a logistic testing-effort function can be expressed as a software-de-velopment/test-effort curve and that it gives a good predictive capability based on real failure-data. Parameters are estimated, and experiments performed on actual test/debug data sets. Results from applications to a real data set are analyzed and compared with other existing models to show that the proposed model predicts better. In addition, an optimal software release policy for this model, based on cost-reliability criteria, is proposed.
Index Terms—Mean value function, nonhomogeneous Poisson
process, optimal software release policy, software reliability growth model, testing-effort function.
ACRONYMS1
AE accuracy of estimation
BMMRE balanced mean magnitude of RE HGDM hyper-geometric distribution model
HPP homogeneous Poisson process
LOC lines of source code
LSE least squares estimation
MLE maximum likelihood estimation
MRE magnitude of RE
MSF mean of square fitting faults
MVF mean value function
NHPP non-HPP
PE prediction error
RE relative error
RMS root mean square
- implies: statistical(ly)
SD software development
SRE software reliability engineering SRGM software reliability growth model TEF testing-effort function
Manuscript received December 1, 1997; revised September 10, 2000. This work was supported by the National Science Council, TAIWAN, R.O.C., under Grant NSC 86-2213-E259-002. Responsible Editor: M. A. Vouk.
The authors are with the Electrical Engineering Department, National Taiwan University, Taipei, Taiwan (e-mail: [email protected]; [email protected]).
Publisher Item Identifier 10.1109/TR.2002.801847.
1The singular and plural of an acronym are always spelled the same.
NOTATION
mean number of faults detected in time , an MVF
: failure intensity for
current testing-effort consumption at time cumulative
mean number of initial faults
fault detection rate per unit testing-effort total testing-effort eventually consumed
consumption rate of testing-effort expenditures in the logistic TEF
constant parameter in the logistic TEF scale parameter in the Weibull-type TEF shape parameter in the Weibull-type TEF conditional software reliability
likelihood function
cumulative testing-effort actually consumed in cumulative number of faults observed in
software life-cycle length
cost of correcting an error during testing cost of correcting an error during operation, cost of testing per unit testing-effort expenditures
I. INTRODUCTION
S
OFTWARE reliability is the probability that a given soft-ware functions correctly under a given environment during a specified period of time. It is a key software-quality factor. Software reliability represents a customer-oriented view of soft-ware quality. It relates to practical operation rather than simply the design of a program. Therefore, it is dynamic rather than static. The aim and objective of software (reliability) engineers are to increase the probability that a designed program works as intended by the customers. Hence, measuring and computing the reliability of software systems are very important. They can be used for planning and controlling all testing resources during development, and can assure us about the correctness of soft-ware. A common approach for measuring software reliability is by using an analytic model whose parameters are generally es-timated from available data on software failures. However, re-search activities in SRE have been conducted over the past 2 decades extensively, and many SRGM have been proposed [1]. SRGM are successful for estimating software reliability and the number of faults remaining in the software systems. They can be used to evaluate SD status and SRE technology quantitatively [2], [3].The testing phase is an important and expensive part of SD. Many research studies assume that the consumption rate of testing resources during the testing phase is constant, or do not even consider such testing-effort. References [2]–[4] show that the effort index (execution time) is a better exposure indicator for software reliability modeling than calendar time because the shape of the observed reliability growth curve depends strongly on the time distribution of the testing-effort. References [2], [3], [5], [6] propose a SRGM which describes the relationship among the calendar testing, the amount of testing-effort, and the number of software faults detected by testing. The testing-effort can be represented as the number of CPU hours, the number of executed test cases, etc. Sometimes the testing time can be represented by the number of tests instead of the execution time [4]. In the area of software relia-bility modeling, SD effort was often described by the traditional exponential, Rayleigh, or Weibull curves [5]–[7]. However, in many software testing environments it is difficult to describe the testing-effort function by these 3 effort consumption curves only. In this paper, we show that a logistic TEF can be expressed as a SD/test-effort curve. Experiments have been performed based on real test/debug data sets. Comparisons of predictive capabilities between various models are presented. The results show that the SRGM with a logistic TEF can estimate the number of initial faults better than the previous approaches.
Section II briefly describes existing TEF in the literature, and discusses the proposed logistic TEF. Section III investigates the SRGM with logistic TEF. The parameters of an SRGM are esti-mated with logistic TEF by using the LSE and MLE. Section IV applies this model to actual software failure data, and shows the numerical results. Section V is concerned with the applications of this model to an optimum release policy based on the cost-re-liability criterion.
II. TESTING-EFFORTFUNCTIONS
This section briefly reviews some TEF which were developed to estimate the SD effort. Most of them are parametric because they predict development effort using a formula of fixed form that is parameterized from historical data records. During the software testing phase, many testing-efforts, such as the man power, the number of executed test cases, and the CPU time, are consumed.
A. A Brief Review of TEF
1) COCOMO2 Effort Algorithms: COCOMO is one of the
best known SD models and was developed by Boehm based on a regression analysis of 63 completed projects. COMOCO re-lates the effort to Delivered Source Instructions [8]. COCOMO provides a combination of various functional forms made acces-sible to the user in a structured manner. The COCOMO effort algorithms all have the basic form:
(1) the effort
the size, typically measured as LOC the productivity parameter
the scale parameter 2COnstructive COst MOdel.
2) Analogies: Reference [9] proposes an alternative
ap-proach to estimation, based on the use of analogies. The underlying principle is to characterize projects in terms of features. However, estimation of software-project effort by analogy has an advantage in that it is very intuitive.
3) Machine Learning Approaches: Reference [10] proposes
machine learning approaches to estimating SD effort using an algorithm for building regression trees, and neural-network learning approach known as back-propagation. The advantage of learning approaches is that they are adaptable and nonpara-metric.
4) Norden/Rayleigh Model: References [11], [12] observe
that the Rayleigh distribution provides a good approximation of the manpower curve for various hardware development pro-cesses. Then, the Rayleigh distribution is used as an approxima-tion to the smoothed labor distribuapproxima-tion curve and is applied to several software projects
(2) the total consumed manpower
the time for manpower to peak
5) Chatterjee TEF: Reference [13] proposes a TEF to
de-scribe the resource consumption during SD. It considered that the test-effort and learning-factor depend on each other; i.e., it assumes that test-effort and learning-factor are inversely pro-portional to each other, and that they have a joint effect on SD. Therefore, the Chatterjee TEF can be written as:
(3) the proportionality constant
the learning factor; an increasing function of
6) Pillai and Nair Gamma Model: Reference [14] proposes
a gamma model for SD-effort and cost-estimation. This model is based on the Gamma distribution and can be represented as
(4) the time for SD effort to peak
the total SD effort expended for the project
7) Thoma Test-Instance Functions: HGDM was first
pro-posed by Thoma and has been developed to estimate the number of remaining software faults after the test/debug phase [15], [16]. The collection of test operations performed in a day or a week is called a test instance. Therefore, various functions were proposed to describe the test instance, such as:
tester (5)
tester (6)
tester (7)
tester the number of workers involved in
8) Hou and Kuo Learning-Factor Functions: References
TABLE I
SUMMARY OFREALDATASETSSTUDIED
and on the S-shaped learning curve, to enhance the HGDM and make it more realistic.
Exponential learning curve:
(8) Logistic learning curve:
(9)
the limit value of learning factor
9) Yamada Weibull-Type TEF: According to [2], [3],
[5], [6], [11], testing-effort should not be assumed constant throughout the testing phase. Instantaneous testing-effort ultimately decreases during the testing life-cycle because the cumulative testing-effort approaches a finite limit. This assumption is reasonable because no software company will spend infinite resources on software testing. Hence, [5], [6] show that the testing-effort can be described by a Weibull-type distribution and have the following 3 cases.
1) Exponential curve: The cumulative testing-effort
con-sumed in is
(10) 2) Rayleigh curve: The cumulative testing-effort consumed
is
(11) 3) Weibull curve: The cumulative testing-effort consumed is (12)
For the Weibull-type curves (12), when or
, the result is the exponential or Rayleigh curve respectively; therefore, they are special cases of the Weibull TEF. For the Weibull-type curves, if 3, 4, or 5, these testing-effort curves have an apparent peak phenomenon (nonsmoothly increasing and degrading consumption curve) during the SD process; i.e., a peak work-rate occurs. This phenomenon seems not so realistic because it is not commonly used to interpret the actual SD/test process [20]. Hence, the Weibull function might not be suitable
for modeling the testing-effort consumption curve, although it can be made to fit or approximate many distributions and repre-sents flexible testing-effort by controlling the shape parameter.
B. Logistic TEF
Because actual testing-effort data represent various expendi-ture patterns, sometimes the testing-effort expendiexpendi-tures are dif-ficult to describe only by an exponential or a Rayleigh curve. Although a Weibull-type curve can fit the data well under the general SD environment and is widely used in software reli-ability modeling, it has the apparent peak phenomenon when [20]–[22]. An alternative is the logistic TEF, first pre-sented in [23]. This function was fairly accurate as reported by the Yourdon 1978–1980 project survey [24]. The cumulative testing-effort consumption in time is
(13) the current testing-effort consumption is
(14)
Therefore, is a smooth bell-shaped function, and reaches its maximum value at
(15) In contrast with the Weibull-type TEF in the initial point,
the logistic TEF . The discrepancies between the
Weibull-type curve and the exist in the earlier stages of SD where progress is often least visible, and where formal accounting procedures for recording the amount of applied testing-effort might not have been instituted. It is possible to judge between these models using some statistical test of their relative ability to fit actual failure data, such as adjusting the origin and scales linearly [23].
C. Comparisons Between Different TEF
To check the performance of the logistic TEF and to compare fairly with other TEF, especially the Rayleigh distribution, this paper applies 4 real data-sets to these proposed models. These
TABLE II
COMPARISONRESULTS FORDIFFERENTTEF BASED ONDS1
TABLE III
COMPARISONRESULTS FORDIFFERENTTEF BASED ONDS2
TABLE IV
COMPARISONRESULTS FORDIFFERENTTEF BASED ONDS3
data sets are in Table I, and the comparison criteria for evalua-tion are described here [9], [10], [14]:
1) Prediction Error PE
Actual Observed Predicted Estimated
2) Variation
PE Bias
3) Bias PE
4) RMS-PE Bias Variation
5) MRE
6) BMMRE
The comparisons between the proposed logistic TEF and the Weibull-type TEF are illustrated in Tables II–V and Figs. 1–4. Figs. 1–4 illustrate the comparisons between the observed failure data and the data estimated by various TEF. The computed RMS-PE, the bias, the Variation, the MRE, and the BMMRE based on different data sets are shown in Tables II–V. Figs. 1–4 and Tables II–V show that: 1) the logistic TEF yields a better fit for the data sets chosen, and 2) the structure of logistic TEF is very flexible for various testing environments.
TABLE V
COMPARISONRESULTS FORDIFFERENTTEF BASED ONDS4
Fig. 1. Observed/estimated TEF for DS1.
Fig. 2. Observed/estimated TEF for DS2.
Fig. 3. Observed/estimated TEF for DS3.
III. SOFTWARERELIABILITYGROWTHMODEL A basic SRGM is based on the assumptions: 1) The fault removal process follows the NHPP.
2) The software system is subject to failures at random times caused by faults remaining in the system.
3) The mean number of faults detected in by is proportional to the mean number of faults remaining in the system.
Fig. 4. Observed/estimated TEF for DS4.
4) The proportionality constant does not change with respect to time.
5) The testing-effort consumption with exposure is modeled by a logistic function.
6) Each time a failure occurs, the fault that caused it is im-mediately and perfectly removed without new faults being in-troduced.
7) Correction of faults takes negligible time, and a detected error is removed with certainty.
Because the -expected current detected fault content is finite at any time, is an increasing function of ; . From these assumptions:
(16) Consequently, if the number of detected faults due to the cur-rent testing-effort expenditures is proportional to the number of remaining faults, then
(17) Solving (17) under the boundary condition gives:
(18)
Therefore,
(19) (20) Thus, the mean number of undetected faults, if a test is ap-plied for an infinite amount of time, is
if (21)
Hence, not all the original faults in a software system can be fully detected with a finite testing-effort because the effort to be eventually used during the testing phase is limited to .
IV. EVALUATION OFSRGM WITHLOGISTICTEFAND
PERFORMANCEANALYSIS
A. Data Description
This section evaluates the performance of SRGM with lo-gistic TEF. The real data set is the System T1 data of the Rome Air Development Center (RADC) projects in [2], and the failure data are generally of the best quality. System T1 is used for a real-time command and control application. The size of soft-ware is about 21 700 object instructions. It took 21 weeks and 9 programmers to complete the test. During the test phase, about 25.3 CPU hours were consumed and 136 software faults were removed.
B. Criteria for Model Comparison
The 4 performance-comparison criteria are given here. 1) AE is defined as [26]:
(22) is the actual cumulative number of detected faults after the test, and is the estimated number of initial faults. For practical purposes, is obtained from software fault tracking after soft-ware testing.
2) RE is defined as [2]:
(23) If failures are observed in test-time , use the failure data up to
time to estimate the parameters of : . The
estimate is compared with the actual number . The procedure is repeated for various . The predictive validity is checked by plotting RE versus .
3) Noise is defined as [27]:
(24) predicted failure rate.
Small values represent less noise in the model’s prediction behavior, indicating more smoothness. A noise measure of indicates that the model has predicted a zero failure rate [27].
4) MSF (for long-term prediction) is defined as [27] (25) MSF is used for quantitative comparisons for long-term predic-tions, because it provides a better-understood measure of the differences between actual and predicted values. A smaller MSF indicates a smaller fitting error and better performance [27].
C. Performance Analysis
This section evaluates the proposed model and several ex-isting NHPP models. First, parameters of all selected models are estimated and the related mean value functions are obtained. Second, all the selected models are compared with each other based on objective criteria.
, , in logistic TEF are estimated using LSE. Using the estimated TEF, the other parameters , in (18) can be solved
numerically by MLE. Therefore, , .
Table VI compares the performance of various SRGM for the data set investigated in this paper. Due to the limitations of paper
TABLE VI
COMPARISONRESULTSBETWEENDIFFERENTSRGM
Fig. 5. m(t) of the proposed model and the 90% s-confidence bounds.
Fig. 6. m(t) of the Delayed S-Shaped model and the 90% s-confidence bounds.
size, only 3 pre-eminent models are used for detailed discus-sions:
• proposed model in this paper, • Delayed S-Shaped Model, • Yamada Rayleigh-type model.
These have better performance as shown in Table VI. Figs. 5–7 show the actual (observed), fitted software failures, and the 90%
s-confidence bounds [3] respectively. A good SRGM should
be able to predict well the behavior of future failures. In gen-eral this can be evaluated by considering the estimated proba-bility distribution of the next failure-time. Thus Fig. 8 shows a -plot analysis of predictions from the selected models on the Musa system T1 data set. The -plot of the proposed model is close to the line of unit slope; and the proposed model has a smaller Kolmogorov-Distance, which is defined as the max-imum vertical derivation between the plot and the line of unit slope, when comparing with other existing SRGM, as shown in
Table VI. Table VII shows the values of and for 3
selected models [28]. Estimates for system T1 between selected models are compared in Table VIII. The RE in prediction is
cal-Fig. 7. m(t) of the Yamada Rayleigh-type model and the 90% s-confidence bounds.
Fig. 8. u-plots for predictions of T , i = 36; . . . ; 136 for Musa system T1 data-set.
TABLE VII
Var[a]ANDVar[r]FORSELECTEDMODELS
TABLE VIII
Fig. 9. RE curve for the Delayed S-shaped model, based on Musa’s system T1 data set.
Fig. 10. RE curve for the Yamada Rayleigh-type model based on Musa’s system T1 data set.
Fig. 11. RE curve for the proposed model, based on Musa’s system T1 data set.
culated for this data set at the end of testing. The results are in Figs. 9–11. From these simulation/comparison results, the pro-posed model performs appreciably better than the others. This model fits the observed data better, and predicts the future be-havior well for this data set.
V. OPTIMALSOFTWARE-RELEASEPOLICY
A. Software Release-Time Based on Reliability Criteria
In general, the software-release time problem is associated with the reliability of a software system. The release policy based on the reliability criterion is discussed first. If the relia-bility of a software system is known to have reached an accept-able level, then we the right time to release this software can be determined. References [29]–[31] first discussed the release
problem by considering the software cost-benefit. The condi-tional reliability function after the last failure occurs at time is:
(26)
Differentiate with respect to , then .
Hence is a monotonic increasing function of . Take the logarithm of (26):
(27) Solving (27) and (18) determines the testing time needed to
reach a desired . is increasing in . Using
(27), one can get the required testing-time needed to reach the reliability objective or decide whether is reached or not in a specified time interval.
B. Optimal Release-Time Based on Cost-Reliability Criterion
This section discusses the cost model and release policy based on the cost-reliability criterion. Using the total software cost evaluated by the cost criterion, the cost of testing-effort expen-ditures during the software testing/development phase and the cost of correcting errors before and after release are [5], [6], [20]:
(28) From [8], because is usually an order of magnitude greater than . Differentiate (28) with respect to and set it to 0:
(29)
(30)
Case 1) ; then , and
Case 2) If ; then , and
Therefore, is monotonically decreasing in . If
Hence, for this case, the optimal software release time . If
then there exists a finite and unique solution, satisfying
Rearranging this equation, gives:
minimizes
(31) Because
for for
the minimum of is at for . [Because
, then is a convex function.]
Section V-A provides the information to get the required testing time needed to reach . The goal is to minimize the total software cost to achieve the desired software reliability, and then the optimal software release time is obtained. That is,
mathematically minimize subject to ,
.
optimal software release time or total testing time , where is finite and the unique solution of
(31), is finite and unique satisfying ,
.
This analysis is summarized in the relationships:
Given: , , , , : 1) If and , then for , or for . 2) If then for , or for . 3) If then for , or for .
To illustrate items 1)–3), use again the first real data-set in Section II-C for a numerical example on the optimal software release problem in Section V-C.
C. Numerical Example
From the previously estimated parameters:
Also assume , , , ,
, .
Then is estimated as 20.98, based on minimizing of (31), and is estimated as 12.79, based on satisfying the
reliability criterion of .
Fig. 12. Total software cost and reliability estimation for the System T1 data-set.
Because
(32)
the is estimated as .
(the plots are shown in Fig. 12). APPENDIX
To validate the proposed SRGM with a logistic TEF in (18), experiments on 3 real test/debug data sets were performed. Two popular estimation techniques are MLE and LSE [1]–[3]. The MLE estimates parameters by solving a set of simultaneous equations and is better in deriving -confidence intervals. But the equation sets are very complex and usually must be solved numerically. The LSE minimizes the sum of squares of the deviations between what is actually observed/gotten and what is anticipated. LSE is generally considered to be the best for medium sample sizes and provides the best point estimates [20]. LSE is used here to fit the logistic curve with the real data set. For the method of “least square sum,” the evaluation
formula S1 is:
Minimize S1 (A-1)
cumulative testing-effort actually consumed in time
, cumulative testing-effort estimated by the
Differentiate S1 with respect to , , and , set the partial derivatives to zero, and rearrange these terms, to solve this type of nonlinear least-squares problems
S1
(A-2) The LSE, , is found by solving (A-1) and (A-2):
(A-3)
Then, S1
(A-4)
(A-5) The other parameters , can be obtained by substituting the LSE into (A-4) and (A-5).
The likelihood function for and in the NHPP model with in (18), is:
(A-6) for
Take the logarithm of the likelihood function in (A-6),
From (18):
Thus,
(A-7)
Consequently, the MLE, and are obtained by solving
(A-8)
(A-9) The and are solved by numerical methods.
If of is sufficiently large, then the MLE and
asymptotically follow a BVN (bivariate s-normal distribution) [29], [32]:
BVN
The mean values of and are the true values of and , re-spectively, and the variance–covariance matrix is given by the inverse matrix of the Fisher information matrix [5], [6], [26].
The Fisher information matrix for and can be derived
from as
(A-10)
Apply and to (A-10) and calculate . The estimated asymptotic variance–covariance matrix is
The is useful in quantifying the variability of the estimated parameters.
ACKNOWLEDGMENT
The authors are pleased to thank Professor M. A. Vouk for his constructive and insightful suggestions for improving the details of this manuscript.
REFERENCES
[1] M. R. Lyu, Handbook of Software Reliability Engineering: McGraw Hill, 1996.
[2] J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability,
Measure-ment, Prediction and Application: McGraw Hill, 1987.
[3] J. D. Musa, Software Reliability Engineering: More Reliable Software,
Faster Development and Testing: McGraw-Hill, 1999.
[4] M. Ohba, “Software reliability analysis models,” IBM J. Research &
[5] S. Yamada, H. Ohtera, and H. Narihisa, “Software reliability growth models with testing-effort,” IEEE Trans. Reliability, vol. R-35, no. 1, pp. 19–23, Apr. 1986.
[6] S. Yamada, J. Hishitani, and S. Osaki, “Software reliability growth model with Weibull testing effort: A model and application,” IEEE
Trans. Reliability, vol. 42, pp. 100–105, 1993.
[7] P. K. Kapur and S. Younes, “Modeling an imperfect debugging phenom-enon with testing effort,” in Proc. 5th Int. Symp. Software Reliability
En-gineering (ISSRE’1994), pp. 178–183.
[8] B. W. Boehm, Software Engineering Economics: Prentice Hall, 1981. [9] M. Shepperd and C. Schofield, “Estimating software project effort using
analogies,” IEEE Trans. Software Engineering, vol. 23, pp. 736–743, 1997.
[10] K. Srinivasan and D. Fisher, “Machine learning approaches to estimating software development effort,” IEEE Trans. Software Engineering, vol. 21, no. 2, pp. 126–136, 1995.
[11] L. H. Putnam, “A general empirical solution to the macro software sizing and estimating problem,” IEEE Trans. Software Engineering, vol. 4, pp. 345–367, 1978.
[12] J. Tian, P. Lu, and J. Palma, “Test-execution-based reliability measure-ment and modeling for large commercial software,” IEEE Trans.
Soft-ware Engineering, vol. 21, no. 5, pp. 405–414, May 1995.
[13] R. Chatterjee, B. Misra, and S. S. Alam, “Joint effect of test effort and learning factor on software reliability and optimal release policy,”
Inter’l. J. Systems Science, vol. 28, no. 4, pp. 391–396, 1997.
[14] K. Pillai and V. S. S. Nair, “A model for software development effort and cost estimation,” IEEE Trans. Software Engineering, vol. 23, no. 8, Aug. 1997.
[15] Y. Tohma, K. Tokunaga, S. Nagase, and Y. Murata, “Structural approach to the estimation of the number of residual software faults based on the hyper-geometric distribution,” IEEE Trans. Software Engineering, vol. 15, no. 3, pp. 345–355, Mar. 1989.
[16] Y. Tohma, R. Jacoby, Y. Murata, and M. Yamamoto, “Hyper-geometric distribution model to estimate the number of residual software faults,” in Proc. COMPSAC-1989, pp. 610–617.
[17] R. H. Hou, S. Y. Kuo, and Y. P. Chang, “Applying various learning curves to hyper-geometric distribution software reliability growth model,” in
Proc. 5th Int. Symp. Software Reliability Engineering (ISSRE’1994), pp.
8–17.
[18] , “Optimal release policy for hyper-geometric distribution software reliability growth model,” IEEE Trans. Reliability, vol. 45, no. 4, pp. 646–651, Dec. 1996.
[19] T. Rivers, “Modeling software reliability during nonoperational testing,” Ph.D. dissertation, Department Computer Science, NC State University, Raleigh, 1998.
[20] C. Y. Huang, S. Y. Kuo, and I. Y. Chen, “Analysis of a software reli-ability growth model with logistic testing-effort function,” in Proc. 8th
Int. Symp. Software Reliability Engineering (ISSRE’1997), pp. 378–388.
[21] C. Y. Huang, J. H. Lo, S. Y. Kuo, and M. R. Lyu, “Software relia-bility modeling and cost estimation incorporating testing-effort and efficiency,” in Proc. 10th Int. Symp. Software Reliability Engineering
(ISSRE’1999), pp. 62–72.
[22] C. Y. Huang, S. Y. Kuo, and M. R. Lyu, “Effort-index-based software reliability growth models and performance assessment,” in Proc. 24th
Ann. Int. Computer Software and Applications Conf. (COMPSAC2000),
pp. 454–459.
[23] F. N. Parr, “An alternative to the Rayleigh curve for software devel-opment effort,” IEEE Trans. Software Engineering, vol. SE-6, pp. 291–296, 1980.
[24] T. DeMarco, Controlling Software Projects: Management, Measurement
and Estimation: Prentice-Hall, 1982.
[25] P. N. Misra, “Software reliability analysis,” IBM Systems J., vol. 22, no. 3, pp. 262–279, 1983.
[26] S. Yamada, J. Hishitani, and S. Osaki, “Software reliability growth mod-eling: Models and applications,” IEEE Trans. Software Engineering, vol. SE-11, no. 12, pp. 1431–1437, 1985.
[27] M. R. Lyu and A. Nikora, “Applying software reliability models more effectively,” IEEE Software, pp. 43–52, Jul., 1992.
[28] Y. Liang and K. S. Trivedi, “Confidence interval estimation of NHPP-based software reliability models,” in Proc. 10th Int. Symp. Software
Reliability Engineering (ISSRE’1999), pp. 6–11.
[29] K. Okumoto and A. L. Goel, “Optimum release time for software sys-tems based on reliability and cost criteria,” J. System Software, vol. 1, pp. 315–318, 1980.
[30] M. Xie, “On the determination of optimum software release time,” in
Proc. 2nd Int. Symp. on Software Reliability Engineering, 1991, pp.
218–224.
[31] P. K. Kapur and R. B. Garg, “Cost-reliability optimum release policies for a software system with testing effort,” OPSEARCH, vol. 27, no. 2, pp. 109–116, 1990.
[32] W. Nelson, Applied Life Data Analysis: Wiley, 1982.
Chin-Yu Huang was awarded an honorary degree (1989) in electronic
engi-neering from Hocking Technical College, Ohio. From 1990 to 1992, he studied in the Department of Transportation Engineering and Management at National Chiao Tung University. He received the M.S. (1994) and the Ph.D. (2000) in electrical engineering from National Taiwan University. He was with the Bank of Taiwan from 1994 to 1999; and was a senior software engineer at Taiwan Semiconductor Manufacturing Company from 1999–2000. He is now a senior specialist at the Central Bank of China, Taipei. His research interests are soft-ware reliability, softsoft-ware testing, and fault-tolerant computing.
Sy-Yen Kuo received the B.S. (1979) in electrical engineering from National
Taiwan University, the M.S. (1982) in electrical and computer engineering from the University of California at Santa Barbara, and the Ph.D. (1987) in com-puter science from the University of Illinois at Urbana-Champaign. Since 1991 he has been with National Taiwan University, where is currently Professor and Chairman in the Department of Electrical Engineering. He was Chairman of the Department of Computer Science and Information Engineering, National Dong Hwa University, Taiwan from 1995 to 1998, a faculty member in the Department of Electrical and Computer Engineering at the University of Arizona from 1988 to 1991, and an engineer at Fairchild Semiconductor and Silvar-Lisco, both in California, from 1982 to 1984. In 1989, he also worked as a summer faculty fellow at Jet Propulsion Laboratory of California Institute of Technology. His current research interests include dependable distributed systems and networks, software reliability engineering, and optical WDM networks. He received the distinguished research awards (1997–2001) from the National Science Council, Taiwan. He also received the Best Paper Award in the 1996 International Sym-posium on Software Reliability Engineering, the Best Paper Award in the simu-lation and test category at the 1986 IEEE/ACM Design Automation Conference (DAC), the National Science Foundation’s Research Initiation Award in 1989, and the IEEE/ACM Design Automation Scholarship in 1990 and 1991.
