Ordering for Efficient HDL Debugging
4.4 An Efficient Probabilistic Confidence Score Calculation Algorithm
The proposed PCS computation algorithm is a topology-based analysis with time frame expansion to handle the sequential behavior of the DUV. While calculating the LOEP of the output variable of vertex v in time frame t=ti, the algorithm will consider each sensitized path from v in time frame t=ti that goes to any PO in each time frame before EOE. This path-oriented computation scheme is defined in (4.9), which can be derived from (4.5).
Set of vertex v at time instance t=ti with respect to POj at t=ck (denoted as MVS(v@t=ti)POj@t=ck). An element of the set other than the correct value can be regarded as an incorrect value that is masked by some vertices on the path(s) from v at t=ti to POj at t=ck, thus keeping the correct value of POj at t=ck. According to (4.9), if it is possible to derive MVS(v@t=ti)POj@t=ck for each POj at each time frame t=ck, then intersecting these sets yields MVS(v@t=ti). After deriving MVS(v@t=ti), PCS of vertex v can be obtained according to formula (4.7) and (4.8).
We may observe that the definition and the derivation of Probabilistic Confidence Score are based on LOEP and Masked Value Set, which were introduced in section 3.2 before. Thus, the computation algorithm for MVS’s and LOEPs can be applied to PCS computation with some modifications. If there is exactly one path from v at t=ti to a PO POj at t=ck, the induction-based computation approach introduced in section 3.3.1 and section 3.3.2 can be applied to compute exact MVS(v@t=ti)POj@t=ck. If there are multiple paths from v at t=ti to POj at t=ck, the quick estimation approach introduced in section 3.3.3 that guarantees lower-bound LOEP estimations will be applied. In addition, to avoid unnecessary back-tracing, the bounded traversal strategy introduced in section 3.3.4.1 is also applied. The entire PCS computation algorithm is represented as the pseudo code shown in Figure 4-6, which incorporates each part mentioned above.
The input of this algorithm are 1) the Design Under Validation (DUV) described in a HDL, 2) the value change dumpfile during simulation, 3) the Error-occurring Edge (EOE), and 4) an error space obtained by any error space identification approach.
Figure 4-6: Pseudo-code of PCS Computation Algorithm
During traversal(s) that starts from a PO (StartPO) at a time instance (StartTime), if vertex v is visited for the first time, a single path case is temporarily assumed. The PCS_computation (DUV, Dumpfile, EOE, Error space)
1: 3-address Code Generation and Conditional statement odification 2: CDFG Construction 3: Initialize each vertex as “untraversed”
4: for each positive edge of clock t=ck before EOE 5: for each primary output POj
6: InitialMVS = {CV(POj@t=ck)}; Find the fanin vertex a1 of POj at t=ck
7: MVS_Com_for_Vertex (InitialMVS, a1, POj, ck, ck) 8: Calculate PCS with the computed MVSs
MVS_Com_for_Vertex(PreviousMVS, vertex v, StartPO, StartTime, time tj) //*** Modification for incorporating MVS computation for multiple paths ***
1: if traversed for first time in traversal starting from StartPO at StartTime 2: if MVS(v@t=tj) = = ∅
9: else //Multiple paths. Recovering to the previous status before intersection 10: MVS(v@t=tj) = MVSforRecovery(v@t=tj)
//*** Modification for incorporating MVS computation for multiple paths ***
11: if v is a control vertex
12: Mark the fanin vertex(es) on the untaken branch(es) as “inactive”
13: for each active fanin vertex u of v 19: MVS_Com_for_Vertex(CurrentMVS, u, StartPO, StartTime, th)
PreviousMVS will be intersected with MVS(v@t=ti), which is already the intersection of many PreviousMVSs. However, if this vertex v is found traversed for two or more times in the traversal starting from StartPO at StartTime, there are multiple paths from v at t=ti to StartPO at StartTime. Then, the previously recorded MVSforRecovery(v@t=ti) is used to chancel intersections made in this traversal before.
During the MVS computation process, if the condition at line 5 of the MVS_Com_for_vertex subroutine is met, according to Theorem 2 introduced in section 3.4, additional traversal and MVS computation cannot affect the already computed MVS. Thus, an immediate return from subroutine MVS_Com_for_vertex at line 6 is made and precious computation time is saved.
The preparation phases of this algorithm are shown at lines 1 and 2. The 3-address code generations and the conditional statement modification introduced in section 3.3.3 must be conducted first for the information required in MVS computation for control vertices (conditional statements). The detailed conditional statement modification algorithm can be found in [19]. Next, a CDFG based on the input DUV described in a HDL is constructed.
The example in Figure 4-3 is used to demonstrate the processes of our PCS computation and its performance in the derivation of a debugging priority. After some initializations, the CDFG of the DUV based on the HDL code in Figure4-3 is constructed as shown in Figure 4-7(a). Then, a backward traversal from PO1 at t=1 commences by calling subroutine MVS_Com_for_vertex with the inputs PreviousMVS={4}, vertex v=”2:if”, StartPO=PO1, and StartTime=1.
Figure 4-7: Computation processes starting from PO1 at t=1
When subroutine MVS_Com_for_vertex is called for the first time, the traversal reaches vertex “2:if” in time frame t=1 also for the first time. As shown in Figure 4-7(b), the recorded MVS(2:if@t=1)={4} and no MVSforRecovery is recorded.
Vertex “2:if” in time frame t=1 is a control vertex. Therefore, there are two fanin vertices “2:<” and “3:=” for further backward traversals. We decide to traverse “2:<”
before traversing to “3:=” and compute CurrentMVS. Due to the fact that PreviousMVS is {4}, the MVS computation for conditional statements will be used and we obtain CurrentMVS {1}. The computation process is shown in Figure 4-7(b).
Subroutine MVS_Com_for_Vertex is then called for the second time and “2:<” in time frame t=1 is reached. The computation status is shown in Figure 4-7(c). Similar computations is repeated by recursively vertex by vertex vertex “6:=” in time frame t=1 is reached. The computation results along the traversal from “2:if” to “6:=” are
shown in Figure 4-7(d), where each set of integers next to an edge is the recorded MVS. Since vertex “6:=” in time frame t=1 has no fanin vertex, the computation will traverse another fanin vertex “3:=” of vertex “2:if.” The repetitious calling of subroutine MVS_Com_for_Vertex can produce the results shown in Figure 4-7(e).
After completing the traversals and MVS computations starting from PO1 in time frame t=1, continue the backward-traversal based MVS computation from PO1 in time frame t=5 can produce the results shown in Figure 4-8(a) and (b). When the computation reaches vertex “5:if” in time frame t=1, PreviousMVS {[0~15]} will include MVS(5:if@t=1)={[0~3]}. The bounding traversal condition is satisfied and the traversal is bounded here.
Figure 4-8: Computation starting from PO1 at t=5 and t=15
After all the POs at all the positive clock edges before the EOE are applied in MVS computation, PCS is calculated with formulas based on the computed MVSs.
With PCS, a debugging priority with PCS (in round brackets) is obtained, as shown in Figure 4-9. It can be seen that the design error S7 is displayed in the first line. A search for design errors according to this debugging priority will succeed immediately.
In the experimental results in the next section, it is also proven that the proposed PCS can efficiently deliver debugging priority with high accuracy, which greatly reduces both the time and efforts required for design error searches in the input error space.
Figure 4-9: Debugging priority and the PCS
4.5 Experimental Results
The experiments are conducted on a subset of ITC’99 benchmark [1] and four other designs written in Verilog HDL. Number of lines (#line) of all the designs and the number of variables (#var) are presented in Table 4-1.
For every design case, one statement is randomly chosen and injected with an artificial design error based on typical bugs that designers usually induce [47]. With the injected error, a simulation is run until some incorrect values occur on POs. Then,
the error space identification approach proposed in [46]5 is applied to obtain an error space. After that, CS calculation in [46] and our proposed PCS calculation are both applied to derive two respective debugging priorities for the same error space.
With a debugging priority, error candidates that a digital circuit designer has to examine before he/she can find a design error are often less than blindly searching. In a sense, we can think that the size of the error space is thus reduced as a result of a debugging priority. With respect to the two debugging priorities, since the injected error may have two different ranked orders, the effectiveness of the two debugging priorities on the size reduction of the same error space are also different. To compare the effectiveness of the two debugging priorities, a quantitative metric called Effective Size Ratio (ESR) is formulated as “the rank of the injected error/the number of error candidates in the error space”. The two debugging priorities sorted with CS and with PCS have their own ESRs, respectively. Smaller ESR means the error has better rank with respect to the size of the error space. That also implies that the effective size reduction contributed by the corresponding debugging priority is larger and the effort required for design error searching in the error space is less.
5 We apply the error space identification method proposed in [22]. However, the proposed PCS is theoretically applicable to any other error candidate identification methods.
With each design case, the above experimental processes are repeated for fifty times. In each repetition, the ESR of CS and PCS are calculated and recorded. The average ESR values with respect to CS and PCS are also presented in the columns
“Avg_ESR_CS” and “Avg_ESR_PCS”, respectively. The number of times, in which the ESR values of CS and PCS appear in three ESR value ranges, are also recorded to show the distribution of ESR values. The three ESR ranges are “ESR<0.2 (0~0.2)”,
“0.2<ESR<0.5 (0.2~0.5)”, and “0.5<ESR (0.5~1.0)”. The number of times is presented in the column “#case_CS” and “#case_PCS”, respectively.
In Table 4-1, it can be observed that when PCS is used to obtain a debugging Table 4-1 A Comparison of Confidence Score (CS) and Probabilistic Confidence Score (PCS)
Performances
Confidence Score (CS) Probabilistic Confidence Score (PCS)
priority, in all the design cases, the average ESR values are all less than 0.2 and are also less than the average ESR values of CS. For example, in design “B02”, if CS is used to derive debugging priority, 38 times out of 50 times the ESR value is less than 0.2. In other words, our created errors are placed in the first twenty percent in the displayed list of error space for 38 times out of 50 experimental times. But, if PCS is applied instead, in each of the 50 repetitions, the injected error always appears in the first twenty percent. If a designer conduct error searching on design “B02” with the debugging priority sorted with PCS, he/she will locate the error by checking less than twenty percent of the derived error candidates. At least eighty percent of searching effort is saved. Moreover, it can be seen that ESR values of PCS is never greater than 0.5 in the fifty repetitions. This means that even in the worst case of the fifty repetitions, a debugging priority sorted with PCS can still save more than half the amount of efforts needed for design error searching in the error space. By contrast, the CS method was not found to offer this benefit.
From the values of Avg_ESR_PCS and Avg_ESR_CS, it can be observed that the effective size reduction with respect to PCS is much greater than the one with respect to CS. The ratio Avg_ESR_PCS to Avg_ESR_CS shown in the column “ESR Ratio” is about 0.49 on average and 0.38 in the best case, meaning that with PCS, a 50% further size reduction, on average, is possible and a 62% size reduction is also achieved in the best case, as compared to CS. Therefore, on average, the proposed PCS method should save much more time/effort needed in the error searching process in the error space than CS. The cost of this improvement is little computation time, as compared to CS. The computation time needed for the two measurements, PCS and
CS, are presented in the columns “t(s)”. It can be seen that in the worst case, it takes 2 extra seconds to obtain PCS as compared to the time required to obtain CS (4.1s).
This extra cost of computation time is acceptable if we notice that it should usually takes more than 2 seconds for a designer to examine one error candidate to justify its correctness, but the number of examinations saved as a result of applying PCS is numerous.
4.6 Summary
This chapter presents a probabilistic measurement, PCS, to derive an accurate and reliable debugging priority for quick error searching among error candidates in an error space. Instead of assuming that the erroneous effects caused by some activated errors are seldom masked, the proposed PCS takes error masking into consideration and estimates the Likelihood Of Error-Propagating (LOEP) of an error candidate. The idea is that if the LOEP is high, error masking is unlikely to occur and the error candidate is a false candidate with high possibility, i.e. the candidate tends to be a correct statement. On the other hand, if the LOEP is low, occurrence of error masking becomes quite possible. The suspicion of the error candidate still remains and this candidate should thus receive a low PCS score.
The experimental results confirm that the proposed PCS measurement is indeed accurate in estimating the likelihood of correctness for error candidates. In most experimental cases, the created design errors can be located in the first few lines of the candidate list of the input error space. As a result, debugging priority sorted with the proposed PCS can effectively speed up error searching process in the input error
space. As compared to CS, the proposed PCS-based debugging priority can save more than half of the efforts (or time) needed for error searching process in an error space in our experiments, at the cost of little extra computation time. The time saving contributed by the proposed PCS method should usually much larger than the extra computation time the PCS calculation needs. Therefore, the gain of the proposed PCS can often outweigh the cost of extra computation time the PCS needs.