Case-by-case Discussions

We will see how NBPT works case-by-case in this section. We go through the NBPT design by following cases and consider each pattern will be encountered more than once:

I. Forward branch

II. Loop (backward branch) III. Method invocation and return IV. Multiple transition targets

Case of Forward Branch

Here we consider if instructions in Java. They are: ifeq, ifne, iflt, ifge, ifgt, ifle, if_icmpeq, if_icmpne, if_icmplt, if_icmpge, if_icmpgt, if_icmple, if_acmpeq, if_acmpne, goto, ifnull and ifnonnull. Their destination can be forward or backward. In case of backward, it usually forms a loop and we will discuss in next subsection. Now we consider the forward case.

Figure 3.5 NBPT: Case of forward branch

See Figure 3.5 (b), there is an if instruction in block Q. If the branch is taken, the program counter will jump to block Ry and restart execution from Ry. Otherwise, it continues to execute the instructions after the if and then transfer to Rx. If the Ry case appears more frequently than Rx, the state goes toward the right hand site along the solid line in Figure 3.5 (a). Note that we predict Ry for the right part and Rx for the left part. If the Rx case is more common, the entry will be invalidated or just stays in the left part of Figure 3.5 (a).

Case of Loop (Backward Branch)

A loop is formed by a backward if. Consider a loop pattern as Figure 3.6 (b). The program counter transfers from Q to Ry every iteration so that the state goes toward and stays in state NS-HC. However, it will leave the loop and transfer to Rx eventually. Then the state will become NS-LC. If the program counter enters the loop again, NBPT still predict Ry at the first iteration and then the state is set back to NS-HC at the second iteration.

Figure 3.6 NBPT: Case of loop

Case of Method Invocation and Return

Method invocations do not be handled well by NLPT as we have mentioned in Subsection 2.3.1. Thus, a key point of NBPT is to prevent an entry of method invocation from being invalidated immediately. Figure 3.7 (b) depicts the program flow of a method invocation and its return. There is an invoke instruction in block Q. When it encounters the instruction, the JVM determines the method location and restart execution from the first block R1 of the target method. After finished the duty of the method, it will return eventually. Thus the program counter transfers back to the subsequent instruction after the

invoke site when the method returns. Then it will move to block R3 after Q completed. Here we consider the case of R3 is sequential to Q. For case that (Q, R3) is non-sequential, this is a situation of multiple targets. The state machine will choose the frequent one between R1 and R3 in this situation as we will describe later.

Figure 3.7 NBPT: Case of method invocation

Now refer to Figure 3.8:

(a) Initially, (Q, R1) and (R2, Q) are both not in the NBPT. After entered Q, because there doesn’t exist any corresponding entry of Q, the prefetch unit predicts R3 sequentially.

(b) After encountered the invoke instruction in block Q, the program counter transfers to R1. At the same time, an entry of (Q, R1) is inserted into the NBPT, where the state is S-LC initially and the I-bit is set to 1.

(c) After the work of the invoked method finished, the program counter returns back to Q from R2. (R2, Q) is put into the NBPT where the state is S-LC and the R-bit is set to 1.

(d) After the program counter left from Q and moved into R3, the corresponding entry of Q should be updated. Note the condition of the arc from S-LC to S-HC is

“sequential and I-bit=0”. Because the I-bit of (Q, R1) is 1, it will go along another arc toward the NS-LC state.

(e) Consider the program counter entered Q again. Nothing has been changed yet if they were not replaced out. Now the prefetch unit predicts R1 for the next block because (Q, R1) is in the NBPT and in state NS-LC.

(f) The program counter entered R1 because of the method invocation. The state of (Q, R1) became NS-HC from HS-LC since its next-block matched.

(g) Afterwards the program counter returned back to Q from R2, (R2, Q) became NS-HC from S-LC since its next-block matched.

(h) The program counter left from Q and entered R3, (Q, R1) became NS-LC because its next-block did not match. Note it will become NS-HC when the invocation occurs again. After several iterations, the NBPT will be like (g) or (h) finally. The state of (Q, R1) moves forth and back between NS-HC and NS-LC.

Figure 3.8 NBPT: A trace of method invocation and return

Besides, we may issue an additional prefetch during method invocation and return.

This is because during invocation and return, it traps to software JVM to do some duties.

Then there would be a period of time for us to prefetch one block extra. See Figure 3.9 (b), block A which has a invoke or return instruction is in method X(). The program counter will transfer to block B which is in another method Y() after met the instruction and then enter C.

Before entering B, it will trap to the software JVM to fix up frames, determine where B is, and do some checks. Compare to traditional programs, these works have been done before the call/return instruction, so a conventional processor is able to jump to the target address directly. Thus, we may prefetch an additional block C during this period of time. For simply, we can just speculate that block C is sequential to block B. See Figure 3.9 (a) as an example, we prefetch block B1 and B2 when entering A2 from A1. At the entry of B3, we prefetch block A2 as well as A3. Note that the software JVM may also produce misses, however, we can still obtain some advantages.

Figure 3.9 Additional prefetch during method invocation and return

Case of Multiple Transition Targets

Finally, we consider the case that the program counter may transfer from one block to multiple target blocks. This is probably caused by a virtual method invocation, an indirect

branch, or there are multiple branches in a block. Take Figure 3.10 (b) for example, if we have 2 possible targets Rx and Ry, we would like to choose the most frequent one since our NBPT only records one target. The selection mechanism is designed in the left part which is circled in Figure 3.10 (a). See Figure 3.11, if (Q, Rx) appears frequently and (Q, Ry) appears occasionally, NBPT will tend to select Rx. However, if (Q, Ry) continually occurs twice, the next-block of the NBPT entry will be replaced by Ry. At this moment, Ry is considered as the most frequent block.

Figure 3.10 NBPT: Case of multiple transition targets

Figure 3.11 NBPT: An example of selection between 2 candidates