By observing the object allocation process of size-ordered segregated list allocator, we know that it must traverses up the free list headers if the free list corresponding to the requested size contains no chunk and a sufficient large chunk is found until the closest
non-empty free list is visited. Since many free lists become empty as the program execution, why not to remember allocation request of each size can be satisfied at which free list?
We can add a table to direct allocation request of each size to the closest non-empty free list holding sufficient large size. The table is essentially a mapping from requested size to the right free list, thus each size has a corresponding table entry. We called the table Next Hit Table (NHT) since by looking up the table, the right free list is located without traversing up.
In the following subsections, we suppose that heap chunks are multiple of 4 bytes because most Java virtual machines use 4-byte as the unit of heap spaces.
Figure 3-4: Next Hit Table
3.2.1 Object Allocation with Next Hit Table
Object allocation with NHT support is done as follow. Upon allocation request of a size S object, the allocator first lookup the table to know that a sufficient large chunk can be found
in free list F, holding 4F-byte objects. If F < N, the allocation request is satisfied by taking the first free chunk in free list F. Otherwise, it searches a best-fitting chunk in the last free list (free list N) to satisfy the allocation request.
3.2.2 Maintaining Next Hit Table
NHT will be initialized to N, which indicates the last free list because in the beginning of execution, the heap is itself a single large free chunk. Note that because there is only a free chunk in the heap until the first invocation of garbage collection, the initially bad allocation performance before the first invocation of garbage collection becomes very quick, a table lookup plus direct allocating the only chunk without searching.
Figure 3-5: Initialization state of NHT
For maintaining the right mapping, NHT must be updated when a free list becomes empty and when a free list becomes non-empty. A free list will become empty if it runs out of chunk in list after object allocation request, and there are two cases that a free list will become non-empty. The first case is when the remainder of an allocated chunk is linked back to a free list and the free list is previous empty. The second case occurs during garbage collection - when a free chunk is swept and then linked back to a free list and the free list has been empty before garbage collection.
When a free list F becomes empty after object allocation, the update process is as follow.
Suppose the NHT entries corresponding to free lists F-1, F -2, …, F - i are of value F, and the
NHT entry corresponding to F - i - 1 is not of value F. Then the NHT entries corresponding to free lists F, F-1, F -2, …, F - i are updated to the value of NHT entry corresponding to free list F + 1. This is to reflect the fact that afterwards allocation requests of size 4F, 4F-4, …, 4F - 4i
can no longer be satisfied at free list F. Figure 3-6 depicts the update operation when the free list holding 20-byte chunk becomes empty.
Figure 3-6: NHT update example 1
When a free list F becomes non-empty, the update process is as follow. Suppose the NHT entry corresponding to free list F -1 is not of value F - 1, the NHT entry corresponding to free list F - 2 is not of value F - 2, …, and the NHT entry corresponding to free list F - i is not of value F - i, but the NHT entry corresponding to free list F - i - 1 is of value F - i – 1, then the NHT entries corresponding to free lists F, F-1, F -2, …, F – i are updated to F to reflect that next allocation request of size 4F, 4F-4, …, 4F - 4i can be satisfied at free list F. Figure 3-7 depicts the update operation when the free list holding 16-byte chunk becomes non-empty due to the remainder being added back.
Figure 3-7: NHT update example 2
If the update occurs during garbage collection, we use special handling. Because there is no object allocation request during garbage collection, we can postpone the update to the end of garbage collection process. By such treatment, the table only needs to be updated once during garbage collection.
3.2.3 Overhead
We have described how our proposed mechanism, NHT, works to reduce the traversals upon object allocation in size-ordered segregated list allocator. We discuss the overhead using NHT now.
NHT mechanism incurs both space and speed overhead. The space overhead comes from the data structure implementing NHT, which can be simply an array of integer. And because typically size-ordered segregated list allocator uses no more than hundreds of free lists which mean hundreds of 4 bytes, the overhead only counts up several KB.
The speed overhead might be more critical than space overhead, since if the speedup gained by fast object allocation can not amortize the time cost to maintain NHT, our ambition will simply fail. Remember that the update only occurs in three cases, and these occur infrequently. A free list might become empty only when it runs out of free chunks, and free chunks are reclaimed only during garbage collection, thus this case rarely occurs. However, the speed overhead of NHT when using immediate coalescing strategy might be higher, because it is more potential to have the remainder after allocating a larger chunk and the remainder will be linked back into a free list. Our simulation result shows that update does not occur frequently – only one update occurs averagely during 1000 allocation request using deferred coalescing strategy (Table 3-1).
Table 3-1: NHT update frequency using deferred coalescing strategy
_201_compress 145 116 261 6479 25
_202_jess 254 225 479 110471 231
_205_raytrace 118 93 211 575724 2729
_209_db 138 113 251 123166 491
_213_javac 152 122 274 222891 813
_227_mtrt 111 81 192 677785 3530
average 153 125 278 286086 1029