2. Serial–Order in the Box and C-SOB
2.1 Serial-Order in a Box model
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
2 . Serial–Order in the Box and C-SOB
In this thesis, I assume that serial recall recognition task share the same encoding process and underlying memory representation. In order to verify this assumption, two selected serial order model, Serial-Order in a Box (SOB) and its successor C-SOB, are extended to account recognition task.
SOB and C-SOB model are connectionist models for short-term serial task.
As mentioned in previous chapter, SOB and C-SOB share plenty of features and mechanisms, and the major difference between two models is the way that position information is encoded. SOB assumes that the memory of item is stored in an auto-association network. The position information is the result of primacy gradient and response suppression. Early studied items are encoded with larger encoding strength and result in higher chance to be recalled early in the list (Farrell & Lewandowsky, 2002). The successor, C-SOB model, retains the primacy gradient, response sup-pression, and the auto-association network. A hetro-association connecting between items and contexts represents the memory of position information. Retrieval is con-ducted by using the previous context as retrieval cue. Since that the early context associate with early items, early learned items are easier to be retrieved in the be-gin of recall(Farrell, 2006; Lewandowsky & Farrell, 2008). The detail of SOB and C-SOB model are instructed in this section.
2.1 Serial-Order in a Box model
SOB models the memory trace with an auto-association network. The serial position is coded through the encoding strength. In this auto-association network (W ), the items (v) associates with themselves and are learned with Hebbin rule. One of the most important feature in auto-association network is that the noisy item feeds into network will results in a less noisy item as output. The accuracy and efficiency of deblurring process is influenced by the encoding strength. Deblurring process is an important part in retrieval, see below.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Items are distributed representation and consist with 1 and −1 in both SOB.
In SOB model, each items are orthogonal to each other in order to avoid crosstalk in deblurring process. The items are randomly selected from Walsh matrix with 254 dimensionality vectors.
In SOB, prior knowledge in the memory trace is carried through pre-train 50 random selected items. pre-training uses Hebbian learning rule as
Wk = Wk−1+ ηpvkvkT (2.1) with very small learning rate (ηp is set to .001). Each chosen items are pre-trained 20 times.
2.1.1 Encoding process
During encoding phrase, items serial present before participants one by one till list running out. SOB model also encodes one item a time till end of list. The same as prelearning, encoding process in SOB is achieved by Hebbian rule as
Wi = Wi−1+ ηeviviT. (2.2) Unlike prelearning, the encoding strength ηe is not fixed. ηe is determined by energy which represents the inconsistency between memory trace and the incoming item.
Energy(E) is calculated through
E =−(ϵ/2)∑i∑
jwijxixj, i̸= j. (2.3) The wij is the element in the row i and column j of W, and xi and xj represents the element i, j in the vector of incoming item (vi). The energy calculation is the comparison between the auto-association network W and the outer product of vi with ignoring the diagonal units. The larger E (care the negative in Equation 2.3) indicates the larger inconsistency between memory trace and incoming item. While more items are encoded into memory trace, the energy increases.
The encoding strength of current learning item is modulated by the energy through
‧
The more inconsistency between item and memory trace results in the lesser en-coding strength of item. For the first item, the inconsistency only comes from the prior in memory trace, thus the encoding strength should be fairly large. The fol-lowing items receive more inconsistency from the prior and previous encoded items.
The encoding strength of later encountered items are smaller than previous. The relationship between energy and encoding strength is illustrated in Figure ??.
1 2 3 4 5 6
Figure 2.1: The energy and encoding strength curve among different serial position in SOB.
2.1.2 Retrieval process
To retrieve information from memory trace, SOB applies the deblurring process which gradually reduces the noise. The beginning of each item recalling is to ran-domly generate a vector which consists of +1 and −1. The length of the vector is normalized1 to .0001, and the normalized vector is considered as the seed of deblur-ring process (v′) and used as the starting point of the deblurring process x(0). x then updates with pre-trained auto-associative network W with
x(t + 1) = G[γx(t) + αW x(t)]. (2.5) In the deblurring process, x(t) is fed into W . The input and output of the network are combined with weighted by γ and α. Function G is a non-linear transformation
1The vector is transferred to having a unit length.
‧
function, which limits the elements xi in mathbf x within the range between−1 and +1 and is described as
G(x) =
The iteration stops when any additional iteration does not change the elements in content or when the iteration times reaches the maximum Imax.
Because the initial seed is randomly generated, the most well learned item will be recalled in highest chance. The first item is generally encoded with largest strength and is the most likely to be retrieved. The retrieved item is then suppressed in memory trace with anti-learning. Anti-learning is the same as the learning while encoding but with a negative learning rate ηs(j) which varies based on the energy ratio between the energy for the first retrieved item and the energy for the current retrieved ones. The learning rate is calculated by
ηs(j) = −Ej
ϕsE1 (2.7)
where ϕs is a scalar to adjust the ratio between the first retrieved energy and the following energy. The anti-learning rate of the first retrieved item should be 1/ϕs.
Response suppress plays a important role in SOB, since the first item is the one which most likely to be retrieved. The following items have limited chance to be retrieved in correct sequence without response suppression. However, with response suppression mechanism, the second item becomes the highest activated item after the suppression of first item and could be retrieved in correct order.