Appendix B Evolution at the High Level: Beliefs

At the low level of evolution, the investor revises and renews his investment strategies with respect to a specific belief selected from a population of beliefs {Bⁱ_j,t}^J_j=1. In other words, at each point in time, the investor may have more than one model of uncertainty in the world. The idea that each agent can simultaneously have several different models of the world, which are competing with each other in a co-evolving process, is a distin-guishing feature of the population learning models (Holland and Miller, 1991; Arthur et. al., 1997; Vriend, 2000; Chen and Yeh, 2001; Arifovic and Maschek, 2003). Of course, these

Population of saving

Figure 6: The Flowchart of the Investment Optimization

models are not equally promising, and the investor tends to base his decision (investment strategies) on one of the most promising ones. However, as times goes on, his beliefs of the world will be revised and renewed in light of the newly incoming information. In this section, we shall describe how genetic algorithms can be applied to modeling the beliefs updating process.

Appendix B.1 Coding and Initialization

In our agent-based comsumption CAPM, each investor’s perception of the uncertainty (finite-state stochastic process) of the market can be characterized by two elements: first, the dependence structure (k), and, second, the sample size (d). Based on this characterization, the investor believes that the market over the last d days follows a kth-order Markov process. According to this belief, he would use a part of the historical data{mt−s}^v_s=v+1^+d+1, referred as to the training period, to estimate the Markov transition matrix, and the rest of the data{mt−s}^v_s=1, referred to as the validation period, to validate the estimated model.

Validation

Choose the best from the new generation.

Choose the best from the new generation.

Figure 7: The Belief Updating Scheme

As a result, each belief can be represented by a binary string, of lengthτ1+ τ2, a1a2...a_τ1

that has the following interpretation: the states follow a Markov process of the order k= (

∑

days. To facilitate estimation, d cannot be too small, and that demands an additional constant of c. In our current model, we simplify and limit the dependent structure (k) to 0 or 1, that is, we only assume the stochastic process to be iid or first-order Markov.

At the initial date (t = 0), all investors are endowed with a population of J beliefs, which are randomly generated. Then every Δ days, this population of belief will be reviewed and revised based on the fitness function, which is a kind of likelihood function to be specified below.

Appendix B.2 Belief Updating Scheme

Agents in our model follow the practice of machine learning. They are supposed to care about the risk of over-fitting, and hence use data in the validation period to perform model selection. One way of ensuring that our agents behave so is to set the fitness function as the fitting error in the validation set, rather than the training set. The belief updating scheme is outlined in Figure 7.

The essence of the belief updating scheme is to maintain a style of on-line learning, while not to overload the computational intensity. As we can see from this figure, at each time t agents retain the most recent v days as the validation period. They use the

data before the validation period, that is, the data of the training period, to estimate the parameters of each belief. Then a fitness measure for a belief Bⁱ_j,tis its associated likelihood, evaluated by the validation set{mt−s}^v_s=1,

Lⁱ_j,t= L({m_t−s}^v_s=1| Bⁱ_j,t). (42) Equation (42) is the likelihood of the observations {mt−s}^v_s=1 in the validation period under the belief Bⁱ_j,t. Every Δ periods, after they finish the evaluation of each belief’s fitness, they apply the genetic operation to update their belief set (see Section Appendix B.3), and the belief with the highest fitness will be chosen. Even in the period that the genetic operation is not applied, say when t ∈ [Δ + 1, 2Δ − 1], they evaluate the fitness of beliefs in their current belief set using the newest data and choose the best from it.

Appendix B.3 Genetic Operation

Once the procedure of evaluating each belief’s fitness (Eval{Bⁱ_j,t−1}^J_j=1) is completed, all beliefs are associated with a fitness which is the output of (42).

Eval:{Bⁱ_j,t−1}_j=1^J → {Lⁱ_j,t−1}_j=1^J (43)

Based on this fitness evaluation, we will revise and renew investor i’s beliefs by using the following four genetic operators: selection, crossover, mutation and election.

Selection: A tournament selection with tournament size 4 is adopted. The best two beliefs will be chosen as the parents (mating pool).

Crossover: With probability pcross, the two parents chosen above will generate an off-spring by the uniform crossover. With this crossover, each bit position of the offoff-spring will be taken randomly either from the father or the mother with a one-half chance for each. For an illustration, let us consider the pair of parents to be B_x,tⁱ ₋₁ = 0010101010, corresponding to a belief of(kx, dx) = (0, 170), and Bⁱ_y,t−1 = 0111110010, corresponding to(ky, dy) = (0, 498). Then, an offspring, Bⁱ_z, can be

Bⁱ_z = 0011100010 → (kz, dz) = (0, 226).

Mutation: There is a small probability pmutate(mutation rate) by which each bit of Bⁱ_z may encounter a change. For example, the mutation which changes the fifth bit from “1”

to “0”, and the last bit from “0” to “1” will result in a new string:

Bⁱ_z = 0011000011 → (kz , dz ) = (0, 195).

Election: Finally, Bⁱ_z will also be evaluated by the observations {mt−s}^v_s=1, and the likelihood will be figured out. We will then compare the likelihood from Bⁱ_z with the likelihood from the parent models, and the best one will be passed to the next generation, {Bⁱ_j,t}_j^J₌₁.

Agent i

i=1

Fitness Evaluation Loop

j=1

selection

crossover

election j=J? Yes

Yes

No No

j=j+1

i = I?

Output

i = i + 1

mutation

agent i chooses his best belief Input agent i's length of

validation

{ }

Figure 8: Flowchart of the High-Level GA

Appendix B.4 Loops

Once a belief is generated, a loop in Figure 8 will lead us back to selection, which is then followed by crossover, mutation and election before the next belief is generated. The loop will continue until all J beliefs of{Bⁱ_j,t}^J_j=1are generated. One of the beliefs, B^i,∗_j,t, will be chosen based on the likelihood criteria,

B^i,∗_t = arg max

j L({m_t−s}^v_s=1| Bⁱ_j,t), (44)

The belief set will remain unchanged for the nextΔ periods, when another loop of revi-sion and renewal process is conducted, and B^i,_t+Δ^∗ is brought about.