Study Outline

A review of related studies is presented in the following chapter. In addition to listing popular artificial societies for problem solving and describing them in terms of the public good/private interest conflict, I will discuss the development of the PD model and common approaches to PD investigations. In Chapter 3, I will present a simple case study outlining the importance of performing model analysis before running a simulation or solving a problem. In Chapter 4, a description of the analytical framework for analyzing PD strategy relationships will be given. An application of the framework to memory-1 strategies will be described in Chapter 5—including evidence that the framework is useful in identifying important PD strategies and simulation phenomena. Chapter 6 contains a discussion of framework extension and generalization to models whose strategies are encoded in other forms, in spatial PD models, and in biased selection PD models. Also in Chapter 6, I will discuss framework methodology and its application to other kinds of artificial society analyses. A discussion of the effects of strategic PD environments is presented in Chapter 7, and conclusions are given in Chapter 8.

Chapter 2.

Related Work

2.1 Artificial Society

2.1.1 Primary Components of Artificial Societies

According to Actikis and Pitt [13], all artificial societies include a) a set of agents, b) a set of societal constraints, c) a language for communication, d) a set of agent roles, e) a set of states of affairs that hold at each time at the society, and f) a set of agents (sometimes referred to as owners). Tim Doran [14, 15] defined an abstract model of an artificial society as a network (or graph) with nodes called sites and links called channels. At any single moment, a network is filled with many agents and items located at individual sites.

From the two definitions, the three primitive components of an artificial society are:

a) Agents (individuals). Considered basic units of an artificial society, agents have

their own characteristics and express their own behaviors. Depending on the level of sophistication of an artificial society, it can be categorized as homogeneous or heterogeneous. Homogeneous agents have similar or equivalent characteristics—for example, chromosomes in the simplest form of a genetic algorithm (GA). Even though a set of chromosomes may have different gene combinations and fitness values, they have the same format and express the same behaviors (e.g., they are evaluated by the same function and interact with other chromosomes via such shared mechanisms as crossover operators).

Agents are said heterogeneous if they have completely different characteristics, including behaviors, goals, or encodings. Well-known examples of heterogeneous artificial societal agents are agents in multi-agent systems. They are purposefully designed with completely different goals and behaviors, cooperating and interacting in order to achieve system goals.

b) Environment. Artificial societies must provide environments in which agents can

instance, the set of all cells in a cellular automata [16, 17]. The actions of a cell are determined by its current status as well as the statuses of its neighboring cells—no other factor is involved, unlike the situation for more complex artificial societies. An example of a more complex scenario is a classifier system, which consists of four basic parts: input interface, classifiers, message lists, and output interface [18]. Messages are read from an input interface and added to a message list that all classifiers use to determine which messages satisfy their conditions. Results are sent to the output interface. In this example, messages are considered the environment that affects all classifiers.

There are two categories of environments: agent viewpoint and global environment.

For most artificial societies, these two are distinctly different. Since information can decay or be lost through propagation, and since each agent has a limited perception of its environment, any environment is usually considered incomplete from an agent’s viewpoint.

It is important that users know if the environmental information they are working with is global or from an individual agent’s viewpoint.

c) Interaction. The most important component that determines the complexity and

characteristics of an artificial society is interactions between the basic entities of agent and environment. In their absence, artificial societies simply constitute a set of static data.

Interactions in artificial societies can be categorized as between agents or between agents and environments. The first category is more common. An example is the crossover operator in GAs, in which two chromosomes interact via gene exchange. In multi-agent systems, agents can negotiate with, take information from, or give information to other agents. Interactions between agents and environments are more common in more sophisticated artificial societies—for instance, ant colony systems [19-21]. Ants secrete pheromones into their environments, and their actions are determined according to the pheromones they encounter. Their interactions are said to be limited to their environment instead of with other ants.

2.1.2 Popular Artificial Societies for Problem Solving

A list of artificial societies commonly used for problem solving is presented in Table 2.1, categorized according to their primary component characteristics.

GAs are considered simple artificial societies that contain chromosomes as homogeneous agents; no other environmental factor is involved. Classifier systems are more complex than GAs in that they accept external input and generate output, both of which can be viewed as environmental factors whose statuses affect the system.

type of artificial society in that there is no interaction between agents, only between agents and their environment. Neural networks are considered a particular type in that interactions between agents (nodes) are fixed; interaction between two nodes is impossible in the absence of a connecting link.

Table 2.1: Artificial Societies Commonly Used for Problem Solving

Agent Environment Interaction

Genetic Algorithm [22] Homogeneous N/A A-A¹ Ant Colony System [19-21] Homogenous Simple A-E²

Classifier System [18] Homogenous Complex A-A, A-E

Neural Network[23] Homogenous Complex A-A, A-E, but fixed

Artificial Immune System

[24-28] Heterogeneous Complex A-A, A-E

Multi-Agent System [29, 30] Heterogeneous Complex A-A, A-E

1 interactions between agents.

2 interactions between agents and environments.

2.1.3 The Public Good/Private Interests Conflict in Artificial Societies

A common real-world conflict is that between the public good and private interests.

However, it remains to be seen whether this conflict also exists in artificial societies that are not inhabited by humans or other living beings—especially artificial societies whose primary goal is to solve problems, instead of simulating real-world phenomena. As mentioned in Chapter 1, in problem-solving artificial societies the society concept is used to solve problems via either individual achievement or societal achievement of a system-level goal. Regardless of the approach, the public good/private interest conflict makes an appearance sooner or later. A list of common artificial societies used for problem solving is presented in Table 2.2, along with a summary of the goals and issues raised by this conflict.

From Table 2.2, it is clear that even in the artificial societies for problem solving, the conflict between private good and public interest is also a crucial issue. I believe that the investigation of this conflict will be helpful for efficient use of those models.

Table 2.2: List of Artificial Societies, their Goal Levels, and Conflict between Individual Agent and Societal Goal

The Level of Goal (System/Individual)

Conflict between Agent’s and Societal Goals

Genetic Algorithm Individual Premature Convergence, [11, 12]

Ant Colony System System N/A

Classifier System System Credit Assignment [31, 32], Premature Convergence Artificial Immune System Individual/System Selection Mechanism [24]

Neural Network System N/A

Multi-Agent System Individual/System

Mechanism Design [33, 34], Action Selection [35], Resource Management [36], Robots Conflict Detection [37]

2.2 The Prisoner’s Dilemma (PD)

2.2.1 PD and IPD (Iterated Prisoner’s Dilemma)

The Prisoner’s Dilemma (PD) is a precise mathematical model that is frequently used by economists [38], sociologists [39, 40], political scientists [41], biologists [42], and psychologists [43] to analyze conflicts of interest [44]. It is considered a powerful tool for

explaining human and animal society organization and for promoting cooperation among agents in virtual environments [45]. Its simplicity is considered one of its outstanding qualities, yet it remains a surprisingly complex problem to analyze.

In a classic version of a PD game, two players decide whether each move they make should be one of cooperation or defection. A payoff is given to each player according to their combined moves. Table 2.3 shows a typical payoff matrix, including value constraints.

Table 2.3: Prisoner’s Dilemma Payoff Matrix and Constraints

Player B

Player A Cooperation Defection

Cooperation

Reward

Reward

Sucker’s

Temptation

Defection

Temptation

Sucker’s

Penalty

Penalty

Note: Temptation > Reward > Penalty > Sucker’s, and 2×(Reward) > Temptation + Sucker’s, or simplified:

If both players know that they will play a PD game one time only, it is to their benefit to continually make defection moves in order to achieve a maximum outcome. If they know that they will play many games (a situation referred to as the Iterated Prisoner’s Dilemma, or IPD), mutual cooperation is a better strategy for both players. Since most real-world dilemmas are iterated, the IPD is of great interest to researchers [46, 47].

2.2.2 Research Topics on IPD

IPD research falls into two broad categories:

a) Model Behavior Investigations. Despite its mathematical simplicity, IPD model behavior is surprisingly complex. Originally, IPD researchers were interested in understanding the behavioral characteristics of this model (e.g., dynamics of population [48, 49], and persistence of cooperation [50]), and of such common strategies as Tit-for-Tat (repeat what your opponent does in the previous round) and PAVLOV (a win-stay-lose-shift strategy) in various scenarios [46, 51, 52]. Other researchers reported on the strengths and weaknesses of strategies encoded in various formats, including memory-n strategies [53, 54], finite automata strategies [55], rule-based strategies [56], and other formats [57-59]. Evolutionarily Stable Strategies (ESS) have also been the focus of numerous studies in terms of properties and their existence in various situations [60-64].

b) Model Sophistication. More recently, researchers have focused on IPD model sophistication in an attempt to apply it to a wider range of problems that are more representative of the real world. Several have introduced spatial IPD models in which agents are positioned in two-dimensional spaces; these agents are limited to interacting with neighboring strategies [65-70] Others have reported that cooperative behavior is more likely in PD models where opponent selection is biased [45, 71-73]. In PD games with more than two players (known as N-player PDs), emergent cooperative behavior differs from that observed in two-person games [74, 75]. In N-choice PD, players may not be limited to choosing between cooperation or defection [71, 76, 77]. An extension of N-choice PD is the continuous PD model, in which player actions are viewed as an infinite set of continuous values instead of a finite set of moves [78-80]. For each of these models, the guiding goal is to determine if more realistic factors affect cooperative behavior.

2.2.3 Methods Used to Investigate PD

Currently the two predominant approaches to PD research are mathematical analysis [60, 62-64, 75, 81, 82] and evolutionary simulation [56, 65, 71, 74, 77, 78, 83-85]. Nowak [81] emphasizes the dynamic complexity and unpredictability of PD games despite their small number of strategies based on a simple set of rules. Besides, after using a spatial

concluded that limited computation resources produce simulation results that are heavily dependent upon the parameters involved. For these and additional reasons, PD games are usually analyzed under certain restrictions. For example, Nowak analyzed the dynamics of only three kinds of PD strategies [81]. In a separate project [82], he investigated a homogeneous population of strategies in an attempt to simulate strategy evolution.

Clear relationships among strategies (i.e., the specifics of player actions during a PD game) must be established prior to studying a PD problem. Furthermore, artificial society component categories underscore the importance of considering agent interactions.

Although the IPD model is used in this study of the public good/private interest conflict in artificial societies, an important distinction is the use of an agent viewpoint of interactions.

It is also important to note the important role played by simulations, regardless of the degree of understanding of model behavior or level of sophistication. Even in analytical studies, the majority of researchers have used simulations to collect comprehensive data on model behavior [76]. The framework described in this dissertation was used to analyze relationships between strategies—a crucial pre-simulation step that is often overlooked.

Most IPD studies make use of one of the payoff matrices shown in Table 2.4. The effect of payoff matrix choice on study results remains unclear. The model used in this study is based on the payoff matrix constraint shown in Table 2.3—in other words, it is

independent of the payoff matrix value.

Table 2.4: Some Commonly Used Payoff Matrices for the Prisoner’s Dilemma

Payoff Matrix Values [R, S, T, P]

Reference

[3, 0, 5, 1] [45, 46, 55, 65, 66, 68, 86]

[4, 0, 5, 1] [84]

[1, -2, 2, -1] [53]

[3, 1, 4, 2] [87]

Chapter 3.

The Importance of Model Analysis Prior to Artificial Society Simulation and Execution

Performing a model analysis prior to model simulation and execution is important regardless of the intended purpose of the artificial society in question. In this chapter I will offer reasons why it is important and present an illustrative example to show how it facilitates the use of artificial societies for problem solving.

Regarding the model execution/simulation process, common methodologies include the following steps: a) model creation (constructing a model based on an existing theory, hypothesis, or empirical data); b) model execution (running a model to produce data); and

c) model verification (assessing a model’s ability to operate as intended) and validation (analyzing data to ensure that a model is working as intended) [88].

Although it is rarely included in lists of model execution/simulation methodologies, theoretical and statistical model analyses play important roles. During the creation phase, model structures and relationships are mostly based on theories or hypotheses.

Theoretical variables are defined and quantified, and relationships among them are encoded [89]. In the words of Hanneman and Patrick, any model being constructed is

“one concrete realization of the prior theory” [10]. During the verification phase, the simulated results are statistically analyzed for purposes of interpretation and/or explanation [6].

There are few discussions in the literature of useful analytical tasks to be performed after a model is created but before simulation begins. At this point, it is important to determine appropriate model parameters or parameter sets based on empirical experience or existing data. The importance of analysis at this phase is the focus of this chapter.

We believe that pre-simulation model analysis can help reduce simulation complexity as well as assist in the identification of appropriate execution/simulation parameters.

3.1 Why Analyze before Model Simulation and Execution?

The most important motivation for a pre-execution/simulation model analysis is the assumption that the more one knows, the easier it will be to properly simulate or run the model. In this section I will describe how a pre-execution/simulation analysis helps in defining model scope, reducing model complexity, and choosing appropriate simulation/execution parameters.

3.1.1 Defining Model Scope

Even though defining a model’s scope is an important first step toward increasing model efficiency, it is surprising how often this step is overlooked by researchers.

Whenever an execution/simulation run provides significant findings, the data and the model clearly need to be inspected in terms of validity. But it is equally important to determine the conditions under which a particular model is successful, as well as the possibility of achieving success under other conditions.

Following model construction, concepts and entities are defined as parameters or variables. Prior to each new execution/simulation run, individual parameters must be set to specific values to satisfy some condition. An execution/simulation run is not equivalent

to a model. In this paper, a run is defined as an instance of the model. In theoretical terms, the comprehensive understanding of a model requires the execution/simulations of all possible model instances, but doing so is usually considered impractical.

A model M can be defined as

M=(P1, P2, … , Pn),

where P1, P2, … , Pn represent n parameters of M. Letting N denote the number of possible model instances and |Pi| denote the number of possible values of parameter Pi, then

N = |P1| • |P2| • … |Pn|

Each parameter has its own constraints. Examples of discrete parameters include the size of a population in a societal model and the number of nodes in a social network model [90]. Here the number of possible values is finite, but other parameters are considered continuous and infinite—for instance, tax rates in a simulation of tax and welfare systems.

Most artificial society models contain both discrete and continuous parameters; even in simple models, the number of instances is usually large or infinite. Each instance represents a tiny part of the model.

Since it is impossible to execute/simulate all model instances, it is important to choose an appropriate single model instance or set of model instances. I want to emphasize the importance of knowing the number of potential choices before making what appears to be the most appropriate choice, since the success of one model instance implies overall model success, but the failure of one model instance does not imply overall model failure. It is easier to figure out the relationship between a model and a model instance once its scope is defined.

3.1.2 Reducing Model Complexity

The second step toward successful execution/simulation involves reducing model complexity. Once the scope of a model is defined, it is no longer necessary to run all possible model instances. Unnecessary instances should be avoided in order to make the execution/simulation process more efficient. The two types of model instances that can be skipped are:

a) Unreasonable instances, meaning that a parameter setting does not match real

world conditions. These can be further divided into two categories: (1) instances with unreasonable parameter values, which are not under the constraints of the corresponding parameters; and (2) unreasonable parameter combinations, meaning that individual parameter values that are considered reasonable become unreasonable once they are

combined with other reasonable parameter values because of their correlational relationships.

b) Equivalent instances, meaning that instances may appear to be completely

different but nevertheless produce identical simulation results, or have identical meanings from the perspective of the model. An analysis of equivalent instances can provide information about whether or not an instance should be executed or simulated. It may be unnecessary to execute/simulate reasonable or important instances in cases where the results from equivalent instances are identified.

Analyses of unreasonable or equivalent instances reduce the number of potentially appropriate model instances. Using the metaphor of a highway map, the scope of a model provides the number of possible ways to get from point A to point B, while reduced model complexity provides answers to questions such as: “Which routes will not get us from point A to point B?” and “Which individual routes lead to the same destination?” By reducing the numbers of unreasonable and equivalent instances, it becomes easier to choose the appropriate parameter settings for successful execution/simulation.

3.1.3 Choosing Appropriate Model Instances

The final analytical step before execution/simulation is to determine appropriate model instances that resemble most other instances or that otherwise have some significant importance. In the literature, most model instance selections are based on empirical data or the testing of hypotheses. Nevertheless, an execution or simulation instance with significant results must still be tested to determine if it is representative of other instances and produce identical or similar outcomes. Answering such questions becomes more difficult when model instance determinations are not based on theoretical or statistical analyses.

3.2 Case Study

The example presented in this section is based on Azuaje’s [91] efforts to use a GA to evolve game strategies and cooperation. In my analytical approach the model is greatly simplified, which allows optimal solutions to be obtained more quickly and easily. Even though the robustness of artificial societies designed for problem solving (including evolutionary approaches) makes them popular among researchers in various disciplines, two important considerations are frequently overlooked: the importance of pre-run model

analysis, and the need to make a conscious decision between evolutionary and analytical approaches to individual problems based on their specific characteristics.

3.2.1 Model Description

In [91], Azuaje proposes an approach to the co-evolution of competing virtual

In [91], Azuaje proposes an approach to the co-evolution of competing virtual