DOI 10.1007/s10957-011-9799-1
n-Person Second-Order Games: A Paradigm Shift
in Game Theory
M. Larbani· P.L. Yu
Published online: 8 February 2011
© Springer Science+Business Media, LLC 2011
Abstract In this paper, we present a new approach to n-person games based on the
Habitual domain theory. Unlike the traditional game theory models, the constructed model captures the fact that the underlying changes in the psychological aspects and mind states of the players over the arriving events are the key factors, which determine the dynamic process of coalition formation. We introduce two new con-cepts of solution for games: strategically stable mind profile and structurally stable
mind profile. The theory introduced in this paper overcomes the dichotomy of
non-cooperative/cooperative games, prevailing in the existing game theory, which makes game theory more applicable to real-world game situations.
Keywords Habitual domains· Games · Coalition formation · Markov chains
1 Introduction
In [1–3], we have introduced the second-order games. These games are based on the theory of Habitual Domains [4–8] and Markov chain theory [9]. Second-order
This research was partially supported by the National Science Council, Taiwan, NSC98-2410-H009-008, NSC99-2410-H009-052.
M. Larbani
Department of Business Administration, IIUM University, Jalan Gombak, 53100 Kuala Lumpur, Malaysia
e-mail:larbani61@hotmail.com
P.L. Yu (
)Institute of Information Management, National Chiao Tung University, Hsinchu 30010, Taiwan e-mail:yupl@mail.nctu.edu.tw
P.L. Yu
games extend and generalize the normal (strategic) form games [10–14] in many important aspects: (i) normal form games are based on more or less fixed sets of strategies, while second-order games involve habitual domains of players instead of an explicit set of strategies; (ii) in normal form games it is assumed that each player have a payoff function, and know the payoff function of the others, and solutions are formulated based on strategies and payoffs; while second-order games do not involve a payoff function, rather they are based on states of mind of players; (iii) normal form games ignore both the interactions among players and between the players and the environment, that take place during the game, and cannot accommodate changes in the structure of the game, while second-order games adapt the structure of the game to the arriving events and the interaction among players and their psychological states.
In [1–3], we have analyzed two-person games only. In the present paper, we for-mulate n-person second-order games. In the framework of second-order games, there is no dichotomy cooperative (characteristic function)/ non-cooperative (normal form) games as in the existing game theory [10]. These games can accommodate cooper-ation and non-coopercooper-ation periods during the same game. In addition to the aspects related to normal form games mentioned in (i)–(iii) above, second order games gen-eralize cooperative games in many aspects: (a) cooperative games involve a charac-teristic function, whereas second-order games involve habitual domains and states of minds of players; (b) in cooperative games, solutions are formulated in terms of imputations, while in second-order games solutions are formulated in terms of states of minds of players; (c) cooperative games ignore the coalition formation process during the game, while second-order games accommodate the interaction between players and capture the dynamics of coalition formation. This is a considerable de-parture from existing game theory [10]. In fact, second order games are a paradigm
shift in game theory.
This paper is organized as follows: Sect.2presents briefly the HD theory and some examples, showing the importance of psychological aspects in n-person games, par-ticularly, the coalition formation. Section3is devoted to the formulation of n-person games in terms of the states of mind of players by HD theory and Markov chains the-ory. We introduce the concepts of [t1, t2]-strategically stable and [t1, t2]-structurally
stable profiles. In Sect.4, we estimate the average number of steps needed to reach a strategically stable profile, when it is possible. Section5provides a sufficient con-dition for the convergence of the transition probability matrices of the game to some constant transition probability matrices. Moreover, sufficient conditions for the con-vergence to a strategically stable profile are provided. Section6concludes the paper.
2 Preliminaries
In this section, we illustrate the importance of psychological aspects in games and briefly present the Habitual Domain theory (HD).
2.1 Importance of Psychological Aspects in Coalition Formation
We show, through examples, the importance of considering the psychological aspects in a coalition formation process.
Example 2.1 In 1990, Synnex, an IT intermediate whole seller and other whole
sellers were competing. No retailer was fully satisfied because competition became costly and dissatisfaction of customers increased as well. All the retailers would have liked to move to a cooperative atmosphere, but they did not know how to achieve it. Synnex came out with a great idea, which unified most of the retailers under its leadership. In Example3.1, we will explore this.
Example 2.2 Adam and his wife, Betty, used to have a peaceful life. Carol, a
col-league of Betty at work, became a friend of Betty. With time the friendship grew stronger and stronger, and Betty often invited her new friend at her house to spend some time. During Carol’s visits, a relationship developed between her and Adam that Betty ignored. After some time, Betty discovered that Adam had an affair with Carol. How did the game between Betty, Adam and Carol evolve? We will explore this case in Example3.2.
Examples2.1–2.2show clearly the importance of psychological aspects in a game and in its coalition formation process. In fact, in these cases, the states of mind of the players determine the coalition structure. The games in Examples2.1–2.2have no formulation within the framework of traditional cooperative games or strategic games, because the states of mind of players change over time according to arriving events, which make the structure of the game dynamic. For instance, in the game of Example2.2, the relationship for Adam with respect to Betty changed gradually from full cooperation to non-cooperation after Carol came into the game. Moreover, the sets of strategies are not determined and the payoffs or utility functions or the worth of the coalitions are not defined or very difficult to estimate in the dynamic environment characterizing this example. In this paper, we will show how the game situations of Examples2.1–2.2can be described and analyzed by second order games as stochastic processes in the framework of Habitual Domain theory.
2.2 Habitual Domain Theory
In this section, because of space constraint, we briefly introduce the Habitual Do-main Theory. For more details, we refer the reader to the books [4,5,7,8] and the papers [1,6].
The collection of ideas and actions (including ways of perceiving, thinking, re-sponding, acting, and memory) in our brain, together with their formation, dynamics, and basis in experience and knowledge, is called our Habitual Domain (HD) [5]. Over time, unless extraordinary or purposeful effort is exerted, our HD will become stabilized within a certain domain. This can be proved mathematically [4,15]. As a consequence, we observe that each of us has habitual ways of eating, dressing, speaking, etc. Some individuals habitually emphasize economic gains, while others pay attention to social reputation. Some habitually persist in their pursuit of goals, while others change their objectives often. Some are habitually positive and opti-mistic, while others are negative and pessimistic. Some habitually pay attention to details, others only to generalities.
The concept of an individual’s HD can be extended to other living entities, such as companies, social organizations, and groups in general. The following are the basic elements of HD.
(i) The potential domain (PD): the collection of ideas and actions that can poten-tially be activated to occupy our attention.
(ii) The actual domain (AD): the set of ideas and actions that are actually activated or occupy our attention.
(iii) The activation probabilities (AP): the probabilities that ideas or actions in PD also belong to AD.
(iv) The Reachable Domain (RD): the set of ideas and actions that can be attained from a given set in an AD.
Thus, the habitual domain can be formally formulated as
HDt= {PDt,ADt,APt,RDt} (1)
where, t represents time. The theory of HD is based on eight hypotheses H1–H8: (H1) Circuit Pattern Hypothesis, (H2) Unlimited Capacity Hypothesis, (H3) Ef-ficient Restructuring Hypothesis, (H4) Analogy and Association Hypothesis, (H5) Goal Setting and State Evaluation Hypothesis, (H6) Charge Structure and Attention Allocation Hypothesis, (H7) Discharge Hypothesis, and (H8) Information Input Hy-pothesis.
Note that, hypotheses H1–H4 describe how the brain functions and hypotheses H5–H8 describe how the mind functions. For details on these hypotheses, we refer the reader to [4,7].
Remark 2.1 Clearly, an analysis of an n-person game, based on hypotheses H1–H8
and habitual domain (1), will capture more psychological aspects than the traditional game theory does (see [12–14] and references therein). Most of aspects of hypotheses H1–H8 are not considered in the traditional game theory framework. In fact, during the game, most of the models of traditional game theory do not consider the psycho-logical states of players and their changes. For example, if the players do not interact among themselves and within the external world, they do not use hypothesis H8 at all. H2 suggests that players have an unlimited capacity to learn if they are willing to. According to H5, players have goal functions and an ideal state for each of them; they continuously monitor where they are relative to the ideal states. According to H6, a charge is a precursor of a mental force to action or inaction, which could lead to drive or stress. In this paper, for the sake of simplicity, the charge structure will be called
charge level. At any point in time, we will pay attention to the event, which has the
most important influence on our charge level. The event or decision problem, with the most significant charge, commands our attention at any given moment. When our attention is allotted to an event, then we use the following modes of action to deal with it: active problem solving or avoidance justification. The former tries to work actively to move the perceived states closer to the ideal states (discharge H7); while the later tries to rationalize the situation so as to lower the ideal states closer to the perceived states. With active problem solving, charge is transferred to drive, while with avoidance justification, charge may be reduced or transferred into stress. 2.3 Analysis of the Examples by Habitual Domain Theory
(i) In Example2.1, the small retailers are competing for market share. All retailers show somewhat non-cooperative behavior toward each other. A circuit pattern
related to this behavior has formed in the brains of the retailers (H1). By goal setting and state evaluation (H5), this behavior is perpetuated. With time, the related circuit pattern in the brains of the retailers becomes stronger and they are trapped in this behavior. Each player looks for his own interest only. In such situations, coalitions with two retailers or more cannot be formed. If there is no new arriving/future relevant event or effort exerted to change the game structure, non-cooperative behavior will prevail among the retailers (H5).
(ii) In Example2.2, a strong circuit pattern (H1) related to mutual love has formed in the brains of Adam and Betty during their long life together. They formed a strong coalition. When Carol started to come to their house, the relation be-tween Betty and Carol was strong; thus, they formed a coalition. The degree of attachment (coalition) between Betty and Carol is not as strong as that of Adam and Betty. After some time, Adam and Carol felt attracted to each other and a mutual emotional charge started to grow in both of them (H5 and H6). The relation between Adam and Carol was getting stronger with time, while the re-lation from Adam toward Betty was getting weaker. The rere-lationship from Betty toward Carol was stable, whereas the relationship from Carol toward Betty weak-ened gradually. The event that Betty came to know that Adam had an affair with Carol (H8), changed Betty’s mind (H3) and created a high level of charge (H6) because many of her life goals including perpetuation of species and self esteem, significantly deviated from their ideal level (H5), which required prompt action (H6 and H7).
In the next section, we will see how the games of Examples2.1–2.2were solved and how second-order games can accommodate such game situations.
3 Formulation of an n-Person Games in Terms of HD Theory
Let us consider a game situation involving n players: denoted by I= {1, 2, . . . , n} the set of players and by HDi = {PDi,ADi, Pi,RDi} for all i ∈ I . In order to simplify
the notations here, we have dropped the time in the expression of habitual domains of players. In terms of HD theory, a state of mind of a player can be considered as a collection of ideas or thoughts that could be activated from his potential domain PD to his actual domain AD. It can also be considered as part of his reachable domain RD. Hence, we will identify a state of mind of a player with its corresponding ideas or thoughts in the potential domain of the player.
Remark 3.1 In traditional game theory [12–14], cooperation is based on utility, while in second order games it is based on charge. A player tends to cooperates with an-other player if he perceives or believes that this player can reduce his charge level (discharge) or at least will not increase it; otherwise, he might show a behavior of non-cooperation (see “Reciprocation Behavior ” [5,7,8]). The stronger the possibil-ities of mutual release of charge (discharge), the better the cooperation between the players and the stronger the willingness to form a coalition. This observation leads to the concept of degree of cooperation (see Example2.2)
Let[0, L] be the period of time in which the game is considered.
Definition 3.1 Let i and j be two players. At a given time t , the degree of cooper-ation of player i with player j is represented by a number, dij ∈ [−1, 1],
monoton-ically, with “−1” being fully non-cooperative, and “1” fully cooperative. In general
dijbeing positive means i is cooperative with j , and negative means non-cooperative.
Remark 3.2 The number dij is the perceived feeling of discharge of player i with
respect to player j . In other words, dijis the degree of discharge (H7) that the player i
expects from player j at the given time t . When dij is positive, it means that player i
feels that player j can reduce his charge level, whereas when dij is negative, it means
that player i feels or believes that player j is hostile to him and he expects him to increase his charge level. Note dij = 1 means extreme cooperation with respect
to player j, dij= 0 means neutrality with respect to player j and dij = −1 means
extreme hostility with respect to player j .
It is noteworthy that we may have dij = dj i. Thus dij may not be symmetric,
which means that the players i and j may not have the same perception about their degrees of discharge with respect to each other. A player may sincerely cooperate with another player, but the latter may show some signs of cooperation just to deceive the former as we have seen in Example2.2between Betty and her husband Adam after Carol came into the game. Also, a player may be sincere in his cooperation with another player, but the latter may show non-cooperative behavior because he perceives the behavior of the former as a threat for his goal functions.
As the states of mind of players determine the degree of their cooperation with respect to the other players, the degree of cooperation of a player with respect to another player reflects his state of mind with respect to this player. Hence, we will use the degree of cooperation dij as a proxy for the state of mind of a player with
respect to another player.
In applications, a player could be hostile (non-cooperative) or friendly (coopera-tive) to another player (two states of mind). If needed, the moods or states of mind could be further decomposed or disaggregated into extremely hostile, hostile, neutral, friendly and extremely friendly. The number of states of mind in almost all practical cases is still finite, even if we further decompose (disaggregate) them. Therefore, we have the following assumption.
Assumption 3.1 The number of states of mind of each player with respect to each of
the other players is finite for the game under consideration.
Definition 3.2 Player i∈ I = {1, 2, . . . , n} is in the state of mind g with respect to
another player j at time t , if g is activated from his potential domain PDito his actual domain ADi at time t . That is, g∈ ADi at time t .
Assumption 3.2 At any point of time, the actual domain AD of each player contains
only one state of mind with respect to another player. That is, at any point in time, only one state can be activated from the potential domain AP to the actual domain for each of the other players.
This assumption reflects the fact that, at any moment, the actual domain (attention) of each player with respect to another player is occupied by only one state of mind.
Without any loss of generality and for the sake of simplicity, we assume that each player have the same set of states of mind with respect to each of the other players, denoted by
T = {h1, h2, . . . , hl} ⊂ [−1, 1].
Since the elements of T are the degrees of cooperation of a player with respect to each of the remaining players, we assume that they are ranked from the weakest to the strongest, as
−1 ≤ h1≤ h2≤ · · · ≤ hl≤ 1.
Remark 3.3 In case the states of mind of a player be given in a nominal scale, then
a cardinal scale can be associated. For example, if the states of minds are hostility, neutrality and cooperation, then one can associate the degrees of cooperation−1, 0, and 1, respectively.
Definition 3.3 Let K be a subset of I , such that K= ∅ and Card(K) ≥ 2. Let dij,
i, j∈ K, i = j be the cooperation degrees between players of K at a given time t.
We define the coherence index of K at time t by
C(K)= Mini,j∈K,i=jdij (2)
C(K)is a measurement of the potential that players in K, interacting among them-selves, can form a coalition. When C(K)= hi, the interaction among players in K
is such that the coalition K can be formed with degree hi. When C(K)= hl, the
interaction among players in K is such that the coalition K can be formed with the highest degree hl. When C(K)= h1, the interaction among players in K is such that
the coalition K can be formed at the weakest degree h1.
Let us explain the computation of the coherence index C(K) of a subset K of the set of players on the game of Example2.1. For the sake of simplicity, let us assume that there are three players with Synex as Player I, the other two players are retailers, denoted by Player II and Player III. Assume that each player has three possible states of mind with respect to each of the other two players as Hostility (H), Neutrality (N) and Cooperation (C). Let us assign degrees of cooperation to these three states from the interval[−1, 1], for example −1, 0 and 1 for H, N and C respectively. Then we obtain T = {h1, h2, h3} = {−1, 0, 1}.
Consider the set of all players K= I = {I, II, III}. Let us compute C(K), the coherence index of K. Assume that dI,III = 0 and dI,II= 0, that is, Player I is
neutral to Player II and Player III; dIII,I = 0 and dII,I= 0, that is, Player II and
Player III are also neutral with respect to Player I; dIII,II = −1, dII,III = −1, i.e.
Player II and Player III are hostile to each other. From formula (2), we get C(K)=
The coherence index of K= {I, II, III} is −1, which means that at least one of the players in K is hostile to another player. Since the coherence index of K is ex-tremely weak, it is very difficult for players in K to form a coalition. We have also
C({I, II}) = 0, C({I, III}) = 0 and C({II, III}) = −1 with similar interpretation. Remark 3.4 In case the cooperation degree dij is available for all i and j , one can
use other types of aggregation of the cooperation degrees than (2) with necessary changes. For instance, the average
C(K)=
i,j∈Q
dij
Card(Q)
where Q= {(i, j) ∈ K × K, i = j}, or the maximum value
C(K)= Max(i,j )∈Qdij
or the median, which is the middle value of the set{dij/(i, j )∈ Q}, after ordering
the dij values from the smallest to the largest.
In this paper we will use the definition given in (2) for C(K).
We assume that the degree of cooperation of a player with himself be 1, i.e. dii= 1
for all i∈ I = {1, 2, . . . , n}. Therefore, we will not consider subsets consisting of one player. Thus, we will focus on the set of subsets of players
= {K/K ⊂ I, K = ∅ and Card(K) ≥ 2}.
Moreover, it happens often that, in a game some subsets of players are irrelevant or do not deserve any attention. For instance, in Example 3.2, the subset {Adam, Carol} was irrelevant to the game in the period before Carol became a close friend of Betty’s. Therefore, when we analyze games, we will focus only on relevant coalitions. Formally, an irrelevant coalition or subset of players can be defined as follows.
Definition 3.4 A subset K of players is said to be irrelevant to the game if dij= 0, for all i ∈ K, j ∈ K\(i)
That is, no player in K has a tendency to cooperate or to be hostile to any other player in K.
We denote by R(, t)= {K/K ∈ , K relevant to the game situation at time t}, the set of relevant coalitions to the game at time t . Note that the determination of irrelevant coalitions may be difficult in real-world games. In this paper we do not deal with this research problem, assuming that at any time the set of relevant coalitions can be identified.
Definition 3.5 At any time t , the set{C(K), K ∈ R(, t)} is called coalition coher-ence structure of the game at time t .
Note that, if some player brought new and relevant ideas from his potential domain to his actual domain as to cause the change of the mood towards cooperation among players in K, then the coherence index of K will be changed from one state to another. Note also that for each subset K∈ R(, t), the transition from a state (coherence index) huto another state hvcould be probabilistic.
Shifting from hu to hv depend on the interaction among players in K,
arriv-ing events, and on how the coherence index C(K) of coalition K is defined (see (2) and Remark 3.4) at time t . We discuss the case of the formula (2), the other cases mentioned in Remark3.4can be discussed similarly. Assume that hu< hvand
C(K)= huat time t , according to (2), the subset K shifts to hvat some later time t
if dij≥ hvfor all i, j∈ K, i = j and the equality holds for at least one pair of
play-ers (i, j). This means that the cooperation degree of each of the players in K with respect to all the remaining players in K has increased to or maintained at a level at least equal to hv and at least one player is exactly at the degree hv with respect to
some other player. In case C(K)= hvat time t , shifting from hvto huat some later
time toccurs when the cooperation degree of at least one player drops from hvto
huwith respect to another player and the other cooperation degrees do not drop
be-low hu. Any arriving relevant event or information may trigger the process of change
in the degree of cooperation among the players of K.
Since the activation of ideas from PD to AD is probabilistic, so are the arriving events, the shifting from hu to hvcan be assumed to be probabilistic. Thus, we can
formulate the evolution of the cooperation state among the players of the subset K as a stochastic process{Xm, m= 0, 1, 2, . . .} with a finite number of states (cooperation
degrees) T = {h1, h2, . . . , hl}.
Now let us define the transition probability from any state to any other state. Given that a state hu is activated at the present step, the probability that a state hvwill be
brought to the actual domain in the next step is given by the conditional probabil-ity (PK)huhv.
Remark 3.5 In practice, the probability (PK)huhv can be evaluated by two ways. The first one is by subjective evaluation (by players or/and experts) through (2); the sec-ond one is by frequency approach i.e. the frequency of activation of the state hvif the
present state is hu, when the game is repeated a certain number of times.
In order to increase the transition probability (PK)dudv from huto hv, the players in K or external players could use the techniques of restructuring game described in [2,6,8] to create charge for moving to the state hvincluding: (a) expansion and
enrichment of HDs of players, (b) effective suggestion of new ideas to catch the players’ attention, and (c) effective integration of the new ideas with the core of the HDs of players.
Assumption 3.3 The state of the process in the next step depends only on the state
of the process in the present step i.e.
P Xp+1= hqp+1/X p= h qp, X p−1= h qp−1, . . . , X 0= h q0 = Phqp+1hqp
Assumption3.3may be justified as follows. According to HD theory [4], unless extraordinary events occur or a special effort is exerted, the activation probability of ideas and actions stabilizes over time. Thus, when there is no occurrence of an extra-ordinary event nor a special effort is exerted by players to restructure the game, we may assume that the transition probability of the processes has the Markov property. Assumption3.3implies that the evolution of coherence index of coalition K can be considered as a Markov chain with a finite number of states. Hence, the results of Markov chain theory can be used to study the evolution of the considered game. The matrix
PK= ((PK)hi,hj)hi,hj∈T
is the transition probability matrix of the process with respect to coalition K. It is a stochastic matrix that describes the coherence index of K with respect to the game situation. According to HD theory, it tends to be stable, but it is subject to changes over time when some new relevant idea or event arrives or some effort is exerted by players in K.
Thus, the game can be represented by the following model
I,{{HDi}i∈I}, {H1–H8}, S, {PK}K∈R(,t)
(3) where H1–H8 describe the dynamics and interaction of HDs of the players, S is the profile space of the game. The space S is defined as follows. Denoted by r the cardinality of the set of relevant subsets R(, t), that is, r= Card(R(, t)). Then S= T × T
rtimes× · · · × T = T
r. A generic element s of S is denoted by
s= (sK)K∈R(,t), where each subset K∈ R(, t) is represented by its coherence
index state C(K)= sK ∈ T . In fact, each profile underlines a coalition
coher-ence structure among the players and vice-versa. Indeed, we have the equivalcoher-ence s= (sK)K∈R(,t)∈ S ⇔ (C(K) = sK)K∈R(,t). Thus, the profile space S is the set
of coalition coherence structures of the game.
For instance, in the game of Example2.1before Carol started to come to Betty’s house, the relevant coalitions are {Betty, Adam}, and {Betty, Carol}. Assume that there were three states of mind weak, neutral and good, for each player, which can be represented by the degrees of cooperation −1, 0 and 1 respectively, that is, T = {−1, 0, 1}. Then, S = T2= {−1, 0, 1}2. According to the game of Exam-ple 2.1, the relationship between Betty and Adam and between Betty and Carol was good, which means the coherence coalition structure was such that the coali-tions {Betty, Adam}, and {Betty, Carol} had a good coherence index to form:
C({Betty, Adam}) = Min{dAB, dBA} = Min{1, 1} = 1 (where A = Adam, B = Betty)
and similarly C({Betty, Carol}) = 1. To this coalition coherence structure corre-sponds the profile (1, 1)∈ S = T2= {−1, 0, 1}2.
If the transition probability matrices{PK}K∈R(,t)change at some time t , then we
obtain a new game of type (3) with new transition probability matrices that starts at time t . Note that the matrices{PK}K∈R(,t)are functions of time,{HDi}i∈I and the
arriving events. The matrices{PK}K∈R(,t)capture the psychological atmosphere of
Let us now define a solution to the game (3). It is worth noting that each player will continuously look for improving his cooperation degree with players he/she be-lieves will reduce his charge level until he/she is convinced (correctly or erroneously) that optimality has been obtained and no further improvement can be obtained in the allowable time. In terms of HD theory, optimality is reached for a player at time t when he cannot reduce further his charge level. Since the players interact with each other and they are affected by arriving events, the game (3) cannot reach a stable coalition coherence structure until every player is convinced that time optimality for himself/herself is obtained. Time optimality has been defined in general in [8] as fol-lows: A player is said to have reached a time optimal solution at time t if this solution is unique and undominated at that time. Suppose a solution is time optimal over an in-terval of time[t1, t2], then we say that this solution is [t1, t2]-optimal. Using this time
optimality concept, we define the following time stability concepts for the game (3).
Definition 3.6 We say that a profile s0∈ S is a strategically stable profile of the game
(3) over the period of time[t1, t2] or simply [t1, t2]-strategically stable profile if
(i) the transition probability matrices{PK}K∈R(,t)and R(, t) are constant over
the period[t1, t2], i.e. the structure of the game remains constant during the
spec-ified period,
(ii) sK0 is an absorbing state for all K∈ R(, t) or equivalently (PK)s0KsK0 = 1, for
all K∈ R(, t).
Remark 3.6 Assume that s0 ∈ S is a strategically stable profile. By definition (PK)s0KsK0 = 1, for all K ∈ R(, t), hence (PK)s0Kh = 0, for all h ∈ T such that
h= sK0. This means that, once the players reach the strategically stable profile s0, the corresponding coalition coherence structure (C(K)= sK0)K∈R(,t)has been reached.
Moreover, the members of any coalition K∈ R(, t) cannot change their coherence index unilaterally or multilaterally in the interval of time[t1, t2]. Given the structure
of the game, the transition probability matrices{PK}K∈R(,t)and the set of players,
when the players reach a [t1, t2]-strategically stable profile, it means that they have reached an absorbing or stable profile from strategic point of view by using all strate-gies available to them. However, some of the players may not be satisfied with this coalition coherence structure. In this case, the players with a high charge level have a strong incentive to take action or generate new strategies or look for external informa-tional input (H8) to restructure the game, that is, to change the transition probability matrices{PK}K∈R(,t)or change the set of players, so that in the new game their
charge level will be lower. The restructuring process may continue until the charge level of the players is such that no player wants to restructure the game. This leads us to the following definition of time stability.
Definition 3.7 We say that s0∈ S is a structurally stable profile of the game (3) over a period of time [t1, t2] or simply [t1, t2]-structurally stable profile iff
(i) the transition probability matrices{PK}K∈R(,t)and R(, t) are constant over
the period [t1, t2], i.e. the structure of the game remains constant during the specified period,
(ii) s0K is an absorbing state for all K∈ R(, t) or equivalently Ps0
KsK0 = 1, for all
K∈ R(, t),
(iii) the charge levels of players are low, such that they do not want to change the structure of the game i.e.{PK}K∈R(,t)and R(, t).
Remark 3.7 A [t1, t2]-structurally stable profile is a [t1, t2]-strategically stable pro-file, where players do not have an incentive to restructure the game because they are satisfied with its structure. Thus, the concepts of stability considered in traditional games are special cases of strategically stable profiles.
Remark 3.8 Assume that s be a strategically stable profile during a period. By
defini-tion, (PK)sK,sK= 1, hence PsK,sK= 0, for all s
K= sKand for all K∈ R(, t). This
means that any deviation, unilateral or multilateral, from s is impossible, once the players reach it. Thus, a strategically stable profile is stable locally and globally. This makes it definitely different and more general than Nash equilibrium [11]. Indeed, Nash equilibrium is immune to unilateral deviations but it is not immune to multilat-eral deviation. Many researchers have introduced various refinements of Nash equi-librium that are immune against multilateral deviations (coalition deviation) such as the strong equilibrium [16], Coalition Proof Nash equilibrium [17] and Strong Berge equilibrium [18]. However, the major difficulty with these equilibria is that they do not exist in most of the games and the few existence results established are restrictive e.g. [19]. Furthermore, Nash equilibrium is based on strategies and utility function, while the strategically stable profile is based on states of mind and charge level.
Remark 3.9 In order to avoid distraction, we show the most important differences
between the traditional games [10] and the second-order game (3) as follows: (i) In traditional games, there are two major classes: the non-cooperative games
and the cooperative games. The two classes are studied by two totally differ-ent models: the former is studied in the framework of games in normal form or strategic form; the latter is studied in the framework of characteristic function games. In non-cooperative games, the coalition formation process is missing, while in cooperative games, the strategic aspect is missing. This artificial di-chotomy does not completely reflect reality. In fact, in real games, most of the time, a game goes through cooperative and non-cooperative stages according to the arriving events, the interaction of players and their psychological states; moreover, coalition formation and strategic aspects cannot be separated, they oc-cur at the same time. Thus, this artificial dichotomy is one of the major reasons that hinder traditional game theory from handling the evolution of real-world games. The second-order games accommodate well the non-cooperative and co-operative (strategic and coalition formation) aspects, that occur simultaneously during a game.
(ii) In traditional games, the outcomes of the game and the worth of coalitions are expressed in terms of a utility function. In the game (3), the outcomes are not in terms of payoff, rather they are evaluated in terms of charge level [5,8] and states of mind of players. The objective of each player is to reduce his/her charge level.
(iii) In traditional games, the players look for a strategically stable profile. Such play-ers are called ordinary or regular or first order playplay-ers because they assume that the structure of the game is given and then they do their best to reduce their charge level within that structure. However, restructuring games for reaching a structurally stable profile requires more effort, patience, negotiation skills, cre-ativity and sometimes power or force. Generally, it can be achieved by superior
or second order players in the framework of second-order games.
(iv) In [20], Aubin has introduced the notion of fuzzy coalition in cooperative games for the first time. This notion considerably improved the original characteristic function model. However, the basic elements of the model remained unchanged and restructuring the game was not considered. In second order games, coalition formation is studied as a dynamic process.
From this comparison, we can conclude that the difference between the two mod-els is conceptual and structural, and second-order games are closer to real-world game situations.
Now let us illustrate by Example2.1how the game (2) can be constructed.
Example 3.1 The game of Example2.1had evolved as follows. Synex Inc. built a good computer information system, inventory delivery system and a group of profes-sional technicians. These services were offered to small retailers, who usually need capital for inventory and technical support, if they subscribed to Synex network for a nominal fee. Synex guaranteed 24 hours supply of inventory (products) and com-pletion of needed repair within 24 hours. This was a big success with the majority of retailers who formed a coalition with Synex Inc. The business for Synex prospered and customers and retailers were happy.
Let us explain this solution by HD theory. Synex restructured the game in such a way that made the other retailers change their minds and proceeded to cooperate with it (H3). The other retailers thought that the Synex’s offer was an opportunity to improve their income (H4 and H5). Therefore, this offer was perceived by retailers as an opportunity to decrease their charge level (H6), which became the drive for change in the game situation that generated a cooperative atmosphere. They accepted the offer in order to release the charge (H7). By (H3), their behavior shifted from non-cooperation to cooperation. It is very important to note here, that the game went through two phases. The first one was a non-cooperative phase; the second one was a cooperative phase. In literature, there is no game theory model, which accommodates such a shift from a non-cooperative game to a cooperative one.
For the sake of simplicity, let us assume that there be three players with Synex as Player I, the other two players, II and III are retailers. The game could be divided into two distinct phases. Phase I covers the period before Synex made the offer; Phase II covers the period after Synex’s offer. Phase I is a non-cooperative phase; Phase II is a cooperative phase. Thus, the game cannot be studied in the framework of traditional game theory. We also assume that the set of states mind of each player, with respect to each of the other players, is non-cooperation, neutrality and cooperation respectively to which we associate the cooperation degrees−1, 0 and 1, respectively. Then T =
{h1, h2, h3} = {−1, 0, 1}. The game can be represented by the model (3) as follows
{I, II, III}, {HDi}
i∈{I,II,III},{H1–H8}, S, {PK}K∈R(,t)
where, the set of profiles is S= T4= {−1, 0, 1}4, R(, t)= {{I, II}, {I, III}, {II, III}, {I, II, III}}. Assume the transition probability matrices for the coalitions {I, II}, {I, III}, {II, III} and {I, II, III} be respectively
P{I,II}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 0.1 0.3 0.6 1 0.1 0.2 0.7 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 0.1 0.5 0.4 1 0.15 0.1 0.75 ⎞ ⎟ ⎟ ⎟ ⎠, P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 0.1 0.4 0.5 1 0.1 0.1 0.8 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 0.3 0.3 0.4 1 0.3 0.2 0.5 ⎞ ⎟ ⎟ ⎟ ⎠ Here, it is important to note that the determination of the entries of the above four transition probability matrices is a challenging task. The provided values are our eval-uations based on the game situation. In Remark3.5, we have mentioned two possible ways for the determination of these entries: the subjective evaluation by the players and/or by experts; and the frequency approach by using historical data. Thus, the implementation of these two methods can be a topic for further research.
Clearly, the state−1 is absorbing for all coalitions, which means that no coali-tion can form, thus, the game is non-cooperative. Moreover, in Phase I of this game, the players are in competition, which means that they are in the profile
s= (−1, −1, −1, −1) ∈ S = {−1, 0, 1}4. Since the players are in the absorbing pro-file s= (−1, −1, −1, −1), they will remain there until some relevant event arrives or an effort is exerted by some or all players to restructure the game. In other words, if there is no restructuring of the game that brings changes in the minds of some of the players, i.e. in transition probability matrices, the players will remain in the com-petition atmosphere s= (−1, −1, −1, −1). Thus, we can conclude that the profile
sis strategically stable from the time the game begun until the present time, and it will continue to be so until restructuring of the game occurs. This means that all the strategies that the players develop and implement will be to enhance or perpetuate the profile s= (−1, −1, −1, −1). With time, competition may be very costly and harmful. Therefore, the persistence of competition may increase the charge level of some or all players (H5 and H6). The increasing charge level will make them look for cooperative solutions. Moreover, in business, one of the most important goals is to increase profit (H5); therefore, players welcome any idea or event that may increase their profit, including cooperation with the competitors. In other words, the players are willing to restructure the game when they perceive that it is beneficial for them to do so, or when they see that it will reduce their charge. This shows that the profile
s= (−1, −1, −1, −1) is not structurally stable, because the players’ charge levels
are still high. At the beginning of Phase II, Player I, Synex, came out with an offer to restructure the game that triggered the cooperative spirit among players.
At the beginning of Phase II, we assume that the transition probability matrices for the four coalitions{I, II}, {I, III}, {II, III}, {I, II, III} be respectively:
P{I,II}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.2 0.8 0 0.0 0.1 0.9 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.15 0.85 0 0.0 0.05 0.95 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠, P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.2 0.5 0.3 0 0.0 1 0.0 1 0.1 0.2 0.7 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.1 0.6 0.3 0 0.0 1 0.0 1 0.1 0.2 0.7 ⎞ ⎟ ⎟ ⎟ ⎠ Clearly, the states 1, 1, 0 and 0, become absorbent for {I, II}, {I, III}, {II, III} and {I, II, III}, respectively, and s = (−1, −1, −1, −1) is no more absorbing. The ob-tained coherence coalition structure (1, 1, 0, 0) can be explained as follows. The co-herence index of{I, II} and {I, III} is good because players II and III accepted the offer of Player I, a cooperative atmosphere is established within these two subsets of players. The coherence index of{II, III} becomes moderate because of the coopera-tive spirit triggered by the offer of Player I. As for the set of all players{I, II, III}, according to the aggregation principle (2) we have adopted in this paper, its coher-ence index becomes moderate because the cohercoher-ence index of{II, III} is moderate, although the coherence indexes of{I, II} and {I, III} are good. In Sect.4, we will compute an estimate of the average number of steps needed for the game to reach the strategically stable profile (1, 1, 0, 0).
The profile (1, 1, 0, 0) is strategically stable and structurally more stable since all the players are satisfied with the new game structure. However, we cannot claim that it is structurally stable because the cooperative spirit among players can be improved by proposing similar offers to reduce costs (inventory, transportation, etc) and improve the quality of the products. Using the concepts of supply chain management may be very helpful to restructure the game as to make the profile (1, 1, 1, 1) structurally stable. Indeed, a game where firms compete is structurally less stable than a game where all firms form a supply chain, because in competition the players increase the charge level of each other, whereas in a supply chain, the players reduce the charge level of each other.
Example 3.2 The game of Example 2.2eventually ended as follows. After Betty came to know that her husband had an affair with Carol, the relations between Adam and his wife started to deteriorate, they eventually divorced. Then Adam married Carol. Let us explain the end of this game by HD theory.
When Betty came to know that her husband had an affair with her friend (H8), her relationships with both Adam and Carol deteriorated because she felt that her life goals were seriously hurt (H5) and her charge level became very high (H6). She immediately cut her relation with Carol (H3 and H7). After a fierce conflict with Adam, they eventually divorced (H7). Adam and Carol married to release their mutual emotional charge (H7). It is worth noting here, that this game cannot be formulated in the framework of traditional game theory because coalition structure and the set of players changed over time. Second order games can accommodate the change of the set of players (H8).
Let us formulate this game as a three-person second-order game: denoted by Play-ers I, II and III, Adam, Betty, and Carol respectively. The evolution of this game can be divided into four Phases: Phase I, the period when Player I and II were living in love and harmony; Phase II, the period where a new player, Player III, entered the game and became a friend of Player I; Phase III, the period when Player III used to come to the house of Player II; Phase IV the period after the Player II came to know that Player I had an affair with Player III.
In Phase I, the game can be formulated using the model (3) as follows.
As in the previous example, assume that the set of states of mind of each player, with respect to each of the other players, be T = {−1, 0, 1}, with −1 representing non-cooperation, 0 neutrality and 1 cooperation, respectively. The game can be rep-resented by the model (3) as follows
{I, II}, {{HDi}
i∈{I,II}}, {H1–H8}, S, {PK}K∈R(,t)
with the set of profiles S= {−1, 0, 1} and R(, t) = {{I, II}} over the Phase I. Note here at the Phase I, the coalitions{I, III}, {II, III} and {I, II, III} are irrelevant to the game situation. Assume the transition probability matrix for the coalition{I, II},
P{I,II}= ⎛ ⎝ −1 0 1 −1 0.0 0.0 1 0 0.0 0.0 1 1 0.0 0.0 1 ⎞ ⎠.
During the Phase I, the Players I and II were in the state 1. Since the state 1 is ab-sorbing and no player wants to restructure the game, the players will remain in the profile 1. Thus, the profile 1 is strategically and structurally stable during the period of Phase I.
A change of the game started with the appearance of the Player III in Phase II. The game became a three person game
{I, II, III}, {{HDi}
i∈{I,II}}, {H1–H8}, S, {PK}K∈R(,t)
with the set of profiles S= {−1, 0, 1}3, where R(, t)= {{I, II}, {I, III}, {II, III}}. The coherence index of the subset{I, II}, C({I, II}) was still good because Player I did not have any close relation with Player III. The coherence index of coalition {I, III}, C({I, III}) was moderate because the interaction between Players I and III was formal. The coherence index of coalition{II, III}, C({II, III}), was good because
Players II and III were friends. The game can be characterized by the following tran-sition probability matrices
P{I,II}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.2 0.8 0 0.0 0.1 0.9 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.5 0.5 0 0.0 1 0.0 1 0.0 0.5 0.5 ⎞ ⎟ ⎟ ⎟ ⎠, P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.15 0.85 0 0.0 0.05 0.95 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠ (4)
Thus, we conclude that the profile (1, 0, 1) is absorbing. Given the relations between players, (1, 0, 1) is strategically and structurally stable during this phase.
A second change occurred when Player III started to come to the house of Player II in Phase III. Player I and Player III developed a relationship that Player II ig-nored. The set of relevant coalitions becomes R(, t)= {{I, II}, {I, III}, {II, III}, {I, II, III}}. This change affected the transition probability matrices (4) of the game in the Phase II as follows.
The coherence index of the subset{I, III}, C({I, III}) = Min{dI,III, dIII,I} became
good because of the developed relation between Players I and III. As a result, the coherence index of the coalition{I, II}, C({I, II}) = Min{dI,II, dII,I} decreased
be-cause dI,IIdecreased and dII,I remained constant because Player I ignored the
rela-tion that developed between Players I and III. The coherence index of subset{II, III},
C((II, III))= Min{dII,III, dIII,II}, decreased because of the relation between Player III
and Player I. The coherence index of the subset{I, II, III}, C({I, II, III}) was also weak. One can characterize this phase by the following transition probability matri-ces P{I,II}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.8 0.1 0.1 0 0.6 0.2 0.2 1 0.5 0.2 0.3 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.15 0.85 0 0.0 0.05 0.95 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠, P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.0 1 0 0.0 0.0 1 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠, P{I,II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 1 0.0 0.0 1 1 0.0 0.0 ⎞ ⎟ ⎟ ⎟ ⎠ (5)
We notice that in this phase, there is no absorbing profile because the matrix P{I,II} has no absorbing state. Therefore, there cannot be any strategically or structurally
stable profile. This phase is a turbulent phase. The continuous increase in the charge level of the Players I and III indicates that an imminent change/restructuring of the game will occur.
A third change of the game took place in Phase IV, when the Player II came to know that the Players I and III had an affair that she ignored. This information af-fected the transition probability matrices (5) of the game in Phase III.
The Player II felt very frustrated and her life goals were seriously hurt. As a re-sult, she cut her relation with Player III and divorced Player I. Therefore, the co-herence index of the subsets{I, II}, {II, III} and {I, II, III}, C({I, II}), C({II, III}) and C({I, II, III}) respectively became weak in the sense of (2) because dII,I, dII,III
significantly decreased. The cooperation tendency of coalition{I, III}, C({I, III}) reached the maximum degree, since Player I and Player III got married. One can characterize this phase by the following transition probability matrices
P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 0.0 0.0 1 0 0.0 0.0 1 1 0.0 0.0 1 ⎞ ⎟ ⎟ ⎟ ⎠,
P{I,II}= P{I,II,III}= P{I,III}=
⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 1 0.0 0.0 1 1 0.0 0.0 ⎞ ⎟ ⎟ ⎟ ⎠.
According to the form of the transition probability matrices {PK}K∈R(,t), the
players shift to the profile (−1, −1, 1, −1), in one step. Since (−1, −1, 1, −1) is ab-sorbing, they will remain there as long as the structure of the game remains the same. The coherence coalition structure of the game that corresponds to{PK}K∈R(,t)and
the profile (−1, −1, 1, −1) is such that the coherence index of {I, II}. {I, III} and {I, II, III} is weak, and the coherence index of {II, III} is maximum. No player has incentive to restructure the game. The profile (−1, −1, 1, −1) is structurally stable.
4 Reaching Stable Profiles
Consider the game (3) with transition probability matrices{PK}K∈R(,t)at time t1.
We address the following problem. Assume that the game has a[t1,¯t]-strategically
stable profile s0∈ S. Is it possible to reach s0within the interval[t1,¯t]? If yes, then
what is the average number of steps required to reach it?
First, we determine the class of transition probability matrices{PK}K∈R(,t)for
which s0can be certainly reached from any starting state when there is no time con-straint. Then we study the possibility of reaching s0within the time constraint[t1,¯t].
4.1 A Class of Transition Probability Matrices for Which s0Can Be Reached In this section, we address the following problem: Given a profile s0 what are the classes of transition probability matrices{PK}K∈R(,t)for which s0will be certainly
reached?
Here we assume that the reader is familiar with basic notions of recurrent and transient state, and recurrent and transient class in a finite Markov chain. For more details, we refer the reader to [9]. Let us first recall some important results from Markov chain theory.
Theorem 4.1 In a finite Markov chain, as the number of steps tends to infinity, the probability that the process is in a transient state tends to be zero irrespective of the state at which the process starts.
In other words, starting at any state, the process will ultimately join a recurrent class and remain there.
Henceforth, when we say that a profile will be reached, we mean that it will be reached with probability 1, i.e. with certainty.
Proposition 4.1 Consider the game (3). Assume the set R(, t) and transition
prob-ability matrices{PK}K∈R(,t)become constant on[t0,+∞[ and for each coalition
K∈ R(, t) the corresponding Markov chain has only one recurrent class RK, and
that RK= {sK0} i.e. sK0 is absorbent and it forms the unique recurrent class in the
Markov chain corresponding to coalition K, then starting from any profile s, the pro-file s0will be reached. Let t
Kbe the first time sK0 is reached for all K∈ R(, t) then
the profile s0will be reached at the time defined by t∗= Sup tK, K∈ R(, t)
(6)
In case tK is finite for all K ∈ R(, t), then s0 is also reached at the finite time
t∗= Max{tK, K∈ R(, t)}.
Remark 4.1 Note that this is possible because R(, t) is finite. Otherwise, s0may be reached at infinity.
Proof Let K be any coalition in R(, t) and PK its transition probability matrix.
Let (PK)RK be the transition probability matrix within RK= {s 0
K}. If the process
starts in the unique recurrent class RK= {sK0}, then it will remain there because sK0
is absorbent. If the process starts in a transient state, then according to Theorem4.1, certainly the process will join the unique recurrent class RK= {sK0}. The second part
of Proposition4.1is straightforward.
Proposition 4.2 Assume that in the game (3), R(, t) and the matrices{PK}K∈R(,t)
be constant over some interval[t1,+∞[, the profile s0∈ S be absorbing and the Markov chain of each coalition K∈ R(, t) have a unique recurrent class, namely,
{s0
K}. Then, the game (3) will reach the[t1,¯t]-strategically stable profile s
0within the
Proof According to Proposition4.1, starting from any profile s, the profile s0will be reached at time t∗= Sup{tK, K∈ R(, t)}, where tK is the first time sK0 is reached
for all K∈ R(, t). Let ¯t > t1, by Definition3.6, the profile s0is[t1,¯t]-strategically
stable. Then, if t∗≤ ¯t, the profile s0will be reached within the interval[t1,¯t]. Remark 4.2 In the game (3), according to Propositions4.1–4.2, when s0is absorbing and for each coalition K, the corresponding Markov chain has only one recurrent class, namely,{sK0}, then the profile s0will be reached. In all other cases, it may be impossible for the game to reach the profile s0. Let us explain this. Recall that in a finite Markov chain there is at least one recurrent class [9]. Moreover, it is well known that once the process joins a recurrent class, it will stay there forever [9]. Therefore, when there is at least one coalition K∈ R(, t) having more than one recurrent class, the process may enter a class different from the class{sK0} and remain there, before it visits the state sK0. Thus, it would be impossible for the process to reach the profile s0. It is important to note that, the assumption that s0is absorbing is essential because in case s0is not absorbent, it may be impossible that all (sK0)K∈R(,t)are reached
simultaneously.
4.2 Average Time for Reaching a Strategically Stable Profile
In Sect.4.1, we have proved that under some conditions the game converges to a [t1,¯t]-strategically stable profile without precision on the time it takes to reach such
a profile. In this section, we will provide an estimation of the average time the game takes to reach a strategically stable profile.
Proposition 4.3 [9] Let K∈ R(, t) be a subset of players in the game (3) and PK
be the corresponding transition probability matrix. Let R be the unique recurrent class of the Markov chain corresponding to K and R= {sK0}. Let us use the standard notations: denote by s0K= i, and by j any other state in TR (the set of transient states). Consider the matrix
˜ PK= 1 0 ˜S Q (7)
where, Q is obtained from PK by deleting its ith row and the ith column and ˜S
is obtained from the ith column of PK by deleting its ith entry. Then, the expected
number of steps starting at any state j in TR until reaching state i is given by Nj=
k=i
Fj,k
where, Fj,kis the expected number of visits to k, starting at j , which is given by the
(j, k)th entry of the fundamental matrix F= (I − Q)−1, here I is the identity matrix.
Moreover, let V (Nj)denote the variance of the number of steps the process stays in
transient states before reaching the state i starting from state j and V = (V (Nj)j=i)
the column vector where the j th component is V (Nj). Then
where N= (Nj)j=i is a column vector where the j th component is Nj and H =
(Nj2)j=i, where the j th component is the square of Nj.
Proposition 4.4 Assume that in the game (3), R(, t) and the matrices{PK}K∈R(,t)
be constant over some interval [t0, t1], the profile s0∈ S be absorbing and the
Markov chain of each coalition K∈ R(, t) have a unique recurrent class, namely, R= {sK0}. Then, starting from some profile s ∈ S, the average time ¯T for the game to reach the profile s0 is finite and can be taken as ¯T = Max{¯tK, K∈ R(, t)}, where ¯tK is the average time for the coalition K ∈ R(, t) to reach sK0,
start-ing from the state s. Moreover, assume that ¯T = ¯tK for some K∈ R(, t) and t1> ¯T + 3V (NsK). Then, the probability that the[t
0, t1]-strategically stable
pro-file s0∈ S is reached within the interval [t0, t1] is 0.89, given by P (|T − ¯T | <
3V (NsK))≥ 1 − 1/9 = 0.89.
Proof The first part of the proposition is a straightforward consequence of
Proposi-tion4.3. The second part of the proposition can be easily derived from the Cheby-chev inequality [21] that states that P (|T − ¯T | > ε) ≤ V (NsK)/ε
2, by taking ε= 3V (NsK), we get P (|T − ¯T | ≤ 3
V (NsK))≥ 1 − 1/9 = 0.89.
Example 4.1 Let us consider the profile (1, 1, 0, 0) at the Phase II of Example3.1. The profile (1, 1, 0, 0) is[t0, t1]-strategically stable in Phase II (t0is the beginning of Phase II), because the states 1, 1, 0 and 0 are absorbing states. We note also that, {1}, {1}, {0}, {0} are the unique recurrent classes in their transition probability matri-ces, respectively. Thus, according to Proposition4.3, the profile (1, 1, 0, 0) will be reached and the average number of steps needed to reach it can be calculated using Propositions4.2–4.4.
First, we calculate the average number of steps to reach the corresponding absorb-ing state for each coalition.
For the coalition{I, II}. Rearranging the states, we get
P{I,II}= ⎛ ⎜ ⎜ ⎜ ⎝ 1 0 −1 1 1 0.0 0.0 0 0.9 0.1 0.0 −1 0.8 0.2 0.0 ⎞ ⎟ ⎟ ⎟ ⎠, P˜= 1 0 ˜S Q = ⎛ ⎜ ⎜ ⎜ ⎝ 1 0 −1 1 1 0 0 0 0.9 0.1 0 −1 0.8 0.2 0 ⎞ ⎟ ⎟ ⎟ ⎠, Q= 0.1 0 0.2 0 , then, (I− Q) = 0.9 0 −0.2 1 , F= (I − Q)−1= 0.9 0 2/9 1 .
The average number of steps for reaching the state 1, starting from the state −1, is E(1|−1) = 2/9 + 1 = 1.2 steps. It is the sum of the entries of the row of F
corresponding to the state −1 (the second row). The average number of steps for reaching the state 1, from the state 0, is E(1|0) = 0.9 + 0 = 0.9. Thus, we can take the average number of steps needed to reach 1, starting from any state as
μ{I,II}(1)= Max{E(1/ − 1), E(1|0)} = Max{0.9, 1.2} = 1.2.
Using similar calculations on the transition probability matrix P{I,III}of the coali-tion{I, III} in Phase II, the average number of steps for reaching the state 1, start-ing from the state−1, is E(1|−1) = 0.16 + 1 = 1.16. The average number of steps for reaching the state 1, from the state 0, is E(1|0) = 1/0.95 + 0 = 1.05. Thus we can take the average number of steps needed to reach 1, starting from any state as
μ{I,III}(1)= Max{E(1|−1), E(1|0)} = Max{1.16, 1.05} = 1.16.
Similarly, the average number of steps for reaching the state 0 for the coalition {II, III} are computed based on the matrix P{II,III}as follows.
The average number of steps for reaching the state 0, starting from the state, 1 is
E(0|1) = 1/2.1+8/2.1 = 4.28. The average number of steps for reaching the state 0,
from the state−1, is E(0|−1) = 3/2.1 + 3/2.1 = 2.85. Thus we can take the average number of steps needed to reach−1, starting from any state as
μ{II,III}(0)= Max{E(0|1), E(0|−1)} = Max{4.28, 2.85} = 4.28.
Similarly, the average number of steps for reaching the state 0 for the coalition {I, II, III} are computed based on the matrix P{I,II,III}as follows.
The average number of steps for reaching the state 0, starting from the state−1, is E(0|−1) = 3.5/1.9 + 0.33/1.9 = 2.01. The average number of steps for reaching the state 0, from the state 1, is E(0|1) = 1/1.9 + 1/1.9 = 1.05. Thus, we can take the average number of steps needed to reach 0, starting from any state as μ{I,II,III}(0)=
Max{E(0|−1), E(0/1)} = Max{2.01, 1.05} = 2.01.
Finally, the average number of steps, that is necessary for reaching the strate-gically stable profile (1, 1, 0, 0), during the Phase II, is the maximum among the four calculated averages ¯T = Max{μ{I,II}(1), μ{I,III}(1), μ{II,III}(0), μ{I,II,III}(0)} = Max{1.2, 1.16, 4.28, 2.01} = μ{II,III}(0)= 4.28. Let us now compute the variance of
T for the coalition{II, III}, we have
P{II,III}= ⎛ ⎜ ⎜ ⎜ ⎝ −1 0 1 −1 1 0.0 0.0 0 0.1 0.4 0.5 1 0.1 0.1 0.8 ⎞ ⎟ ⎟ ⎟ ⎠, ˜ P= 1 0 ˜S Q = ⎛ ⎜ ⎜ ⎜ ⎝ 0 −1 1 0 1 0 0 −1 0.5 0.2 0.3 1 0.2 0.1 0.7 ⎞ ⎟ ⎟ ⎟ ⎠, F= (I − Q)−1= 3/2.1 3/2.1 1/2.1 8/2.1 .
Then, by Proposition4.3, we have V = (2F − I)N − H = 5/2.1 6/2.1 2/2.1 15/2.1 6/2.1 9/2.1 − 36/4.41 81/4.41 = 10.88 12.24 .
Hence, the maximum variance of the number of steps for reaching the profile
(1, 1, 0, 0) is the variance for reaching the state 0 from the state 1, V (N0)= 12.24,
hence the variance of T is V (T )= 12.24. Using Proposition4.3, we conclude that if t1> ¯T + 3√12.24, and (1, 1, 0, 0) is[t0, t1]-strategically stable profile, (1,1,0,0)
will be reached within[t0, t1], with probability 0.89.
Remark 4.3 As we see, the game of Example 2.1 consists of two phases: non-cooperative and non-cooperative. The shift from the non-non-cooperative phase to non-cooperative phase took place via restructuring the game by the offer of Player I. Here, Player I can be considered as a second order player because he could restructure the game so as to reduce his own charge level and the charge level of the other players. Clearly, the traditional game theory framework cannot accommodate such games, while second order games can.
5 On the Convergence of the Transition Probability Matrices
By definition, a necessary condition for the games (3) to reaches a [t0, t1 ]-strategically stable profile s0is that the transition probability matrices{PK}K∈R(,t)
become constant after t0. In Sects.4.1–4.2, we have established sufficient conditions for the game (3) to reach a[t0, t1]-strategically stable profile s0 after the time t0. However, we did not consider the period before the transition matrices become con-stant. The following question arises: Are there any sufficient conditions for the tran-sition probability matrices{PK}K∈R(,t) to converge to constant matrices? In this
section, we address this question. In [4], it has been proven that the activation proba-bilities from the potential domain to the actual domain of a decision maker stabilize over time (by using differential equations as a means of expression of the dynamics of these probabilities). We provide another sufficient condition for the stability of these probabilities that is based on monotonicity. Then, using this result, we provide some sufficient conditions for the transition probability matrices{PK}K∈R(,t)of the
game (3) to converge to constant matrices.
Proposition 5.1 Consider a player in the game (3), and x an idea or state or concept
in his PD (potential domain). Let Pt(x)be the activation probability of x from his
PD to his AD (actual domain) at time t . Assume that starting from some time t0, the
activation probability Pt(x)becomes monotonic, that is,
(i) Pt+1(x)≥ Pt(x), for all t≥ t0, or (ii) Pt+1(x)≤ Pt(x), for all t≥ t0.
