TABS Customization for MagePowerCraft

As we have mentioned in Section 4.5 earlier, some game-dependent parts of TABS need to be customised for each game to help developing the candidate controllers more efficiently. We describe them as follows.

5.2.1 Inputs and Outputs of ANNs in MagePowerCraft

Inputs of ANNs:

We collect the useful gaming information in MagePowerCraft for combating and feed the information as inputs to the ANNs in every simulation time step. Note that all input values are normalised to the interval of [0, 1]. We give the descriptions of the inputs along with their normalisation formulae as below:

• Ready_i^{N orm}: The ready-rate of a skill i, normalised as Equation 5.6, where T otalCD_i is the total CD time of skill i and RemainCD_i is the remain CD time of skill i, i = 1, 2, ..., n, and n is the total skill number of the character.

Ready^{N orm}_i = (T otalCD_i− RemainCDi)/T otalCD_i (5.6)

• HP_i^{N orm}: The current HP of all characters, including self, friendly and hostile targets, normalised as Equation 5.7, where HP_i and M axHP_i is the current and maximum HP of character i, i = 1, 2, ..., T otalCharacterN umber.

HP_i^{N orm}= HPi/M axHPi (5.7)

• DeBuffT ime^{N orm}_i,j : The remaining time of de-buffing status i of hostile target j, normalised as Equation 5.8, where Debuf f T ime_i,j is the remaining time of de-buffing status i of hostile target j, M axDebuf f T ime is the maximum duration time of de-buffing status i, i = 1, 2, ..., T otalDebuf f StatusN umber and j = 1, 2, ..., T otalHostileCharacterN umber.

Debuf f T ime^{N orm}_i,j = Debuf f T ime_i,j/M axDebuf f T ime_i (5.8)

• Dist^{N orm}_i : The distance between the character all other character, normalised as Equation 5.9, where Dist_i is the distance between the character and target i, i = 1, 2, ..., (T otalCharacterN umber− 1) and MaxDist is the maximum distant of any two character, i.e. the size of the arena.

Dist^{N orm}_i = Dist_i/M axDist (5.9)

• Casting_i^{N orm}: Indicating the skill that the hostile target i is casting, computed as Equation 5.10, where SkillID is the mapped skill ID from 1 to T otalSkillN um_i

of which hostile target i is currently casting. If the hostile target i is not casting any skill, then SkillID is set assigned as 0. T otalSkillN um_i is the total number of skill of hostile target i, i = 1, 2, ..., T otalHostileCharacterN umber.

Casting^{N orm}_i = CastingSkillID/T otalSkillN um_i (5.10)

Outputs of ANNs:

The output interval [0, 1] of each output is evenly divided into several sub-intervals as needed, and each sub-interval is mapped to an action or decision of the character. All the ANNs in MagePowerCraft have four outputs, they are:

• F riendlyT gt: Indicating one of the friendly (including self-targeting) character target. If an aiding skill is chosen to cast by the character, then the aiding skill would be cast on this target or to the direction of this target.

• HostileT gt: Indicating one of the hostile character target. If an attacking skill is chosen to cast by the character, then the attacking skill would be cast on this target or to the direction of this target.

• MovementT gt: May be indicating any character target except self-targeting.

This target is used as a reference target for movements of the character.

• CastSkill: Indicating whether to cast a skill, if yes, which skill of the character is selected for casting.

• Movement: Indicating to move forward to the MovementT gt, to move away from the M ovementT gt or to stay and not moving.

We differentiate the two cases about skill casting into friendly (F riendlyT gt) or hos-tile (Hoshos-tileT gt) targets. An advantage is that the controllers do not need to learn about the skills that are cast on friendly targets and hostile targets, respectively. This is reason-able as an experienced player should know well about that.

5.2.2 Fitness Function for MagePowerCraft

We design a fitness function for evaluating the performance scores of the controllers for MagePowerCraft. The total fitness is computed as follows:

T otalF itness = max(0, F itness_gain− F itnessloss), (5.11) where F itness_gainand F itness_lossare the gain and loss values, respectively. F itness_gain is given by:

F itness_gain =AtkP t + AidP t + SkillCasting + SkillF iring+

SkillBreaking + SkillEvading + LearnedSkills,

(5.12)

where the variables are expained as below:

• AtkP t: The damage points inflicted to the hostile targets.

• AidP t: The healing points done to/from the friendly targets.

• SkillCasting: The reward value for starting to cast skills. Each time when the character starts to cast a skill, a value is added to SkillCasting as reward.

• SkillF iring: The reward value for casting skills successfully. whenever a skill is cast and fired successfully, a value is added to SkillF iring as reward.

• SkillBreaking: The reward value for interrupting the skills being cast by hostile targets.

• SkillEvading: The reward value for moving away from the attack range of the skills of the hostile targets.

• LearnedSkills: The reward value for learning to cast different skills.

The fitness value is decreased by F itness_losswhich is given as:

F itness_loss= HurtP t + CastW hileCD + OutOf Range + SkillBroken, (5.13)

where the variables are explained as follows:

• HurtP t: The damage points inflicted by the hostile targets.

• CastW hileCD: The punishment for casting skills in while the skills are still in CD.

• OutOfRange: The punishment for casting skills on the invalid targets which are too far away from the character and out of range the range of the casting skills.

• SkillBroken: The punishment for being interrupted by the hostile targets while casting skills.