Particle Swarm Optimization (PSO) is a swarm intelligence optimization algorithm. It was first proposed by J. Kennedy and R. C. Eberhart in 1995 [11] to simulate the foraging behavior of bird flocks. The swarm is composed of particles that move around in a given multi-dimensional search space to find the best solution. Each particle updates its velocity according to its historical experience, as well as the information of the neighboring par-ticles. The neighborhood of a particle is a set of information links defined by the swarm topology. PSO iteratively updates the swarm topology, the velocities, and the positions of each particle until the global optimum solution is found.
Throughout the years, numerous variants of PSO have been proposed to improve per-formance. As a result, a standard PSO, composed of clear principals, is needed as the baseline for comparison. Standard PSO (SPSO) provides a well defined version that fol-lows the common principals of PSO design. It is intended to be a milestone with simple and clear implementation for future comparison. So far, there have been three successive versions of standard PSO on the Particle Swarm Centeral1: SPSO 2006, SPSO 2007, and SPSO 2011. The underlying principals of these three algorithms are generally the same as all PSO variants. The exact formula and implementation are slightly different due to latest theoretical progress. A detailed description of SPSO 2011 [23] is given in the following paragraphs.
1https://www.particleswarm.info/
2.2.1 Initialization of the swarm
For a search space E with a given dimension D, E is confined by a set of minimum and maximum bounds in each dimension. The hyperparallelepid search space E can be formally defined as the Euclidean product of D real intervals [4]:
E =
⊗D
d=1
[mind, maxd].
For each position x within the multi-dimensional search space E, there exists a corre-sponding numerical value f (x), i.e. fitness. A swarm is composed of particles, which explore different positions in the search space to find the best corresponding fitness value.
At time t, each particle in the swarm possesses the following vectors with D coordinate:
• xi(t) is the position of the particle i at time t.
• vi(t) is the velocity of the particle i at time t.
• pi(t) is the previous best position the particle i had been to, at time t.
• li(t) is the best position of all the previous best positions in the neighborhood of particle i at time t.
Let U (mind, maxd) be a random number drawn from a uniform distribution within [mind, maxd], and Ni(t) be a set of neighbors of particle i at time t defined by the swarm topology. In SPSO 2011 [4], each particle is initialized with a random position and velocity defined as following:
xi(0) = U (mind, maxd),
vi(0) = U (mind− xi,d(0), maxd− xi,d(0)), pi(0) = xi(0),
li(0) = argminj∈Ni(0)(f (pj(0))).
The swarm size of SPSO 2011, denoted as S, is suggested as
S = 40,
yet it can also be defined by user [4].
2.2.2 Velocity update equations
Velocity update equations differs in different variations of PSO, since it is the core pro-cedure of optimization that determines the performance of PSO. To prevent premature convergence, SPSO follows a random permutation order to update the positions of par-ticles in the swarm [4]. The position of particle i at time t + 1 depends on the previous position and the current velocity:
xi(t + 1) = xi(t) + vi(t + 1).
The velocity of particle i at time t + 1 can be described as a combination of three vectors:
vi(t + 1) = wvi(t) + α(pi(t)− xi(t)) + β(li(t)− xi(t)),
where w mimics the momentum of the particle, and α, β describes how successive actions are effected by the personal previous best position and the neighborhood previous best position.
In SPSO 2011, the velocity update equations eliminates the coordinate dependency by creating a hypersphere according to xi, piand li, shown in Figure 2.3. The hypersphere is defined as:
Hi(Gi,||Gi− xi||),
with center Giand radius||Gi−xi||. When the personal best pi(t) is not the neighborhood previous best li(t), the center Gi is defined as:
Gi = 1
3(xi+ (xi+ c(pi− xi)) + (xi+ c(li− xi))) = xi+ c
3(pi+ li− 2xi).
Figure 2.3: Velocity update of SPSO 2011.
If the personal best is the best in the neighborhood, i.e. pi(t) = li(t), the center Gi is defined as:
Gi = 1
2(xi+ (xi+ c(pi− xi))) = xi+ c
2(pi− xi).
In both cases, the acceleration constant c is:
c = 1
2+ ln(2)≃ 1.193.
As shown in Figure 2.3, a random sample x′iis drawn from the hypershpere with uni-form random direction and uniuni-form radius:
r = U (0,||Gi− xi||).
Therefore, the velocity update equation and new position of SPSO 2011 is defined as:
vi(t + 1) = wvi(t) + x′i(t)− xi(t),
xi(t + 1) = xi(t) + vi(t + 1) = wvi(t) + x′i(t),
Figure 2.4: Different topologies used by PSO.
where the inertia weight is also:
w = 1
2 ln(2) ≃ 0.721.
As mentioned before, the search space is defined as a hyperparallelepid with confine-ment [mind, maxd] in each dimension d. Therefore, after calculating the new velocities and positions of particles, we have to go through confinement check to make sure all par-ticles stay inside the search space. There are multiple options to handle confinement in SPSO 2011. The following method described in [4] treat boundary as “wall”, so that all particles bounce back with half of previous velocity after they reach the border. For the new position xi,d(t + 1) of particle i in dimension d,
if (xi,d(t + 1) < mind), xi,d(t + 1) = mind,
vi,d(t + 1) =−0.5vi,d(t + 1).
if (xi,d(t + 1) > mind), xi,d(t + 1) = maxd,
vi,d(t + 1) =−0.5vi,d(t + 1).
2.2.3 The adaptive random topology
The swarm topology defines “who informs whom” to help distribute the information of optimum solution in the swarm. Multiple topologies shown in Figure 2.4 are adopted in different version of PSO. The random topology adopted in SPSO 2011 is updated under
Figure 2.5: Probability of different numbers of informants in the random topology.
two circumstances [4]:
• at the very beginning
• after an iteration with no improvement of the best known fitness value
For each particle pi in a swarm of n particles S = {p1, p2, ..., pn}, a good topology should allow each pito inform itself, along with a set of randomly selected particles from the other n− 1 particles [3]. We would like most of the particles to be informed by two to four particles in the topology. Some of the particles can have no informant except for itself. There should also exist a non-zero probability for a particle to be informed by all the other particles.
In order to construct a random topology that satisfy the desirable features mentioned above, Clerc proposed a method [3] that results in a distribution of the sum of n − 1 independent binary Bernoulli variables, shown in Figure 2.5. We use the distribution K = 3, meaning that each particle tells three other particles about their previous best fitness and position.
Figure 2.6: Comparison of discrete PDF continuous Gaussian kernel PDF.