Literature study
2.5 Large-scale SO problems
It was said earlier in this chapter that the focus in solving large-scale SO problems was on the search mechanisms used to explore the vast, and sometimes complex, solution spaces (Fu et al.,2000). It was also said in Chapter1that techniques capable of finding good enough solutions in reasonable computational time were favoured in practice.
These are the techniques that were alluded to by the author in Section2.2.2. In effect, many large-scale SO problems can be expensive to run in terms of time, money or resources (Amaran et al.,2014). The use of efficient techniques or search mechanisms in solving these problems is therefore key.
Though the literature has a number of techniques for solving large-scale SO prob-lems as discussed in Section 2.2.2, metaheuristics seem to be preferred in practice (Amaran et al. (2014), Fu (2002)). For more details on reasons why that is the case, the reader can refer to Fu (2002), where the researcher contrasts between the focus of researchers in the SO field and the techniques being adopted in practice. Nevertheless, it is widely known that many of the solution approaches that are specifically devised to handle large SO problems in the research community (seeAndradottir(1998) andHong
& Nelson(2009a)) are often limited in practice. A brief discussion on these methods is provided in this section.
Metaheuristic algorithms such as the genetic algorithm (GA) (briefly discussed in Section 3.2.1), the simulated annealing (SA), the tabu search (TS), cross-entropy method (CEM) and the ant colony optimisation (ACO), however, have been proven to be effective search mechanisms for many practical large-scale complex deterministic
problems, including those with multi-objectives. This logically makes them good can-didates for large-scale SO problems as well, despite some of their own limitations. In the next section, an attempt to formally define metaheuristics is made and the different metaheuristics mentioned above are discussed in more detail.
2.5.1 Metaheuristics
Metaheuristics are a class of approximate solution methods that have developed dra-matically since their inception in the early 1980s. They are designed to attack complex (deterministic) optimisation problems where classical heuristics and optimisation meth-ods have failed to be effective and efficient (Osman & Laporte,1997).
The literature has a number of formal definitions for the word metaheuristic (see for exampleBlum et al.(2008)). There does not seem to be a consensus on a singular definition for the word, possibly due to the generality of the metaheuristic concept.
Most definitions seem to include many important aspects of the workings of many metaheuristics. However, the more one learns about new metaheuristics (which there are a large number of), the more one realises how challenging it is to cover, in a single concise definition, what a metaheuristic is exactly. The following formal definition was thus selected as it tries not to be very specific and, in the author’s opinion, captures well the broadness of the concept (Dorigo et al.(2006)):
A metaheuristic is a set of algorithmic concepts that can be used to de-fine heuristic methods applicable to a wide set of different problems. In other words, a metaheuristic is a general-purpose algorithmic framework that can be applied to different optimisation problems with relatively few modifications.
Most metaheuristics are created to address, in an approximative way, determinis-tic optimisation problems for which no exact algorithms exist to solve the problems efficiently i.e. in a practical manner. Metaheuristics are able to do this because they are not problem structure-dependent (at least not as much as many methods in the research community), a characteristic that makes them robust heuristics according to Hong & Nelson(2009a). Rather, they rely on simple principles of nature that they are able to model in generic mathematical frameworks and apply to a variety of optimisa-tion problems. But why nature? According to Yang (2010), nature has evolved over
millions of years and has found perfect solutions to almost all the problems she met. We can thus learn the success of her problem-solving (ability) and develop nature-inspired heuristic algorithms.
Metaheuristics are generally globally convergent; meaning that if iterated long enough, under the right user-defined parameters, they may converge to the optimum (or optima, in the case of MOO problems). But in any case, they guarantee at least good solutions in a reasonable amount of computational time.
For the purpose of this study, the metaheuristics presented next are believed to be good candidates for the SO context, due to their effectiveness in solving deterministic problems. They are discussed in some detail, narratively and using pseudo-codes, and additional references are provided for more information. A brief discussion on other methods (non-metaheuristics) available for SO problems is also provided at the end of the section.
2.5.1.1 Simulated annealing
The simulated annealing algorithm is believed to be the oldest among the metaheuris-tics. According to Weise (2009), Kirkpatrick et al. (1983) pioneered the utilisation of SA for global optimisation in the early 1980s after being inspired by the work of Metropolis et al.(2002). The algorithm developed was initially applied to various com-binatorial (discrete) optimisation problems and since then, there have been extensive studies on the topic.
The SA algorithm mimics the annealing process in material science where a mate-rial (e.g. metal or glass) is strengthened through heat treatment that is followed by a carefully controlled cooling process. This allows the material to reach a stable state whereby its defects are removed and its strength is increased (Radin (1998), Bandy-opadhyay et al.(2008),Gendreau & Potvin (2010)).
Let X be the solution space and f : X → Y be an objective function defined on the solution space. The goal is, without loss of generality, to find a global minimum x∗∈ X such that f (x∗) ≤ f (x) for all (x ∈ X). Now, define N (x) as a set of solutions constituting the neighbourhood function for x. Associated with every solution or system design (x ∈ X), therefore, are neighbouring solutions N (x) that can be attained from x in a single iteration or a single move. Algorithm 4illustrates how the metaheuristic works (Eglese,1990).
Algorithm 4 Simulated annealing metaheuristic
1: Select an initial state x ∈ X, an initial temperature T > 0.
2: Set temperature change counter t = 0.
3: while n < N (t) do
17: Until stopping criterion is true.
Applications of SA are numerous and the range of problems the algorithm is able to solve is vast. The reader is referred toGendreau & Potvin(2010),Weise(2009) and Osman & Laporte (1997) for more detail. There are also many MOO variants of the SA algorithm. As an example,Bandyopadhyay et al.(2008) adapted the SA algorithm for MOO problems. The researchers proposed AMOSA, a simulated annealing-based multi-objective optimisation that finds a set of trade-off solutions.
2.5.1.2 Tabu search
According to Weise (2009), Glover (1986) initially introduced the basic ideas of tabu search and later in future works (Glover (1989), Glover (1990)), developed it into a general framework.
TS is one of many metaheuristics devised to overcome the limitations of traditional local search (LS) heuristics by using extended search strategies where traditional LS would normally stop. According to Blum et al. (2008), the basic idea of TS is the explicit use of search history, both to escape from local optima and to implement a
strategy for exploring the search space.
TS introduces into the LS scheme the concept of memory, in the form of the so-called tabu list (Blum et al., 2008) (a list that, momentarily, remembers a number of prohibited candidate solutions) to help avoid the local optima trap.
Suppose a function f (x) is to be minimised over some domain. TS-based algorithms can be generalised in two main steps, namely, the initialisation and the search step (Gendreau & Potvin,2010). Consider the following notation (Hertz & de Werra(1990), Gendreau & Potvin(2010)): x is the current or incumbent solution, x∗the best-known solution, f∗ the performance of x∗, N (x) the neighbourhood of x, x0 the admissible subset of N (x) i.e. non-tabu candidate solutions, and T the tabu list. Algorithm 5 illustrates how the metaheuristic works.
Algorithm 5 Tabu search metaheuristic
1: Initialisation:
2: Construct initial solution x0.
3: Set x∗← x0, f∗ ← f (x0), T ← ∅.
4: Search:
5: while termination condition is not met do
6: Select x = arg minx0∈N (x)[f (x0)].
7: if f (x) < f∗ then
8: f∗ ← f (x), x∗ ← x
9: end if
10: Record x in T and delete the oldest entry if necessary.
11: end while
According toHertz & de Werra(1990), TS is one of the most efficient metaheuristics for handling large optimisation problems. Hertz (1991) used TS to solve a large-scale timetabling problem. InToth & Vigo(2003), TS is used for a wide class of combinatorial optimisation problems whileCaballero et al. (2007) adapted a metaheuristic for multi-objective combinatorial optimisation problems based on TS to solve a multi-multi-objective location routing problem.
2.5.1.3 Cross-entropy method
The cross-entropy method was motivated by an adaptive algorithm for estimating prob-abilities of rare events in complex stochastic networks (Rubinstein,1997). It was soon
realised that a simple cross-entropy modification of Rubinstein (1997) could be used for solving difficult optimisation problems as well (Rubinstein,1999).
The CEM involves an iterative procedure where each iteration can be broken down into two phases (de Boer et al., 2005). Before the iterative procedure, however, the CEM associates with each optimisation problem a rare event estimation problem, the so-called associated stochastic problem (ASP) (Kroese et al., 2006). After the ASP is defined, the two iterative phases are as follows:
1. Generate a random data sample according to a specified mechanism.
2. Update the parameters of the random mechanism based on the data to produce a “better” sample in the next iteration.
So the algorithm first samples randomly from a chosen probability distribution over the space of decision variables. For each sample, a corresponding function evaluation is obtained. Based on the function values observed, a predefined percentile of the best samples is picked. A new distribution is then built around this “elite set” of points via a fitting method such as the maximum likelihood ratio estimator and the process is repeated. Algorithm 6illustrates how the metaheuristic works (Amaran et al.,2014).
Algorithm 6 Cross-entropy method metaheuristic
1: Requirement: θ, an initial set of parameters for a pre-chosen distribution p(x; θ) over the set of decision variables; s, a number of simulations to be performed; e, the number of elite samples representing the top δ percentile of the s samples.
2: while not converged or within simulation budget do
3: for i = 1 → s do
The CEM is often classified as a model-based metaheuristic. These are metaheuris-tics that attempt to build a probability distribution over the space of solutions and use it to guide the search process (Amaran et al.,2014).
In the literature, Alon et al. (2005) applied the CEM to the well-known buffer allocation problem in a SO context. Bekker & Aldrich (2011) adapted the CEM for MOO and validated the proposed algorithm to known test problems. InBekker(2012), the algorithm inBekker & Aldrich(2011) is integrated with the Arena software package and used to solve MOSO problems.
2.5.1.4 Ant colony optimisation
Inspired by the research done byDeneubourg et al. (1983) on real ants, Dorigo et al.
(1996) developed the ant colony optimisation algorithm (Weise,2009).
ACO is one of many swarm intelligence methods. Swarm intelligence is a relatively new approach to problem-solving that takes inspiration from the social behaviours of insects and of other animals (Dorigo et al.,2006).
ACO is a set of search algorithms that takes inspiration from the foraging behaviour of real ants. Most ant species’ way of foraging enables them to find the shortest paths between food sources and their nests. When foraging, a swarm of ants communicates indirectly in their local environment by the laying of scent chemicals or pheromone, creating trails that link the food source with their nest (Yang,2010). The first members of the colony that find their way to the food source do it randomly by trying different routes. Future members, however, are able to decide on what routes to follow thanks to the pheromone deposited by the members of the colony gone before them. The higher the pheromone concentration on a route, the higher the probability it will be selected by an ant. Experiment shows that as time progresses, the shortest route will start to have higher traffic density, causing a gradual increase on its pheromone concentration while the pheromone of the other routes experiencing low traffic evaporates progressively.
Eventually, the great majority of ants in the colony converge into a single route, the shortest one.
In ACO algorithms, artificial ants are stochastic solution construction procedures that build candidate solutions for the problem under consideration by exploiting ar-tificial pheromone information that is adapted based on the ants’ search experience (Gendreau & Potvin,2010). The pheromone trails are simulated via a parameterised
probabilistic model that is called the pheromone model. It consists of a set of model parameters whose values are called the pheromone values. These values act as the mem-ory that keeps track of the search process. The basic ingredient of ACO algorithms is a constructive heuristic that is used to, probabilistically, construct solutions using the pheromone values. Algorithm7illustrates how the metaheuristic works (Dorigo et al., 2006).
Algorithm 7 Ant colony optimisation metaheuristic
1: Set parameters.
2: Initialise pheromone trails.
3: while termination condition is not met do
4: Construct ant solutions.
5: Apply local search (optimal).
6: Update pheromones.
7: end while
ACO algorithms are often classified as both model- and population-based meta-heuristics; population-based because they use a set of solutions rather than a single solution at each iteration. In the literature, ACO is mostly used for discrete opti-misation problems, though variants of the metaheuristic for continuous problems also exist. InMerkle et al.(2002), the researchers use ACO to solve a resource-constrained scheduling problem whereasBella & McMullen(2004) use a variant of the algorithm to solve a vehicle-routing problem. Efforts have also been made to adapt ACO for MOO problems and variants of the algorithm for this purpose can be found inGendreau &
Potvin (2010).
2.5.2 Other search mechanisms
It was mentioned in Section 2.5.1 that metaheuristics are generally devised for de-terministic problems. There are other search mechanisms, however, that have been specifically designed for SO problems. The main distinguishing aspect of these tech-niques is that, contrary to metaheuristics, they all have a “noise handling strategy”
in the form of simulation allocation rules (SAR) embedded in their algorithmic proce-dures. Despite such an advantage, nonetheless, most of these algorithms are generally less robust than metaheuristics. Unlike metaheuristics that can be easily adapted to
different problems (e.g. problems with decision variables with different nature), here algorithms can be too problem-specific, a factor that often limits their use in practice.
There is thus a clear distinction between SO methods for continuous problems and SO methods for discrete problems here. In the literature, these methods are better known as COvS and DOvS (as seen before in Section2.2.2) algorithms.
2.5.2.1 COvS algorithms
COvS methods have been researched intensively in the past, arguably more than DOvS.
COvS algorithms include, among others, the stochastic approximation, the gradient estimation and the sample path optimisation methods. For more detail, the reader is referred to Andradottir (1998), Tekin & Sabuncuoglu (2004) and Hong & Nelson (2009a), which give reviews on the topic.
2.5.2.2 DOvS algorithms
In recent years, research on DOvS methods have been trying to close the gap on some of the advances made in the COvS field. There are many DOvS algorithms that can be found in the literature (Andradottir(1998),Hong & Nelson(2009a),Yoon
& Bekker (2017)). Among them is the Convergent Optimization via Most Promising Area Stochastic Search (COMPASS) algorithm due toHong & Nelson(2009b).
COMPASS solves DOvS problems by implementing an adaptive region called the most promising area, where preferable solutions can be found with high probability. It has been shown that, in both constrained and unconstrained search space, the algorithm asymptotically converges to one of the local optima (Li et al., 2015). Li et al. (2015) also successfully adapted the algorithm for MOO problems with the Pareto approach.
Algorithm8 illustrates how the method works (Hong & Nelson,2009a).
Despite the “non-metaheuristics” having some statistical features in the form of their SAR, these are often not as efficient nor as effective as those used in R&S pro-cedures. There are therefore efforts being made, in this particular field, to further improve the noise-handling ability of these large-scale SO algorithms. For example,Xu et al.(2010) suggested that OCBA be used as the SAR for COMPASS to improve the efficiency of the algorithm. Efforts to improve SO methods are thus continuously being made to make the algorithms more effective.
Algorithm 8 COMPASS algorithm
1: Build the most promising area in each iteration around the current sample best solution based on geometry.
2: Sample new solutions from the most promising area in each iteration.
3: Simulate new solutions and solutions that define the most promising area a little bit more.
4: Calculate the cumulative sample average for each active solution, and choose the solution with the best cumulative sample average. Go to Step 1.
Metaheuristics are also continuously being improved in various ways. One way to make them more effective is to combine them with other algorithms in order to benefit from the synergy. This concept is known as hybridisation and is discussed in the following section.