[633]. The path selection procedure may be viewed as a constrained local search where the set of candidate solutions is constrained and the neighbor selection strategy is gener-alized. For nonbinary representations, measuring the distance between solutions may be difficult.
3.6 SWARM INTELLIGENCE
Algorithms inspired from the collective behavior of species such as ants, bees, wasps, termite, fish, and birds are referred as swarm intelligence algorithms [85,617,672]. Swarm intelligence originated from the social behavior of those species that com-pete for foods. The main characteristics of swarm-intelligence-based algorithms are particles are simple and nonsophisticated agents, theycooperateby an indirect com-munication medium, and do movements in the decision space14.
Among the most successful swarm intelligence inspired optimization algorithms are ant colony and particle swarm optimization.
3.6.1 Ant Colony Optimization Algorithms
The basic idea in ant colony optimization algorithms (ACO) is to imitate the co-operative behavior of real ants to solve optimization problems. ACO metaheuristics have been proposed by M. Dorigo [215]. They can be seen as multiagent systems in which each single agent is inspired by the behavior of a real ant. Traditionally, ACO have been applied to combinatorial optimization problems and they have achieved widespread success in solving different problems (e.g., scheduling, routing, assign-ment) [219].
The main interest of real ant’s behavior is that simple ants using collective be-havior perform complex tasks such as transportation of food and finding shortest paths to the food sources. ACO algorithms mimic the principle that using very sim-ple communication mechanism, an ant colony is able to find the shortest path be-tween two points. Figure 3.32 illustrates an experiment done by Goss et al. [338] with a real colony of Argentine ants (Iridomyrmex humilis). Notice that those ants cannot see very well. The colony has access to a food source linked by two paths to the colony’s nest. During their trips, a chemical trail (pheromone) is left on the ground. The pheromone is an olfactive and volatile substance. The role of this trail is to guide the other ants toward the target point. The larger the amount of pheromone on a particular path, the larger the probability that the ants will select the path. For a given ant, the path is chosen according to the smelt quantity of pheromone.
FIGURE 3.32 Inspiration from an ant colony searching an optimal path between the food and the nest.
Furthermore, this chemical substance has a decreasing action over time (evap-oration process) and the quantity left by one ant depends on the amount of food (reinforcement process). As shown in Fig. 3.32, when facing an obstacle, there is an equal probability for every ant to choose the left or right path. As the left trail is shorter than the right one and so requires less travel time, the ant will end up leaving a higher level of pheromone. The more the ants take the left path, the higher the pheromone trail. Hence, there is an emergence of the shortest path. This fact will be increased by the evaporation stage. This indirect form of cooperation is known asstigmergy.
Algorithm 3.12 presents the template algorithm for ACO. First, the pheromone information is initialized. The algorithm is mainly composed of two iterated steps: solution construction and pheromone update.
Algorithm 3.12 Template of the ACO.
Initialize the pheromone trails ; Repeat
Foreach ant Do
Solution construction using the pheromone trail ; Update the pheromone trails:
Evaporation ; Reinforcement ; UntilStopping criteria
Output:Best solution found or a set of solutions.
• Solution construction:The construction of solutions is done according to a
solution components to partial ones until a complete solution is derived (see Section 1.3.4). The target optimization problem can be seen as a decision graph (or construction graph) where an ant will construct a path. Usually, this iterative process takes into account
– Pheromone trails:Indeed, the pheromone trails memorize the characteristics of “good” generated solutions, which will guide the construction of new solu-tions by the ants. The pheromone trails change dynamically during the search to reflect the acquired knowledge. It represents the memory of the whole ant search process.
– Problem-dependent heuristic information:A problem-specific information gives more hints to ants in their decisions to construct solutions.
• Pheromone update: The update of the pheromone is carried out using the
generated solutions. A global pheromone updating rule is applied in two phases:
– An evaporation phase where the pheromone trail decreases automatically. Each pheromone value is reduced by a fixed proportion:
τij =(1−ρ)τij, ∀i, j∈[1, n]
where ρ∈]0,1] represents the reduction rate of the pheromone. The goal of the evaporation is to avoid for all ants a premature convergence toward “good” solutions and then to encourage the diversification in the search space (exploration).
– A reinforcement phase where the pheromone trail is updated according to the generated solutions. Three different strategies may be applied [155]:
◦ Online step-by-step pheromone update:The pheromone trailτij is
up-dated by an ant at each step of the solution construction [217].
◦ Online delayed pheromone update:The pheromone update ofτis applied
once an ant generates a complete solution [527]. For instance, in the ACS, each ant will update the pheromone information with a value that is propor-tional to the quality of the solution found. The better the solution found, the more accumulated the pheromone.
◦ Off-line pheromone update:The pheromone train update is applied once
allants generate a complete solution. This is the most popular approach where different strategies can be used [541]:
valueis added:
τiπ∗(i)=τiπ∗(i)+, ∀i∈[1, n]
■ Rank-based pheromone update:The bestkant solutions are allowed to update the pheromone with an amount depending on the ranking of the solutions [95].
■ Worst pheromone update:The ant generating the worst solution will decrease the pheromone trails related to the components of the solu-tion [154].
■ Elitist pheromone update:The best solution found so far will update the pheromone to enforce an intensification of the search [728]. A minimum (resp. maximum) value for the pheromone is defined such as each choice has a minimum (resp. maximum) probability to be selected.
Theoretical convergence analysis of ACO algorithms may be found in Ref. [348]. The first convergence results were presented in Ref. [727]. The author proves conver-gence with probability 1−ǫto an optimal solution of a particular ACO algorithm, the graph-based ant system (GBAS). In Ref. [727], a convergence result has been obtained for a more general class of ACO algorithms among the most efficient in practice, the class of algorithms that uses a lower boundτminto all pheromone val-ues. Indeed, the lower bound allows the probability to generate any solution to be different from zero. For a comprehensive survey of theoretical results for ant colony optimization, the reader may refer to Ref. [216].
In addition to classical and common search components in metaheuristics (e.g., representation), the main issue in designing an ACO is the determination of the following:
• Pheromone information:The pheromone model represents the central
compo-nent of ACO algorithms. It consists in defining a vector of model parametersτ calledpheromone trail parameters. The pheromone valuesτi∈τshould reflect the relevant information in the construction of the solution for a given problem. They are usually associated with the components of a solution.
• Solution construction:In the solution construction, the main question is
con-cerned with the definition of the local heuristic to be used in guiding the search in addition to the pheromone. The ACO metaphor is easy to adapt in solving problems where relatively efficient greedy algorithms already exist.
• Pheromone update: Mainly the reinforcement learning strategy for the
pheromone information has to be defined.
Initialize the pheromone information ; Repeat
Foreach ant Do
Solution construction using the pheromone trails:
S= {1,2, . . . , n}/∗Set of potentially selected cities∗/
Random selection of the initial cityi; Repeat
Select new cityjwith probabilitypij=
τα ij×η
β ij
k∈S τα
ik×η β ik ;
S=S− {j};i=j; UntilS= ∅
End For
Update the pheromone trail: Fori, j∈[1, n] Do
τij=(1−ρ)τij/∗Evaporation∗/; Fori∈[1, n] Do
τiπ(i)=τiπ(i)+ /∗π: best found solution∗/; UntilStopping criteria
Output:Best solution found or a set of solutions.
• Pheromone trails:A pheromoneτij will be associated with each edge (i, j) of the graphG. The pheromone information can be represented by ann×nmatrixτ where each elementτijof the matrix expresses the desirability to have the edge (i, j) in the tour. The pheromone matrix is generally initialized by the same values15.
During the search, the pheromone will be updated to estimate the utility of any edge of the graph.
• Solution construction:Each ant will construct a tour in a stochastic way. Given an initial arbitrary cityi, an ant will select the next cityjwith the probability
pij=
τij
k∈S
τik
, ∀j∈S
where the setSrepresents the not yet visited solutions of the graphG. The ants may use a randomly selected initial city in the construction phase.
The additional problem-dependent heuristic is defined by considering the values
ηijequal to 1/dijwheredijrepresents the distance between the citiesiandj. The
higher the heuristic valueηij, the shorter the distancedijbetween citiesiandj.
Computing the decision transition probabilities,pijis performed as follows:
pij=
τα
ij ×η β ij
k∈S
τα
ik×η β ik
, ∀j∈S
15The values ofτ
where α and β are parameters representing the relative influence of the pheromone values and the problem-dependent heuristic values. If α=0, the ACO algorithm will be similar to a stochastic greedy algorithm in which the closest cities are more likely selected. Ifβ=0, only the pheromone trails will guide the search. In this case, a rapid emergence ofstagnationmay occur where all ants will construct the same suboptimal tour. Hence, a good trade-off must be found in using those two kinds of information.
Then, the pheromone update procedure has to be specified. For instance, each ant will increment the pheromone associated with the selected edges in a manner that is proportional to the quality of the obtained tourπ:
τiπ(i)=τiπ(i)+, ∀i∈[1, n]
where=1/f(π). Then, good tours will emerge as the result of the cooperation be-tween ants through the pheromone trails.
The classical evaporation procedure is applied for the pheromone trails. For each edge, its pheromoneτijwill evaporate as follows:
τij=(1−ρ)τij, ∀i, j∈[1, n]
whereρ∈]0,1] represents the reduction rate of the pheromone.
Compared to other metaheuristics, an important aspect in ACO is the differ-ential path length (DPL) effect of ants. The decentralized and asynchronous na-ture of ants is crucial in solving distributed problems where there is no global view of the objective function. Decisions must be taken under a local view of the problem.
Example 3.19 Solving distributed and dynamic problems with ACO. This ex-ample illustrates the efficient way ACO can solve distributed network problems. For instance, in network routing, the problem deals with building and using rout-ing tables to route the data traffic so that a given measure of network performance (e.g., throughput) is maximized. The network can be modeled by a weighted graph
G=(V, E) where the vertices of the setV represent the network nodes (processing/ queuing/forwarding capabilities) and the edges of the graph represent the links of the network (directional transmission system). The weight associated with the edges is de-fined by the physical properties of the link (static measure) and the crossing traffic flow (dynamic measure). Given a network application with data flow from any source to any destination, the problem consists in constructing the routing tablesRifor each node of
the network. The routing tableRi=rijdat the nodeiand for message destined todwill
forward the message to the neighbor nodej.
two properties characterize this problem:
• Distribution:Each ant will solve a partial problem. The global problem is to build all paths defined by (f, d) whereas an ant will find a single path of the whole problem.
• Dynamicity:The weights associated with the edges are not static and can change dynamically according to the traffic.
ACO algorithms can be designed for this problem as follows [108]: ants are launched from each node of the network taking into account the traffic patterns. Each ant will search for the minimum cost path from its assigned starting nodef to its assigned destination noded. When an antkis in the nodei, it will select the next neighborjaccording to a probabilistic transition rule that depends on the pheromone trails associated with the edges and a heuristic value. A heuristic valueηij, which depends on the destination node
d, is associated with each edge (i, j) as given below.
ηij=1−
qij
l∈Ni
qil
whereqij is the length of the message queue in the link connecting nodei with its
neighborj.
The solution construction is based on the routing tableAi=[aijd(t)] at each nodei.
This routing table is a composition of the pheromone trailsτijdand the heuristic values
ηij:
aijd(t)=
wτijd(t)+(1−w)ηij
w+ 1−w
|Ni|−1
wherej∈Ni,dis the destination node,w∈[0,1] a weighting factor, and the
denomi-nator is a normalization factor. Then, the antklocated in the nodeifor the destination
dwill choose a not yet visited nodejwith the probability
pkijd(t)=
aijd(t) ifj /∈Mk
0 ifj∈Mk
whereMkis the set of already visited nodes. If all nodes are already visited by the ant,
a neighbor nodej∈Niis chosen randomly.
The reinforcement procedure of the pheromone trails works as follows: once an antk
completes a path, it updates the pheromone trails by an amountτk(t) that is proportional to the quality of the builded path. So the ant moves back to the source node by using the same reverse path. For each linklij, the ant increases the pheromone trail intensity by
τk(t).
Then, the pheromone values of all the output connections of the same nodeievaporate.
τijd(t)=
τijd(t)
(1+τk(t)), ∀j∈Ni
whereNiis the set of neighbors of the nodei.
3.6.1.1 ACO for Continuous Optimization Problems ACO algorithms
have been extended to deal with continuous optimization problems. The main is-sue in adapting the ACO metaheuristic is to model a continuous nest neighborhood by a discrete structure or change the pheromone model by a continuous one. Indeed, in ACO for combinatorial problems, the pheromone trails are associated with a finite set of values related to the decisions that the ants make. This is not possible in the continuous case. ACO was first adapted for continuous optimization in Ref. [75]. Further attempts to adapt ACO for continuous optimization were reported in many works [224,708,778]. Most of these approaches do not follow the original ACO frame-work [709]. The fundamental idea is the shift from using a discrete probability dis-tribution to using a continuous one (e.g., probability density function [709]).
Initializing the numerous parameters of ACO algorithms is critical. Table 3.6 summarizes the main parameters of a basic ACO algorithm. Some sensitive parame-ters such asαandβcan be adjusted in a dynamic or an adaptive manner to deal with the classical trade-off between intensification and diversification during the search [635]. The optimal values for the parametersαandβare very sensitive to the target problem. The number of ants is not a critical parameter. Its value will mainly depend on the computational capacity of the experiments.
3.6.2 Particle Swarm Optimization
Particle swarm optimization is another stochastic population-based metaheuristic in-spired from swarm intelligence [459]. It mimics the social behavior of natural organ-isms such as bird flocking and fish schooling to find a place with enough food. Indeed, in those swarms, a coordinated behavior using local movements emerges without any central control. Originally, PSO has been successfully designed for continuous opti-mization problems. Its first application to optiopti-mization problems has been proposed in Ref. [457].
TABLE 3.6 Parameters of the ACO Algorithm
Parameter Role Practical Values
α Pheromone influence –
β Heuristic influence –
ρ Evaporation rate [0.01,0.2]
