MESTRADO EM CIˆENCIA DA COMPUTAC¸ ˜AO
TABAJARA KRAUSBURG RODRIGUES
CONSTRAINED COALITION FORMATION AMONG HETEROGENEOUS AGENTS FOR THE MULTI-AGENT PROGRAMMING CONTEST
Porto Alegre 2018
CONSTRAINED COALITION
FORMATION AMONG
HETEROGENEOUS AGENTS
FOR THE MULTI-AGENT
PROGRAMMING CONTEST
TABAJARA KRAUSBURG RODRIGUES
Thesis submitted to the Pontifical Catholic University of Rio Grande do Sul in partial fullfillment of the requirements for the degree of Master in Computer Science.
Advisor: Prof. Dr. Rafael Heitor Bordini
Porto Alegre 2018
This Dissertation has been submitted in partial fulfillment of the requirements for the degree of Master of Computer Science, of the Graduate Program in Computer Science, School of Technology of the Pontifícia Universidade Católica do Rio Grande do Sul.
Sanctioned on March 26, 2018.
COMMITTEE MEMBERS:
Prof. Dr. Luís Alvaro de Lima Silva (UFSM)
Prof. Dr. Renata Vieira (PPGCC/PUCRS)
HETEROGÊNEOS PARA A COMPETIÇÃO DE PROGRAMAÇÃO
MULTIAGENTE
RESUMO
Esta dissertação apresenta um estudo sobre formação de coalizões entre agentes heterogê-neos para a competição de programação multiagente de 2017. Foi investigado e aplicado a formação de estruturas de coalizões entre agentes para resolver problemas logísticos simulados sobre o mapa de uma cidade real. A fim de atingir o objetivo deste trabalho, foram integrados algoritmos forma-dores de coalizões na plataforma JaCaMo por meio de um artefato CArtAgO chamado CFArtefact. Foi utilizada a implementação provida pelo time SMART-JaCaMo (time participante da competição multiagente), para experimentar a formação de coalizões na competição. Três abordagens foram ava-liadas no domínio da competição em diferentes configurações. A primeira abordagem utiliza somente alocação de tarefas para resolver o problema. A segunda e a terceira abordagem utilizam a técnica de formação de coalizões anteriormente à alocação de tarefas; dentre estas abordagens, uma utiliza um algoritimo ótimo para resolver o problema e a outra um heurístico. As análises dos experimentos realizados mostram que algoritimos formadores de coalizões podem melhorar a performance do time participante da competição quando a taxa de trabalhos gerados pelo simulador é baixa. Entretanto, conforme a taxa de trabalhos aumenta, a abordagem que realiza somente alocação de tarefas obtém um desempenho melhor quando comparada as demais. Mesmo a abordagem heurística tem desem-penho próximo à abordagem ótima para coalizões. Desta forma, é possível concluir que formação de coalizões possui grande valia para balancear os agentes para um conjunto de trabalhos que precisa ser completado.
Palavras-Chave: formação de coalizões, sistemas multiagents, JaCaMo, competição de programa-ção multiagente.
This work focuses on coalition formation among heterogeneous agents for the 2017 multi-agent programming contest. An multi-agent is a computer system that is capable of independent action to achieve its goals. In order to increase the effectiveness of the agents, we can organise them into coalitions, in which the agents collaborate with each other to achieve individual or common goals. We investigate and apply coalition structure generation (the first activity of the coalition formation process) in simulated scenarios, specifically the 2017 contest scenario, where the agents forming a competing team cooperate to solve logistic problems simulated on the map of a real city. In order to achieve our goal, we integrate coalition formation algorithms into the JaCaMo platform by means of a CArtAgO artefact, named CFArtefact. We use the implementation of the SMART-JaCaMo team for experimenting with the coalition formation approach in the contest scenario. We experiment on three approaches in the contest domain with different configurations. In the first, we use only a task-allocation mechanism, while the other approaches use an optimal coalition formation algorithm and a heuristic coalition formation algorithm. We conducted several experiments to compare the advantages of each approach. Our results show that coalition formation algorithms can improve the performance of a participating team when dealing with low job rates (i.e., how quickly new jobs are created by the simulation). However, as we increase the job rate, the approach using only task allocation has better performance. Even a heuristic coalition formation approach has close performance to the optimal one in that case. Coalition formation can play an important role when we aim to balance each group of agents to accomplish some particular goal given a larger team of cooperating agents.
2.1 View of an agent [Woo09]. . . 20
2.2 Coalition (A) and Team (B) organisational paradigms [HL04]. . . 21
2.3 BDI framework adapted from [Woo13]. . . 22
2.4 Types of AgentSpeak terms in Jason [BHW07]. . . 25
2.5 CArtAgO concepts in the Unified Modelling Language notation [RPV11]. . . 27
2.6 The components of the EIS model [BHD11]. . . 27
2.7 MOISE scheme for a soccer team [HSO07]. . . 29
2.8 Dimensions in JaCaMo [BBH+13]. . . 30
2.9 The cooperation lifecycle, adapted from [Woo09]. . . 32
2.10 Representations of coalition structure, adapted from [RMWJ15]. . . 33
2.11 The coalition structure graph for four agents, adapted from [SLA+99]. . . . 34
2.12 The integer partition graph for four agents, adapted from [MRE+15]. . . . 34
2.13 Comparison of search space representation between CSG and IPG in an example of four agents for ODP and IDP algorithms. . . 36
2.14 Root expansion from agent 1 in SlyCE, reproduced from [VRJ12]. . . 39
2.15 Graph representing the space of coalition structures, reproduced from [BFC+14]. . . 41
2.16 An example of transformation algorithm given A = {1, 2, 3, 4, 5, 6, 7, 8}, P = {1, 5, 8}{2, 5, 7}{5, 7, 8}, and N = {1, 2, 3}{2, 3, 5}, reproduced from [RME+11]. . 44
2.17 The order in which the sub-spaces are searched, reproduced from [RME+11]. . . 45
2.18 An example of C-Link’s execution instance for a set of four agents, reproduced from [FBBR16]. . . 47
3.1 Map of Paris for the 2017 multi-agent programming contest. . . 49
4.1 The overall relationship between the CFArtefact and the coalition formation algo-rithms. The agents use the artefact to send/get information to/from the algoalgo-rithms. . 57
5.1 Number of completed jobs after 1000 steps considering different job rates. We depict the average of completed jobs for 50 simulations. . . 70
5.2 Money improvement through steps of the round considering different facilities den-sity. We fixed the job rate to 10% of the maximum capacity and collected the average money at each step of 50 simulation repetitions. . . 72
5.3 Performance of O-CF and TA-1 when the job rate is 10% of its maximum capacity. We depict the number of completed jobs after 1000 steps for 50 simulations. . . 74
5.4 The frequency of steps interval between jobs for the DC algorithm simulation. It depicts the mean of 50 simulations. . . 75
task allocation (TA Only (3 jobs)). The job rate is fixed in 25% of its maximum capacity. We depict the money average, received jobs and completed jobs for 50 simulations. . . 104 D.3 Performance of three teams using different approaches. Two of them using coalition
formation algorithms before allocating tasks (CLINK and DC) and one with only task allocation (TA Only (3 jobs)). The job rate is fixed in 50% of its maximum capacity. We depict the money average, received jobs and completed jobs for 50 simulations. . . 105 D.4 Performance of three teams using different approaches. Two of them using coalition
formation algorithms before allocating tasks (CLINK and DC) and one with only task allocation (TA Only (3 jobs)). The job rate is fixed in 75% of its maximum capacity. We depict the money average, received jobs and completed jobs for 50 simulations. . . 106 D.5 Performance of three teams using different approaches. Two of them using coalition
formation algorithms before allocating tasks (CLINK and DC) and one with only task allocation (TA Only (3 jobs)). The job rate is fixed in 100% of its maximum capacity. We depict the money average, received jobs and completed jobs for 50 simulations. . . 107
3.1 Characteristics of each agent type. . . 49 5.1 Total number of shops, workshops, and storage on the contest city map considering
AOP – Agent-Oriented Programming API – Application Programming Interface APL – Agent Programming Language BDI – Belief Desire Intention
BRF – Belief Revision Function
CCF – Constrained Coalition Formation
CDPS – Cooperative Distributed Problem Solving CFG – Characteristic Function Games
CFSS – Coalition Formation with Sparse Synergies CNP – Contract Net Protocol
CSG – Coalition Structure Graph CSP – Complete Set Partitioning
DCOP – Distributed Constraint Optimization Problem EIS – Environment Interface Standard
EV – Electric Vehicle GL – Gain Link
GPS – Global Positioning System
GRASP – Greedy Randomized Adaptive Search Procedures HAC – Human-Agent Collectives
IDE – Integrated Development Environment JSON – JavaScript Object Notation
KA – Knowledge Area MAS – Multi-Agent System MC – Marginal Contribution
NTU – Non-Transferable Utility Game OSM – Open Street Map
PFG – Partition Function Games PRS – Procedural Reasoning System SCG – Synergy Coalition Groups
SCSP – Semiring Constraint Satisfaction Problem TU – Transferable Utility Game
A – the set of agents . . . 31
v – the characteristic function . . . 31
Ci– the ithsubset of a set . . . 31
CS – a coalition structure . . . 31
ΠA– the space of all partitions over set A . . . . 31
ad i – the ithagent in A of type drone . . . 31
O – the big O notation . . . 35
CS – the set of feasible coalition structures . . . 38
G – the characteristic function game . . . 38
φ – a logical formula . . . 38
BA– the set of Boolean variables over the set A . . . . 38
bi – the ithBoolean variable in B . . . 38
C – the set of feasible coalitions . . . 38
S – the set of size constraints . . . 38
P – the set of positive constraints . . . 38
P – the set of agents that forms a positive constraint . . . 38
N – the set of negative constraints . . . 38
N – the set of agents that forms a negative constraint . . . 38
E – the set of edges of a graph . . . 39
G – a graph . . . 39
O – the set of pivotal agents . . . 42
I – the set of skills . . . 43
Iai – the set of skill of the i thagent . . . . 43
τi– the ithtask . . . 43
Li– the ithlist containing subsets of the set A . . . 44
lf – the linkage function . . . 46
Li– the ithLevel in a graph . . . 46
P L(i)– the ithiteration of the partition linkage table . . . . 46
A – the set of agents in the CFArtefact . . . 58
mcN – the negative rule of one marginal contribution rule . . . 58
mcP – the positive rule of one marginal contribution rule . . . 58
2.1 Example of AgentSpeak(L) plan implemented in Jason. . . 24 4.1 Jason listing for event manipulation within the coalition formation module. . . 61 4.2 Jason listing for agent’s marginal contributions. . . 65
2 BACKGROUND . . . 20
2.1 MULTI-AGENT SYSTEM . . . 20
2.1.1 AGENT-ORIENTED PROGRAMMING LANGUAGES . . . 23
2.1.2 ENVIRONMENT-ORIENTED PROGRAMMING . . . 26
2.1.3 ORGANISATION-ORIENTED PROGRAMMING . . . 28
2.1.4 MULTI-AGENT SYSTEMS DEVELOPMENT PLATFORMS . . . 29
2.2 COALITION FORMATION . . . 30
2.2.1 SEARCH SPACE REPRESENTATION . . . 33
2.2.2 ALGORITHMS FOR COALITION STRUCTURE GENERATION . . . 34
2.2.3 ALGORITHMS FOR CONSTRAINED COALITION STRUCTURE GENERATION . 38 2.2.4 CHARACTERISTIC FUNCTION REPRESENTATION FOR CSG . . . 42
2.2.5 THE DC ALGORITHM . . . 43
2.2.6 THE C-LINK ALGORITHM . . . 45
2.3 TASK ALLOCATION . . . 46
3 PROBLEM DOMAIN . . . 48
3.1 MULTI-AGENT PROGRAMMING CONTEST 2017 . . . 48
3.2 SMART-JACAMO TEAM . . . 52
4 A COALITION-FORMATION APPROACH TO THE MAPC PROBLEM . . . 56
4.1 COALITION FORMATION ALGORITHMS . . . 56
4.2 INTEGRATION OF THE COALITION FORMATION ALGORITHMS INTO THE JACAMO PLATFORM . . . 57
4.3 COALITION FORMATION FOR THE SMART-JACAMO TEAM . . . 59
4.4 CHARACTERISTIC FUNCTION FOR THE CONTEST SCENARIO . . . 62
5 EXPERIMENT RESULTS AND ANALYSIS . . . 67
5.3 DENSITY OF FACILITIES PER QUADRANT OF THE MAP . . . 71
5.4 IS IT WORTH WAITING FOR JOBS? . . . 73
6 RELATED WORK . . . 77
6.1 COALITION FORMATION DOMAIN OF APPLICATIONS . . . 77
6.2 TASK ALLOCATION CONSIDERING COALITIONS . . . 79
7 CONCLUSIONS AND FUTURE WORK . . . 82
REFERENCES . . . 84
APPENDIX A – DC Algorithm . . . 93
APPENDIX B – C-Link Algorithm . . . 97
APPENDIX C – JSON Configuration File to the Multi-agent Programming Contest Server . . . 98
requirements. When more than one agent is acting on the same environment we have a multi-agent system, which consists of a number of agents capable of interacting with each other by exchanging messages [Woo09].
In order to increase the effectiveness of the agents, we can organise them into groups, in which the agents collaborate with each other in order to achieve individual, common, or global (i.e., system level) goals. Such organisation of agents was studied by Horling and Lesser in [HL04]; their research brings out several organisational structures that are applicable to multi-agent systems, in-cluding coalitions. A coalition is a short-lived and goal-directed structure, in which the agents join forces to achieve a goal [HL04]. Coalitions are also used in the context of cooperative game the-ory [Woo09]. From this point of view a coalition is a set of agents who may or may not work together. Each coalition has the ability to obtain a certain utility represented by a numeric value. We use the coalition formation process to form disjoint coalitions over a set of agents (virtual agents). We try to maximise the overall value of all coalitions in the environment. In this sense, we are able to form the best set of coalitions. Coalition formation approaches have been applied to sev-eral applications such as in e-marketplaces [MTMW09], ride-sharing [BFCR17], collective electric-ity consumption shifting [AC17], cognitive radio network [THI17], content uploading in cellular systems [MOA+16], supply chain [BJKK17], smart grids [dORBB14] and in task allocation ap-proaches [PS15, IF16, MSG+17, AHG17].
Although many applications have used coalition formation to address real-world problems, they do not evaluate their techniques on coalition formation against other kinds of approaches. This is important in order to state if the coalition formation is the most suitable technique for the problem. In this work, we evaluate the coalition structure generation on the 2017 MAPC [ADFng]. The MAPC is an annual international event that aims to stimulate research in the field of programming multi-agent systems. We selected one of its participating teams, the chosen one is the SMART-JaCaMo team [CKB+ng]. We studied how to apply coalition formation techniques to its implementation and
we evaluate the performance of the coalition formation on the MAPC scenario. In the MAPC domain we have heterogeneous agents that must deliver jobs posted by a server; to do so they need to collect items in various locations on a real city map and assembled the parts before delivery. The SMART-JaCaMo team uses task allocation approach to address this problem and we include the formation of coalitions as a previous step to task allocation.
The coalition formation process is integrated into the JaCaMo platform [BBH+13, BHR16] in a generic way, so it can be used in other projects too. We use two algorithms of coalition formation with different designed purposes. The first one is an optimal coalition formation algorithm that takes
into account constraints; it is named the DC algorithm and was proposed by Rahwan et al. [RME+11]. The second algorithm was proposed by Farinelli et. al. [FBRZ13] and it consists of a heuristic algo-rithm for coalition formation based on clustering algoalgo-rithms; it is named the C-Link algoalgo-rithm. These algorithms for coalition formation are encapsulated into a CArtAgO artefact [RPV11] and the con-straints, agents, tasks, and so on, can be added by means of operations on it. The coalitions are formed and stored in agents’ belief bases through observable properties of the coalition formation artefact. This approach adds a great value to the JaCaMo platform, in the sense that, coalition formation al-gorithms can be applied in various different JaCaMo domain applications. It can benefit researchers, lecturers, and students in their activities.
We perform several experiments focusing on the performance of the team while working on jobs. We run the coalition structure generation algorithms in order to get the best partition of agents to work on the jobs; this way we are able to balance the agents for each job. When we take into account a set of jobs, we reason about all the jobs and how we can accomplish all of them instead of allocating the best agents to only one job. This is particularly important when jobs sporadically appear in the environment, so concentrating all of agent’s effort in one job could be a bad strategy.
1.1 Motivation
The motivation behind this research is the desire to bring out theories and algorithms one step closer to real-world applications. Even though usually it is not possible to concretely implement agent techniques in real scenarios, any initial work on such direction is important in order to perceive and report the difficulties. Such initial steps can be in terms of simulations of real-world scenarios in controlled environments. Moreover, the JaCaMo platform is ideal for this research because it fills the gap in using multi-agent systems into simulated environments as well as into real environments, and it has been widely used in teaching multi-agent systems. Another motivational issue is to increase the use of multi-agent systems which have great potential for both academic and industrial applications.
1.2 Goal
Our goal is to investigate and apply coalition structure generation in simulated scenarios, specifically on the 2017 multi-agent programming contest. In order to achieve our proposed goal, we define several objectives for our research development.
• We propose an integration of coalition formation algorithms into the JaCaMo platform by means of a CArtAgO artefact, named CFArtefact;
• Implement the following algorithms for coalition structure generation into CFArtefact:
1.3 Main Contributions
We provide an implementation to integrate coalition formation algorithms into JaCaMo platform. Such integration is capable of handling constraints, types, skills, tasks, and agents’ con-tributions. We have firstly integrated the algorithms proposed in [RME+11] and [FBRZ13]. We experiment the coalition formation technique on the 2017 MAPC domain, in which jobs are sent to agents in a discrete environment based on a real city, and we show in which situations it is better to use the coalition formation approach.
1.4 Dissertation Outline
This document is organised as follows. In Section 2, we describe the background for un-derstanding our research. Section 3 introduces the domain of the 2017 MAPC. It also describes the SMART-JaCaMo team’s approach for the contest; its strategies, techniques, and other features. In Section 4, we explain the implementation of the DC and C-Link algorithms and the integration of the coalition formation approach into the SMART-JaCaMo’s implementation. Next, in Section 5 we discuss and analyse our experiments for the 2017 MAPC using the coalition formation algorithms. In Section 6, we describe some related work to coalition formation applications and the use of this technique jointly with task allocation. Finally, in Section 7 we make some final considerations about our work and discuss future work.
2.
BACKGROUND
In this section we provide the background required for understanding our work. The first topic covered here is multi-agent systems. We discuss what is an agent, how the agents organise themselves, and how to program such entities. In Section 2.2 we explain the coalition formation process. We describe different perspectives to this well-studied research topic. Finally, Section 2.3 briefly describes task allocation approaches.
2.1 Multi-Agent System
There is no universally accepted definition of what is an agent. In [Woo09], Wooldridge provides a good definition related to this subject: "An agent is a computer system that is situated in some environmental, and is capable of autonomous action in this environment in order to meet its delegated objectives". An agent observes its environment, chooses the best strategy and takes an action. Figure 2.1 illustrates this behaviour. The collected information is processed by the agent, it reasons about its perceptions, taking into account its objectives. The actions performed by the agent in the environment are guided through its goals. Reasoning about how to act in order to accomplish its objectives is fundamental for autonomous agents.
Figure 2.1: View of an agent [Woo09].
Another important definition is that of intelligent agent. Intelligent agents [Woo09] have three main capabilities: (i) Reactivity, they are able to perceive their environment and respond to the changes that occur in it; (ii) Proactiveness, they take the initiative in order to satisfy their design objectives; and (iii) Social ability, they are able to interact with other agents. In multi-agent systems, we have more than one agent interacting in the environment. These gathered agents can share the
directed and short-lived; they are formed for accomplishing a purpose, after that its members disband. In a coalition we may have agents that are both cooperative and self-interested. Within a coalition one member may act as a leader, although coalitions have no explicit hierarchy. Coalitions can be viewed as a single, atomic entity, but overlapping coalition can also exist. One key characteristic is that members from different coalitions do not coordinate their actions. Teams are similar to coalitions, what differs one from the other is the member’s goals. A team consists of a number of cooperative agents which have agreed to work together toward a common goal [HL04]. The team’s members attempt to maximise the utility of the team itself. In general, each agent takes one or more roles required by the team’s goal. An example of coalition and team is depicted in Figure 2.2.
Figure 2.2: Coalition (A) and Team (B) organisational paradigms [HL04].
One of the key parts of an agent is its architecture. It provides ways for the agents to think about their actions in order to satisfy its designed objectives. One well-known architecture is the Procedural Reasoning System (PRS), based on the BDI model. As pointed out by Wooldridge in [Woo13], the BDI (Belief Desire Intention), model is inspired in the theory of human practical reasoning developed by the philosopher Michael Bratman [Bra87], which is focused on understanding the role of intentions in practical reasoning. Practical reasoning is the process of deciding which action to perform in the advancement of our goals [Woo09].
The basic components of the BDI architecture are data structures and functions. The data structures represent the beliefs, desires, and intentions of agent. Beliefs represent the state of the world, the desires represent what the agent would like to achieve, and the intentions are what the agent is committed to do in order to achieve some of its desires. The functions represent the delib-eration (what to do) and means-ends reasoning (how to do it), as described by Wooldridge [Woo13]. The conceptual framework of the BDI model is detailed in [BIP88] and depicted in Figure 2.3. The practical reasoning is composed of seven main components. The Belief Revision Function (BRF)
de-termines a new set of beliefs based on the perceptual input and the current belief base. The generate options generates the desires on the basis of its current beliefs and intentions. The filter represents the agent’s deliberation process, it determines the agent’s intentions based on the beliefs, desires and intentions. Finally, the action function determines which action to perform based on the current in-tentions [Woo13]. The PRS is based on the notion of rational agent that can reason and plan under stringent constraint on both time and information [IGR92]. The architecture of PRS consists of: (i) belief database; (ii) a set of current goals; (iii) library of plans; and (iv) an intention structure. The in-tention structure is a set of all plans chosen for execution at run-time, these plans are partially ordered and partially instantiated. The PRS interact with the environment through the belief database and the actions that it performs. PRS also has the notion of goal, which is a description of desired tasks and behaviours. The knowledge to accomplish such goals is represented by declarative procedure specifications called knowledge area (KA). Each KA has a body that describes the steps of the pro-cedure, a triggering part and a context. The triggering part describes which event must occur and the context is a logical expression specifying the conditions that must be true. PRS influenced the design of well-known agent programming languages such as the AgentSpeak language [Rao96], which has been implemented in Jason [BHW07], a development platform with a freely available open-source distribution.
Figure 2.3: BDI framework adapted from [Woo13].
AgentSpeak(L) is a programming language based on a restricted first-order language with events and actions and its syntax and semantics was first formalised in [Rao96]. An agent in AgentS-peak(L) consists of a set of beliefs, plans, events, and actions. Three selection functions are also defined for the deliberation process. The state of the agent, its environment and other agents are represented as beliefs states. The events can be external or internal. External events are changes in the environment and user requests. On the other hand, the internal events are generated by the agent.
be true given the current beliefs). The remaining plans are called the applicable plans or options. The selection function for options selects one of the applicable plans. The last selection function selects an intention to be executed. When an intention is selected to be executed, its first action or goal from the body is executed [Rao96].
In AgentSpeak(L) the variables start with upper-case and the constants with lower-case as in Prolog. Apart from the first-order connectives, AgentSpeak(L) also uses ! (for achievement goal), ? (for test goal), ; (for sequencing), and ← (for implication, it is used after the context of plan). Standard first-order definitions of terms, first-order formulas, closed formulas, and free and bound occurrences of variables are also used [Rao96]. Listing 2.1 shows an example of agent plan. When the triggering event is unified with the selected event, the context of robot at X and bin at Y is tested against the belief base (line 2). This applicable plan is turned into an intention and its steps of execution are: (i) action pick on line 4; (ii) achievement goal location (and all of its actions and goals) on line 5; and (iii) action drop on line 6.
In the following subsections we describe some important features on agent-oriented devel-opment. In Subsection 2.1.1 we introduce some programming languages for implementing agents. Subsection 2.1.2 discusses environment representation from agent’s point of view. Subsection 2.1.3 introduces how to organise the agents into multi-agent systems. Finally, Subsection 2.1.4 describes the JaCaMo platform that we have used for the multi-agent system development.
2.1.1 Agent-Oriented Programming Languages
Agent-Oriented Programming (AOP), provides a way to implement concepts and abstrac-tions that are proposed by some multi-agent architectures [Das14]. Several AOP languages have been proposed through the years and in this section we summarise and briefly introduce some of them. The agent behaviour is directed by the programmer by means of these AOP languages and its outcome is called agent program. An agent program runs on an agent architecture, like BDI, which effectively determines what an agent does. It puts an abstract layer on the development process of an agent pro-gram [BHW07]. For further details in AOP languages we refer the reader to [MW15, Das14, Woo13]. The first AOP language was proposed by Shoham in [Sho93], it is called AGENT-0. It is a simple interpreter that paved the way for more complex languages and interpreters. One is JACK Intelligent Agents (JACK for short) [BRHL99], a multi-agent framework that extends the Java
lan-1 +!location(waste,X)
2 : location(robot,X) & location(bin,Y) 3
<-4 pick(waste);
5 !location(robot,Y);
6 drop(waste).
Listing 2.1: Example of AgentSpeak(L) plan implemented in Jason.
guage. It is a proprietary software and supports the BDI model and SimpleTeam (an extension to support team-based reasoning). Agents in JACK are designed to be used as components of larger environments and they are not bound to any specific agent communication language. Another agent framework is Jadex1[BPL05]. It is built upon the JADE [BBCP05] middleware platform and it adopts
the BDI model. JADE2is a software environment to build agent systems for communication
manage-ment in compliance with the FIPA specifications [FIP16]. It facilitates the developmanage-ment of distributed multi-agent applications based on a peer-to-peer communication architecture. Agents communicate by exchanging asynchronous messages and JADE is responsible for providing mechanisms to authen-ticate and verify such messages. Jadex allows for programming BDI concepts in XML and Java in a flexible module-concept way. It also allows deployment on different kinds of middleware, and it supports an explicit and declarative representation of goals.
The 2APL is a BDI-based multi-agent programming language [Das08]. It provides practical constructs for the implementation of cognitive agents. It also has formal semantics, so in principle it is possible to verify whether the agent programs satisfy formal specifications. This semantics was implemented in the form of an interpreter3 that executes 2APL multi-agent programs. The platform
provides a graphical interface and a state tracer tool for debugging purposes. Another AOP language is the Goal-Oriented Agent Language (GOAL) [HdBvdHM01], which covers the declarative concept of a goal. It is inspired by the language UNITY [Cha88]. The idea is that a set of actions, which executes in parallel, constitutes a program. GOAL is an agent-oriented programming language that incorporates belief, goal and agent’s capability notions.
Jason [BHW07] is a implementation of an extension of AgentSpeak [Rao96], it is intended to develop agent programs for multi-agent systems. As AgentSpeak is based on the BDI model, Jason has the BDI concepts encapsulated in language constructs which are divided into three main categories: beliefs, goals, and plans [BHW07]. Each agent has its belief base, which is a collection of literals and the information is represented by predicates. The symbolic representation of logical statements and variables are closely related to Prolog. The basics types in it are: constants, variables, and structures, all of which are referred to as terms. Figure 2.4 depicts the types of AgentSpeak terms in Jason.
1https://www.activecomponents.org/#/project/news 2http://jade.tilab.com/
Figure 2.4: Types of AgentSpeak terms in Jason [BHW07].
Another important syntactical representation is the annotation. It provides further infor-mation about some predicate, and it is enclosed in square brackets following a literal. For instance, annotations are used to store the source of information. There are three types of information sources: Perceptual information: the agent acquires information through sensing the environment, this
in-formation can be added to the belief base and it is called a percept;
Communication: in multi-agent systems, the agents need to communicate to each other, this ex-change of messages are stored as beliefs annotated with the name of agent that provide the information;
Mental notes: they are beliefs added by the agent to its own belief base, in order to remind itself of things that happened in the past.
Jason also has the concept of strong negation, that express that an agent explicitly believes something to be false [BHW07]. Besides, there is the notion of rules, which allows the agent to conclude new things based on beliefs the agent already has.
Goals in Jason represent the state of the world that the agent wants to bring about [BHW07]. We have essentially two kinds of goal, achievement and test goals. The test goals are usually used to retrieve information from the agent’s belief base. With an achievement goal the agent will try to achieve that state of the world. The adoption of a new goal leads the agent to the execution of a plan, which is a course of action that the agent expects will achieve the goal [BHW07]. . In order to execute an intention, the intention selection function of Jason has the form of Round-Robin scheduling. It means that each intention is selected in turn and only one formula of the plan body is executed. For more details about the Jason’s reasoning cycle we refer to [BHW07]. The communication in Jason is based on speech-act theory. In speech-act theory utterances are actions, in the sense that the agent
makes an utterance in an attempt to change the state of the world. For example, it wishes to modify the mental states of other agents. In Jason, the messages have a performative, which describes the type of the utterance. For further details we refer to [VMWB07].
Jason also provides modules. Modules can be very useful in order to keep the agent’s mental state organised [CDH11], since it provides a mechanism to maintain units of code in a separate, independent, and reusable fashion. The use of modules in Jason was first proposed in [OHGHR13, OHB+16]. Jason uses namespaces to organise its components (beliefs, plans, and goals) into modules.
All of these components are prefixed with a namespace reference; when no such reference is defined, the components are set to the default namespace. A namespace might be global or local: global namespaces can be used by any module; conversely, local namespaces can only be accessed by the module that defines it. These modules can be loaded into the agent at compiling time (static loading), or at run-time (dynamic loading).
2.1.2 Environment-Oriented Programming
The environment is a key part of multi-agent systems. This is where the agents interact per-ceiving and acting, in order to change the state of the world. Few programming proposals to support such development of agent’s environment were proposed and we introduce them in this section.
A typical abstraction at environment level is the use of artefacts [BD13]. The artefacts needs to encapsulate the notions of action and perception, in order to be connected with agents. One artefact abstraction was proposed by Ricci et al. in the CArtAgO framework [RPV11]. CArtAgO is a Java implementation based on the A&A meta-model [ORV08, RVO08], which is characterised in terms of three basic abstractions: (i) agents; (ii) artefacts; and (iii) workspace. Agents represent the pro-active components of the system, artefacts are the reactive components used by agents, and workspace are conceptual containers of agents and artefacts [ORV08].
In the CArtAgO framework, artefacts can be used to enable access to resources, hide low-level details of implementation, and encapsulate coordination mechanisms. They encapsulate specific features and can be added, removed, and organised by agents at run-time. The CArtAgO concepts are depicted in Figure 2.5.
Operations are triggered by agents or other artefacts, they represent computational processes executed inside artefacts. Such operations can generate observable events, also called signals, that are non-persistent events occurred inside the artefact. Persistent events are perceived by agents through observable properties which store the state variables of the artefact; they become beliefs if the agent is focusing on that particular artifact. These artefacts are organised into workspaces which are contained in the environment. Agents can join or quit workspaces at run-time, choosing the artefacts they are in-terested in, and through operations and observable properties interact with the environment [RPV11]. Another approach is the development of a Environment Interface Standard (EIS), to be applied to arbitrary agent platforms as proposed in [BHD11, BHB+12]. Their interface facilitates the
Figure 2.5: CArtAgO concepts in the Unified Modelling Language notation [RPV11].
sharing and easy exchange of environments for agents. The standard is based on a meta-model for agent-environment interaction and a set of principles. The model defines the components and their interactions based on the proposed standards and the principles lead to a correct implementation. The model is depicted in Figure 2.6, where EIS is put between agents and environment.
An agent, according to the proposal in [BHD11], is anything able to perceive and act upon the environment. The platform is responsible for instantiating and executing the agents. The actions and perceptions of agents are applied to the environment through controllable entities which are con-tained in the environment model. Such entities might or might not be controlled by agents and they can be created or removed from the environment [BHD11]. The agents and controllable entities are abstracted to unique identifiers, which allows integration with different APLs. The environment man-agement system provides all the actions for managing the environment. The EIS box in Figure 2.6 connects the platform, the environment management system and the agents to the environment; such relation is depicted as black lines in the figure.
2.1.3 Organisation-Oriented Programming
In multi-agent systems, the agents may organise themselves in organisations and define norms that rule the agents’ interaction. Some frameworks related to organisations have been pro-posed in the literature, as for example the MOISE Model [HSO07, HBKR10] and Electronic Institu-tions [SAN+04].
The MOISE organisation modelling language [HSO07], allows the specification of struc-tural, functional, and normative dimensions of an organisation. At the structural specification there are three levels:
• Individual: the behaviour that an agent is responsible for when it adopts a role;
• Social: the relationship between the roles. It can be authority, acquaintance or communication; • Collective: the roles that compose a group.
The functional dimension describes how the organisation goals will be achieved. This specification is done be means of schemes which are goal decomposition trees. In such models, the root is a global goal and it is decomposed into sub-goals that can be achieved by the agents. An example of scheme for a soccer team is depicted in Figure 2.7. In order to fulfil a scheme, a plan can be defined which is an instance of one goal decomposition tree. Three operators are used for designing a plan: (i) sequence; (ii) choice; and (iii) parallelism. With a sequence operator, a goal can be achieved only after the achievement of the previous one. In the choice operator, only one of the set of goals must be achieved. Finally, in the parallelism operator, all goals must be achieved, but they can be executed in parallel. Another important concept in MOISE is that of a mission. A mission is a set of goals from a scheme that is allocated for an agent to execute. Once the agent is committed to a mission, it must fulfil the goals designated for the mission.
At the normative dimension, it is defined what is permitted or obliged. The roles played by the agents are connected to the missions. When a mission is permitted to a given role, the agent that plays this role can commit to that mission. In an obligation statement, the agent playing the role must
Figure 2.7: MOISE scheme for a soccer team [HSO07].
commit to that mission. The organisational specification is stored in a XML file and the agents can reason about the organisation and modify it at run-time [HSO07].
In the Electronic Institutions (EI) platform [SAN+04], agents adopt roles and interact in order to achieve their individual and organisational goals. It is a social middleware. The interaction between agents is by exchange of speech-act messages through agent group meeting (called scene), in which a role represents a standardised behaviour. When more than one scene are connected, a workflow is composed and it is called the performative structure. Agents can enter, leave, create, and destroy scenes. The ontology, common language, and knowledge representations are encapsulated in the dialogical framework. In EI the agents’ behaviour are affected by obligation and prohibition rules. EI provides a IDE which is called IDE-eli. This IDE is composed of: (i) ISLANDER; (ii) AMELI; (iii) aBUILDER; and (iv) SIMDEI. The rules and protocols are specified in ISLANDER. The agents are developed with aBUILDER. The programs written in ISLANDER are run by AMELI, the software platform. Finally, the SIMDEI is a simulation tool to animate and analyse ISLANDER specifications [SAN+04].
2.1.4 Multi-Agent Systems Development Platforms
Developing multi-agent systems is a complex task. We need to consider several aspects from the agent, the environment, and the organisations point of view. As pointed out in [BD13], it is from the combination of each of these three different levels of abstraction in multi-agent systems that we are able to develop complex distributed systems. JaCaMo [BHR16] was developed to address such challenges, it is the first approach to consider the agent-environment-organisation dimensions in a single development platform (which is freely available and open source).
Boissier et al. [BBH+13] state that JaCaMo is a conceptual framework and platform, which provides a high-level first-class support for developing agents, environments, and organisations in synergy. Jason [BHW07] is used for programming agents. CArtAgO [RPV11] is used in environ-ment programming, and MOISE [HSO07] for programming organisations. The integration of the dimensions in JaCaMo is depicted in Figure 2.8.
Figure 2.8: Dimensions in JaCaMo [BBH+13].
2.2 Coalition Formation
In multi-agent systems, there are a number of agents interacting on the same environment. These agents can execute their own tasks, but sometimes one task may require more effort than one agent can offer. To address this issue, the agents can strategically and temporarilly get together in order to improve their performance or to accomplish task they could not do alone. The agents may be cooperativeor self-interested. Cooperative agents share a common goal, while self-interested agents only care about their own goals. After achieving a goal, the agents usually receive a reward for it. Coalition formation studies how we address the problem of how to group agents so as to maximise the rewards they get for their efforts. When considering coalitions, we mean just a set of agents that will work together. A partition of this set of agents into coalitions is called a coalition structure.
2. Solving the optimisation problem of each coalition: Once we have defined the coalition structure, the coalitions begin to define how to address their own problems to be solved. 3. Dividing the solution value among the agents: This is related to the division of the coalition
reward between the coalition’s members right after the coalition has achieved its goal. Usually, the outcome of the optimisation problem is used as reward4.
Wooldridge in [Woo09] referred to these activities as the cooperation life-cycle; we depict this cycle in Figure 2.9. It summarises how the agents may cooperate with one another to accomplish their desired goals. From single agents we put them together based on some criteria; after that, the coalition begins to decide which agent will be responsible for which task, then they create a plan to accomplish the goal.
We will formally describe the coalition formation process through Characteristic Function Games(CFG) under the perspective of Cooperative Game Theory (also called Coalitional Games).
In such games, we have a pair G = hA, vi, where A is the set of agents and v is a function which assigns a real value to every coalition, v : 2A→ R, which is called the characteristic function of the
game. A characteristic function assigns a coalition value to the coalition as a whole, not individual members [CEW11]. A coalition is a subset of A, C : C ∈ A. A coalition structure CS is a partition of A into mutually disjoint coalitions, i.e., for all C, C0 ⊆ CS, C ∩ C0 = ∅, C 6= C0andS
C∈CS = A.
The value of a coalition structure CS is defined as V (CS ) = P
C∈CSv(C). Example 1 introduces
an instance of this problem. This is the classic form of the game; alternatively, some extensions have been proposed to address such problem, we will discuss them in the next sections.
Example 1. Consider a set of agents A = {a1, a2, a3}. The feasible coalitions are: {a1}, {a2},
{a3}, {a1, a2}, {a1, a3}, {a2, a3}, {a1, a2, a3}. In this sense, the space of all partitions over the A is
ΠA= {{{a
1}, {a2}, {a3}}, {{a1}, {a2, a3}}, {{a2}, {a1, a3}}, {{a3}, {a1, a2}}, {{a1, a2, a3}}.
In CFG, the value that each coalition can get is the same regardless of values the other coali-tions can get. In other words, the value of a coalition is equal in all the possible coalition structures. In the opposite way, Partition Function Games (PFG) takes into account the values that the other coalitions receive. The value of each coalition depends on the chosen coalition structure [ERJ13]. The coalition structure generation can be done endogenously or exogenously [MTMW09]. In en-dogenous coalition structure generation, the agents decide among themselves the best way to split
4In order to divide the reward among the coalition’s members, we can use game theory concepts such as the core and
Agents (resources+capabilities+goals) Coalition Formation Coalitions (set of agents) Team Formation Teams (coalitions+plans) Coordinated Cooperative Action Agreement on who will work
together
Agreement on who does each task with which
resources Agreement on how to dynamically coordinate TIME
Figure 2.9: The cooperation lifecycle, adapted from [Woo09].
the set of agents. In the exogenous approach, there is a system designer that puts the agents together. In [RMWJ15], Rahwan et al. introduced a survey on coalition structure generation. They address sev-eral aspects of the coalition structure and summarise the research area using the classification depicted in Figure 2.10.
The CFC is in the middle of Figure 2.10; in it, every coalition is admissible and the value of the coalition is a real number that only depends on the identities of its members. Inside this set, we have two well-known representations: distributed constraint optimisation problem (DCOP) and induced subgraph. In the former, the value of a coalition is equal to the solution of DCOP involving the coalition members. In the induced-subgraph approach, the characteristic function is based on a weighted graph in which each node represents an agent, and the sum of the weights correspond to the coalition value. There are some alternative representations for CFG: (i) skill-based, in which the value of coalitions depends on the skill of its members; (ii) agent-type, the contribution of each agent type is summed in the coalition value; (iii) synergy coalition groups, where only the coalitions with synergy are modelled; and (iv) marginal contribution net, in which the characteristic function is the sum of the rules applicable to the coalition (the embedded marginal contribution net is applicable to PFG). When constraints must be taken into consideration we have graph-restricted games and constrained coalition games.
Figure 2.10: Representations of coalition structure, adapted from [RMWJ15].
In order to explain the phases of the coalition formation, we divided in sections some im-portant concepts regarding this subject. First, in Section 2.2.1 we describe how we can represent the search space while evaluating coalitions. It is an important topic to improve the performance of algorithms that solve the coalition structure generation problem. Section 2.2.2 describes algorithms that solve the coalition structure generation problem. In Section 2.2.3 we explain how to apply con-straints to narrow down the coalition structure search space. After that, Section 2.2.4 introduces some compact representations of the characteristic function, as it is normally considered a black box function [RMWJ15]. Finally, Section 2.2.5 and Section 2.2.6 explain in details two algorithms for coalition formation that are important for our work.
2.2.1 Search Space Representation
In order to find an optimal coalition structure, we first need to represent the search space ΠA, the space of all coalition structures over the set of agents A. The first representation was proposed by Sandholm et al. [SLA+99] and it is called coalition structure graph. In their approach, the coalition structure process is viewed as a graph representation. The coalition structures are represented by vertices, and the graph is divided into levels, ΠA1 . . . ΠA
n. Each node in level ΠAi contains exactly i
links two vertices if and only if they belong to two consecutive levels ΠAi and ΠAi−1, and if one coalition inside the C ∈ ΠAi can be divided into two coalitions in C0 ∈ ΠA
i−1.
{a1,a2,a3,a4} {a1,a3}{a2,a4} {a2}{a1,a3,a4}
{a1,a2}{a3,a4}
{a1}{a2,a3,a4} {a3}{a1,a2,a4} {a1,a4}{a2,a3} {a4}{a1,a2,a3} {a1}{a3}{a2,a4} {a2}{a4}{a1,a3}
{a3}{a4}{a1,a2}
{a1}{a2}{a3,a4} {a1}{a4}{a2,a3} {a2}{a3}{a1,a4} {a1}{a2}{a3}{a4} Level (4) (3) (2) (1)
Figure 2.11: The coalition structure graph for four agents, adapted from [SLA+99].
Another approach for representing the search space was proposed in [RRDJ07], and it is based on the coalition structure configuration. A configuration is the grouping of the coalition struc-tures according to the size of its coalitions. For instance, the coalition strucstruc-tures {{1}, {2, 3}} and {{3}, {1, 2}} follow the configuration {1, 2}. The representation in [RRD+07] is encoded in [RJ08a]
by a integer partition graph. Levels are also present in this representation, each level Lsrepresents a
subspace containing s coalitions per coalition structure. For instance, the level L2 in Figure 2.12 has
the nodes {{3, 1}, {2, 2}}, the node {3, 1} has all the coalition structures that have one coalition of three members and another coalition of one member. An edge connects two nodes when one of the coalition sizes inside a subspace is split out into two new parts, where the sum is the previous value.
{4} {3,1} {2,2} {2,1,1} {1,1,1,1} (L4) (L3) (L2) (L1) {{a1,a2,a3,a4}} {{a1,a3,a4},{a2}} {{a1,a2,a4},{a3}} {{a2,a3,a4},{a1}} {{a1,a2,a3},{a4}} {{a1,a2},{a3,a4}} {{a1,a3},{a2,a4}} {{a1,a4},{a2,a3}} {{a1,a2},{a3},{a4}} {{a1,a4},{a2},{a3}} {{a2,a4},{a1},{a3}} {{a3,a4},{a1},{a2}} {{a2,a3},{a1},{a4}} {{a1,a3},{a2},{a4}} {{a1},{a2},{a3},{a4}}
Figure 2.12: The integer partition graph for four agents, adapted from [MRE+15].
2.2.2 Algorithms for Coalition Structure Generation
Having explained the forms we can represent the search space of coalition structures, now we explain some algorithms designed to search for coalition structures. Rahwan and Jennings [RJ08b]
ability to prune, discrimination, anytime, and worst-case guarantee. The space of possible coalition structures is represented as integer-partition subspaces. The amount of search needed is reduced with a branch-and-bound technique. A minimum and a maximum bound are evaluated for each subspace. These bounds are used to speed up the search in two ways: (i) if the minimum bound of a subspace is greater than the maximum of another subspace, the latter subspace must not be searched; and (ii) if the current best solution is greater than the maximum of a subspace, this subspace is cut off. The IP algorithm is guaranteed to find an optimal solution when run to completion [RRJG09].
A distributed version of IP was introduced in [MSR+10], the first decentralised algorithm for resolving the CSG problem. This algorithm uses filters on the input reducing the communication and it applies the techniques from [RJ07], which is an algorithm to distribute the calculation of the possible coalitions among the agents. Other algorithms developed to find the optimal coalition struc-tures are [SLA+99, DJ04]. In [SLA+99], an algorithm is introduced which searches through the first
two levels and the last level of the coalition structure graph. After that, a breadth-first search is used and it continues as long as there is time left or until the entire graph has been searched. In [DJ04] the algorithm proposed by Sandholm et al. [SLA+99] is improved. It searches through the same three levels as Sandholm et al. does; after that, instead of searching for all the remaining levels, it searches some subsets of each remaining level based on the coalition structure cardinality within the level.
In the Dynamic Programming class of algorithms the main advantage is the lower time complexity. Some algorithms are guaranteed to find an optimal solution in O(3n) for n agents. The drawbacks are: (i) it requires more memory; and (ii) it needs to complete the execution of the algo-rithm in order to output a solution. The first algoalgo-rithm that used the dynamic programming paradigm is DP [Yeh86] algorithm; it is designed to solve the complete set partitioning (CSP) problem. The time complexity and space complexity were shown to be O(3m), where m is the number of rows of
an auxiliary table necessary to represent the number of elements in binary notation. For instance, for seven elements m is equal to three. DP iterates over all coalitions of size 1, then coalitions of size 2, and so on, up to the grand coalition. When the size of a coalition is greater than 1, the algorithm compares the value of the coalition with the sum of the previous coalitions’ sizes that compose this bigger coalition. For instance, DP evaluates the coalition {a2, a3} with the sum of coalitions {a2}
plus {a3} and it stores the best way of splitting the coalition.
In [RJ08a], Rahwan et al. showed that the DP algorithm performs many redundant opera-tions. They introduced the ODP algorithm which avoids approximately two thirds of the operations of DP [RMWJ15]. They identify these redundant operations when DP is evaluated on the coalition
structure graph. The optimal coalition structure can be reached from different paths. ODP identifies a prioriwhat are the relevant movements to be executed without any need for extra memory require-ments. If there is one path from any node to any other node in the graph, ODP is able to find the optimal coalition structure.
The fastest version of ODP was proposed in [MRE+15]. It combines the ODP and IP algorithms, introducing the ODP-IP algorithm. This hybrid algorithm avoids the limitations of its two components and improves the advantages of each. First the authors identify that the movements made by DP in the coalition structure graph can also be visualised on the integer partition graph, which is used by IP. When DP makes a movement from coalition structure node {{a1}, {a2, a3, a4}}
to {{a1}, {a2}, {a3, a4}}, it also moves from subspace node {3, 1} to {2, 1, 1} in an integer partition
graph as depicted in Figure 2.13. One edge in the integer partition graph is represented by some edges in the coalition structure graph. ODP can avoid only some edges from a coalition structure graph of a given size which means the edge that links two subspaces in the integer representation cannot be cut off. In order to address this problem, Michalak et al., in [MRE+15], proposed a size-based version of ODP which evaluates all or none of the movements that link two nodes in an integer partition graph; it is called sb-ODP or IDP. Combining IP and sb-ODP results in ODP-IP, which is an anytime algorithm and its time complexity is O(3n).
{a1,a2,a3,a4} {a1,a3}{a2,a4} {a2}{a1,a3,a4}
{a1,a2}{a3,a4}
{a1}{a2,a3,a4} {a3}{a1,a2,a4} {a1,a4}{a2,a3} {a4}{a1,a2,a3} {a1}{a3}{a2,a4} {a2}{a4}{a1,a3}
{a3}{a4}{a1,a2}
{a1}{a2}{a3,a4} {a1}{a4}{a2,a3} {a2}{a3}{a1,a4} {a1}{a2}{a3}{a4}
(a) CSG representation of ODP.
{4} ODP {3,1} {2,2} {2,1,1} {1,1,1,1} (b) IPG representation of ODP. {a1,a2,a3,a4} {a1,a3}{a2,a4} {a2}{a1,a3,a4} {a1,a2}{a3,a4}
{a1}{a2,a3,a4} {a3}{a1,a2,a4} {a1,a4}{a2,a3} {a4}{a1,a2,a3} {a1}{a3}{a2,a4} {a2}{a4}{a1,a3}
{a3}{a4}{a1,a2}
{a1}{a2}{a3,a4} {a1}{a4}{a2,a3} {a2}{a3}{a1,a4} {a1}{a2}{a3}{a4} (c) CSG representation of IDP. {4} IDP {3,1} {2,2} {2,1,1} {1,1,1,1} (d) IPG representation of IDP.
Figure 2.13: Comparison of search space representation between CSG and IPG in an example of four agents for ODP and IDP algorithms.
Another approach uses the inclusion-exclusion principle [BHK09] to solve the set parti-tioning problem. In such an approach, the information is encoded into extremely large numbers and has a theoretical time complexity O(2n). However, it does not take into account the time required to manipulate such huge numbers. In practice the algorithm runs in O(6n) [RMWJ15].
is O(2n), but it can be reduced to O(nk) by inserting a constraint on the coalition size, in which k represents the highest size allowed. However, it still remains a NP-Complete problem [SK98].
A genetic algorithm was introduced in [SD00]. The algorithm starts with a randomly gen-erated coalition structure. It iterates through three stages: (i) evaluation; (ii) selection; and (iii) re-combination. It evaluates all the members of the coalition structure, selects members based on these evaluations and modify or exchange their content, so new members are generated. An algorithm based on simulated annealing was proposed in [Kei09]. The algorithm starts generating random coalition structures. It takes one CS and evaluates its neighbourhood. For a neighbour CS0, the algorithm compares if it is better than the current coalition structure. If it is, then the algorithm moves to CS0. Otherwise, the algorithm only moves to the neighbour based on a probability that decreases with time. The greedy algorithm proposed in [MBFE10] is based on Greedy Randomized Adaptive Search Procedures (GRASP) [FR95]. GRASP is a iterative process composed of two phases: (i) a constructive phase in which a feasible solution is produced; and (ii) a searching phase in which a local search is performed on the neighbourhood of the constructed solution. In [MBFE10], the coalition structure starts empty, then one agent is added to it, as a singleton coalition or as member of some coalition. After that, a local search is performed in the neighbourhood of this coalition structure. The neighbourhood is defined according to five operations (for more details we refer to [MBFE10]). The algorithm ends when all the agents were added to the coalition structure.
Farinelli et al. [FBRZ13] proposed the C-Link algorithm, which goes trough a coalition structure graph. It starts at the top node in the CSG and out of all downward movements it picks the one with best immediate reward. The algorithm finishes when no downward movement is beneficial. It is a heuristic algorithm and its time complexity is O(n3). Section 2.2.6 describes in details the
C-Link’s execution.
Integer programming can also be used for coalition structure generation problems. Once a formulation of the problem is designed, it can be applied to any integer programming solver [RMWJ15]. However, this approach is slower than both IDP and IP, and it runs out of memory for problems with around 20 agents when the experiments were conducted [RRD+07].
2.2.3 Algorithms for Constrained Coalition Structure Generation
After introducing algorithms that solve the coalition structure generation problem, we now describe some algorithms that put constraints on the search space in order to reduce the number of feasible coalitions and coalition structures. In practice, many applications do not need to take into account all possible combinations of agents in order to create suitable coalition structures. For example, sometimes there are constraints that narrow down the number of different coalitions allowed. These constraints may occur for many reasons, they may be due to communication constrains, trust relations, or physical constraints [VRJ12]. In this sense, algorithms for coalition formation may need to take into account such constraints.
Rahwan et al [RME+11] introduced the first systematic study of Constrained Coalition
For-mation (CCF for short). Formally, the CCF game is defined as G = hA, CS, vi where A is the set of agents, CS is the set of feasible coalition structures and v(.) is a function which assigns a real value to every coalition in some coalition structure. In order to represent the constraints, the authors propose the use of propositional logic in which the Boolean variables correspond to agents. More formally, let BA= {bi | ai ∈ A}. A CCF game is propositionally definable if there is a logical formula φ over
BAsuch that CS = {CS | CS |= φ}. In other words, the coalition structure must satisfy the formula
φ.
Such games also have the notion of locally-constrained coalition structure [RME+11]. This is the basis for the CCF framework. In that definition, the feasibility of a coalition structure is de-termined by the coalitions that are member of it. In other words, the constraints are on each of the coalitions rather than the overall coalition structure. Formally, a CCF game is locally constrained if there exists a set of feasible coalitions C ⊆ 2A such that CS = {CS ∈ ΠA | CS ⊆ C}, where ΠA
denotes the set of all partitions of A. The constraints defined in the CCF games are expressed in the form of: (i) sizes of the coalitions; and (ii) subsets of agents whose presence in any coalition is viewed as desirable5 or prohibited. The size constraints are denoted by S ⊆ {1, . . . , n}, where S ⊆ N is the set of permitted coalition sizes. The set of desirable constraints (called positive constraints), which are denoted as P ⊆ 2A such that a coalition C satisfies a constraint P ∈ P if P ⊆ C. In the same way, the set of prohibited constraints (negative constraints), which are denoted as N ⊆ 2A such that C satisfies a constraint N ∈ N if N * C. This kind of game is known as basic CCF game, and is represented as a tuple G = hA, P, N , S, vi. A coalition is considered feasible if it has at least one positive constraint, no negative constraint, and the size of the coalition is permitted. Formally, a coalition C ⊆ A is feasible for G = hA, P, N , S, vi, if: (i) P ⊆ C for some P ∈ P; (ii) N * N for all N ∈ N ; and (iii) |C| ∈ S. The function that returns the set of all feasible coalitions is denoted by c(A, P, N , S). A coalition structure CS is only feasible if CS ⊆ c(A, P, N , S). The value of a coalition structure is represented as v(CS) =P
C∈CSv(C).
5Desirable here refers to a subset of agents that must be desirable in the same coalition. A feasible coalition must fulfil
to find the optimal feasible coalition structure, titled DyCE. In their proposed model, the coali-tion formacoali-tion with sparse synergies (CFSS), the set of feasible coalicoali-tions is constrained by a graph G = (A, E), where A is the set of agents and E is the set of edges between the agents. The coalition is considered feasible if and only if there exists a connected subgraph G0 = (C, E0), where C is the feasible coalition and E0 ⊆ E. Each coalition is assigned a real value by a characteristic function v : (C) → R.
The SlyCE algorithm conducts a depth-first search over a tree representation of the set of feasible coalitions in a graph G. SlyCE has been proved to be correct (i.e., it only evaluates feasible coalitions), complete (i.e., it can enumerate all feasible coalitions), and non-redundant (i.e., it never evaluates the same coalition twice). Figure 2.14 depicts an example of a SlyCE execution. The algorithm runs for each ai ⊆ A in order to find the values of all coalitions. From the root, the possible
Figure 2.14: Root expansion from agent 1 in SlyCE, reproduced from [VRJ12].
coalitions are put in the frontier, but it must obey one constraint: the ID value of each node must be higher than the ID of the current root. In the example in Figure 2.14, in the frontier of root {1} are the coalitions {1,2}, {1,3}, {1,4}, {1,2,3}, and so on. The frontier is expanded to include agents that are not currently included, obeying the constraint on higher node IDs. The coalitions evaluated are stored in order to prevent redundancy. For instance, considering root agent 5 in Figure 2.14, it only has one possible coalition {5}; this is because the other possible coalitions have already been explored. The SlyCe algorithm has a possible parallel behaviour, in the sense that the computations of this algorithm can be distributed among the agents. The algorithm which addresses this problem is D-SlyCE [VRJ12, BFC+14]. It has the same properties as SlyCE, that is, it is correct, complete, and non-redundant.
DyCE [VRJ12] uses a dynamic programming technique to find the optimal coalition struc-ture. It is based on IDP [RJ08b] with the addition of an initial phase in which the feasible coalitions are enumerated using the SlyCE algorithm. Two nodes in DyCE are connected by an edge if, from two feasible coalition structures, the first coalition structure can have one coalition split into two smaller feasible coalitions. The algorithm evaluates only some edges that connect the nodes of a coalition structure graph, like IDP does. The evaluation of the coalition subsets is done through the w(.) func-tion. It compares two coalition’s values in order to decide whether it is better to split the coalition or not. At the beginning of DyCE, a memory block containing all the coalitions of A is initialised with the value −∞. All the feasible coalitions are evaluated and their values are stored in memory (v(C)). SlyCE provides the feasible coalitions to DyCE. After that, DyCE goes through each coalition of size s = 1, 2, ..., n calculating the w(.) value. Here, SlyCE is used again in order to provide the feasible subsets of the coalition that is under evaluation.
DyCE has some drawbacks, as mentioned in [BFC+14]: (i) it requires an exponential amount of memory in the number of agents; and (ii) it is not an anytime algorithm. In order to overcome these limitations, they proposed a new way of searching through the search space in the CSG problem. They use the edge contraction technique to go from one coalition structure to another. The edges are coloured using the notion of 2-coloured graph [CLRS09]. The red colour means the edge cannot be contracted, and a green edge means it can be contracted. The coalition structure gen-eration is represented in a tree. The root is the origin of the graph with constraints. Each new node represents a feasible coalition structure. In order to generate the node, one green edge is chosen to be contracted, meaning two coalitions are merged. After merging the coalitions, a new graph G0 is generated and the contracted edge is coloured red at the parent graph G. The algorithm selects an-other green edge in G to contract, this contraction generates a new graph G00and the selected edge is coloured red, and so on for all the remaining green edges.
For instance, consider the root in Figure 2.15, called in this example G. When the edge be-tween the coalitions {A} and {B} is contracted, a new graph is generated, G0 = {{A, B}{D}{C}}. This edge in G is coloured red. The edge between coalitions {B} and {C}, in graph G, is contracted and a new graph is generated, G00 = {{B, C}{A}{D}}. Note that the edge coloured previously is present in graph G00. The algorithm paints the contracted edge in red and it goes on with the execution. The CFSS algorithm proposed in [BFC+14] makes use of branch-and-bound techniques that
focus on m+a (monotonic-antimonotonic) functions, a general class of characteristics functions. The m+a function is the sum of a supperadditive and a subadditive function. A supperadditive function has a monotonic behaviour in the sense that, given two coalitions C, C0 : C, C0 ∈ A and C ∩ C0 = ∅,
the value of the union of these coalitions is no less than the sum of the coalitions’ separate values, v(C ∪ C0) ≥ v(C) + v(C0). Conversely, a subadditive function has an antimonotonic behaviour; the value of the union of these coalitions is no greater than the sum of the coalitions’ separate values, v(C ∪ C0) ≤ v(C) + v(C0) [BFC+17]. The maximum bound is established by the sum of v(C ∪ C0)
Figure 2.15: Graph representing the space of coalition structures, reproduced from [BFC+14].
than the best solution found so far, the sub-tree is pruned. For anytime performance, in the worst-case scenario, the CFSS algorithm provides solutions that are at least 88% of the optimal.
Another approach considering constraints on coalition structure was proposed in [SMM+16].
It does not take into account constraints in coalitions (games known as locally constrained). They no-tice that many physical and organisational restrictions are in the coalition structures rather than the coalitions. For instance, in a organisation, any coalition is feasible, but no coalition structure may contain more coalitions than the numbers of leaders of this organisation. In order to address this scenario, they introduce the k-coalitional cooperative games [SMM+16]. It is a subclass of PFG in which all coalitions in the coalition structure affect each other values. In such game, any coalition is feasible, but the number of coalitions in the coalition structure is limited by a constant k.
In [AA13], a novel constrained coalition formation model is proposed. It allows self-interested agents to negotiate towards the formation of coalitions. The agents negotiate based on constraints instead of their coalition structures. The authors use the Semi-ring Constraint Satisfaction Problem (SCSP) to model the negotiations for coalitions and to evaluate the proposal. This problem is modelled as a CCF game. Each agent makes its own decisions based on its preferences. The agents have a degree of desirability for each proposal, which is composed of constraints. In the coalition process we have three steps: (i) initialisation; (ii) the negotiation on the agents’ constraints; and (iii) the validation of the solution. At the beginning of the protocol, each agent identifies the actions it intends to achieve and it informs the other agents. After that, the agents analyse the actions that may be interesting to it and compute the possible coalitions they want to join. The agent who initiates the protocol sends its proposal to the interested agent. This agent can accept or reject the proposal. When it rejects, it may ask the sender to change its proposal or it can leave the negotiation. The negotiation continues until a solution is found or a deadline is reached.
