Multi-agent coordination applied to UAVs

(1)

UNIVERSIDADE FEDERAL DE SANTA CATARINA GRADUATE PROGRAM IN AUTOMATION AND SYSTEMS

ENGINEERING

Marcelo Sousa Menegol

MULTI-AGENT COORDINATION APPLIED TO UAVS

Florian´opolis 2018

(2)

(3)

Dissertation presented to the Grad-uate Program in Automation and Systems Engineering in partial ful-fillment of the requirements for the degree of Master in Automation and Systems Engineering.

Advisor: Prof. Jomi Fred H¨ubner, Ph.D. Co-advisor: Prof. Leandro Buss Becker, Ph.D.

Florian´opolis 2018

(4)

Ficha de identificação da obra elaborada pelo autor,

através do Programa de Geração Automática da Biblioteca Universitária da UFSC.

Menegol, Marcelo Sousa

Multi-Agent Coordination Applied to UAVs / Marcelo Sousa Menegol ; orientador, Jomi Fred Hübner, coorientador, Leandro Buss Becker, 2018. 95 p.

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2018.

Inclui referências.

1. Engenharia de Automação e Sistemas. 2. Sistemas Multi-Agentes. 3. Agentes BDI. 4. VANT. 5. Sistemas Embarcados. I. Fred Hübner, Jomi. II. Buss Becker, Leandro. III. Universidade Federal de Santa Catarina. Programa de Pós-Graduação em Engenharia de Automação e Sistemas. IV. Título.

(5)

This Dissertation is recommended in partial fulfillment of the require-ments for the degree of “Master in Automation and Systems Engineer-ing”, which has been approved in its present form by the Graduate Program in Automation and Systems Engineering.

Florian´opolis, April 13th 2018.

Prof. Daniel Coutinho, Ph.D Graduate Program Coordinator Universidade Federal de Santa Catarina

Dissertation Committee:

Prof. Jomi Fred H¨ubner, Ph.D. Advisor

Universidade Federal de Santa Catarina

Prof. Leandro Buss Becker, Ph.D. Co-advisor

Universidade Federal de Santa Catarina

Guilherme Vianna Raffo, Ph.D. Universidade Federal de Minas Gerais

Ubirajara Franco Moreno, Ph.D. Universidade Federal de Santa Catarina

Felipe Meneguzzi, Ph.D.

Pontif´ıcia Universidade Cat´olica do Rio Grande do Sul (Videoconference)

(6)

(7)

ACKNOWLEDGEMENTS

Gostaria de agradecer `a minha fam´ılia, Osmar, Rita e Matheus, por me dar suporte a vida inteira e permitir que eu chegasse t˜ao longe.

Agrade¸co a minha amada Gabriela, que me inspira e motiva a seguir em frente apesar das adversidades.

Agrade¸co do fundo do meu cora¸c˜ao aos meus amigos de perto e longe, passado e presente, por me ajudarem a seguir esta jornada e a me transformar na pessoa que sou hoje.

Agrade¸co os meus orientadores, Jomi e Leandro, sem os quais este trabalho n˜ao seria poss´ıvel.

Agrade¸co `a CAPES e ao CNPq por terem ajudado a financiar este trabalho.

(8)

(9)

RESUMO

Esfor¸cos já foram realizados pare se embarcar agentes e tecnolo-gia orientada a agentes em robôs e outros sistemas. Para este propósito, várias arquiteturas e plataformas já foram desenvolvidas para permitir que agentes de software interajam apropriadamente com o hardware desses sistemas. Porém, poucas propostas avaliam sistemas nos quais há vários agentes envolvidos que precisam se coordenar. Esta disserta¸cão compara a abordagem de sistemas multi-agentes, utilizando uma lin-guagem de programa¸cão orientada a agentes, contra outra aboragem mais tradicional, com programa¸cão imperativa, no desenvolvimento de sistemas embarcados que necessitam de coordena¸cão. Os experimentos foram realizados com ve´ıculos aéreos não tripulados (VANT) e agentes BDI embarcados.

Uma arquitetura de integra¸cão agente-hardware foi desenvolvi-da para estabelecer uma conexão de agentes de software com com o hardware do VANT. Esta arquitetura foi testada em um VANT real. Dois outros experimentos, criados para se testar a coordena¸cão, foram fei-tos: um simples, para testar as funcionalidades básicas da estratégia de coordena¸cão, e outro mais complexo, simulando uma missão de busca e resgate cooperativa. Estes dois experimentos foram desenvolvidos e testados com hardware-in-the-loop.

Os resultados dos experimentos defendem o argumento de que a programa¸cão orientada a agentes oferece benef´ıcios a aplica¸cões embar-cadas, principalmente as que envolvem coordena¸cão. Mesmo que uma solu¸cão em uma linguagem imperativa possa ser computacionalmente mais eficiente, a aboordagem com agentes também é eficaz e oferece facilidades no processo de desenvolvimento.

Palavras-chave: Sistemas Multi-Agentes. Agentes BDI. VANT.

(10)

(11)

RESUMO ESTENDIDO

Introdu¸c˜ao

O Brasil é um pa´ıs com um grande número de desastres pluvi-ais, como enchentes e deslizamentos de terra. Quando estes desastres ocorrem, não é incomum que v´ıtimas fiquem desamparadas em meio a destrui¸cão a espera de resgate. Nestes casos, uma opera¸cão de busca e resgate é normalmente necessária para se prover socorro. Por se tratar de uma área de risco para a própria equipe de resgate, tem crescido o caso destes tipos de opera¸cões com robôs.

A ado¸cão de VANTs (Ve´ıculos Aéreos Não Tripulados) em todos os tipos de aplica¸cões tem crescido recentemente, inclusive em opera¸cões de busca e resgate. Este é um tipo de opera¸cão em que a utiliza¸cão de mais de um robô pode agilizar o cumprimento da missão. Porém, estes robôs precisam agir de forma coordenada para alcan¸car o objetivo em comum.

A área de sistemas multi-agentes tem algumas solu¸cões para se coordenar diferentes entidades comprometidas a objetivos em comum. Portanto, este trabalho compara uma abordagem orientada a agentes com uma abordagem mais tradicional para se resolver um problema de coordena¸cão de vários VANTs.

Objetivos

Este trabalho tem como principal objetivo avaliar o desempenho e flexibilidade de solu¸cões de sistemas multi-agentes em uma opera¸cão de busca e resgate. Nest contexto, desempenho se refere ao uso de re-cursos computacionais, como memória e prcessador, para se concluir a tarefa. Flexibilidade ´e um critério subjetivo que se refere ao esfor¸co necessário para se migrar entre diferentes aplica¸cões usando as mesmas ferramentas.

Os objetivos espec´ıficos deste trabalho s˜ao: averiguar se a solu¸c˜ao baseada em agentes funciona em um VANT real (n˜ao simulado) e avali-ar as solu¸c˜oes baseadas em agente para coordena¸c˜ao quanto a desem-penho e flexibilidade.

Metodologia

Para alcan¸cas os objetivos propostos, realizam-se várias ativida-des. Primeiro, para permitir que os agentes possam comandar um VANT, cria-se uma arquitetura de integra¸cão agente-hardware. Em seguida, o método de coordena¸cão é definido e adaptado para o caso com vários robôs. Com esta infra-estrutura definida, aplica¸cões são programadas em duas abordagens diferentes: orientada a agentes e imperativa. Para

(12)

a abordagem imperativa, utilizou-se a linguagem de programa¸c˜ao C. A abordagem orientada a agentes foi feita em AgentSpeak.

No total, três aplica¸cões são desenvolvidas: uma para averiguar se a arquitetura de agentes funciona integrada a um VANT real, e ou-tras duas para se avaliar a coordena¸cão. Destas suas, a primeira é uma aplica¸cão simples, e a segunda é uma aplica¸cão que simula uma opera¸cão de busca em resgate em águas com VANTs. Estas duas ti-veram seu desmepenho medido e a flexibilidade foi medida entre as aplica¸cões.

Resultados e Discuss˜ao

Conclui-se que o desempenho da abordagem orientda a agentes ´

e inferior ao da solu¸cão imperativa. Porém, nenhuma das duas solu¸cões são proibitivas, visto que ambas ficam abaixo de 85% e 45% de uso de memória e processador, respectivamente, no hardware embarcado utilizado.

Por outro lado, a solu¸cão baseada em agentes se mostra mais flex´ıvel principalmente devido ao desacoplamento do comportamento individual dos agentes e da descri¸cão das missões. Enquanto na aborda-gem em agente é poss´ıvel se alterar a ordem de execu¸cão das missões (por exemplo) sem se alterar código dos agentes, a abordagem impe-rativa necessita que esta ordem este explicitamente descrita no código dos programas que comandam os VANTs.

Considera¸c˜oes Finais

Este trabalho apresenta evidencias que apontam para as vanta-gens de se utilizar a abordagem de agentes em sistemas embarcados (em especial, VANTs), mas ele não é conclusivo. Outras aplica¸cões ou métodos de avalia¸cão podem apontar para outras dire¸cões. Portanto, é necessário que mais trabalhos semelhantes sejam feitos para se levantas mais dados acerca das vantagens e desvantagens do uso de agentes.

Palavras-chave: Sistemas Multi-Agentes. Agentes BDI. VANT.

(13)

(14)

(15)

ABSTRACT

Efforts have already been made to embed agents and agent-oriented technology into robots and other systems. For this purpose, many architectures and frameworks have been developed to allow soft-ware agents to interact properly with hardsoft-ware components of theses systems. However, few proposals have evaluated a system where multi-ple agents are involved and require coordination. This thesis compares the multi-agent system approach with an agent-oriented programming language, against a more standard approach with imperative program-ming in the development of embedded systems that require coordina-tion. The experiments were performed using unmanned aerial vehicles (UAV) with embedded BDI agents.

An agent-hardware integration architecture was developed to interface the software agents with the UAV’s hardware. This architec-ture was tested on a real UAV. Two other experiments, designed to test the coordination, were performed: a simple one, to test the basic functionalities of the strategy, and another more complex, simulating a cooperative search and rescue mission. These two experiments were implemented and evaluated with hardware-in-the-loop.

Experiments’ results defend the argument that agent-oriented programming offers benefits to embedded applications, specially to those that require coordination. Even though a solution using an im-perative language can be more computationally efficient, the agent approach is also effective and offers advantages to the development process.

Keywords: Multi-Agent Systems. BDI Agents. UAV. Embedded

(16)

(17)

LIST OF FIGURES

2.1 A simple scheme and its basic elements. . . 16

3.1 SK450 UAV assembled. Image from \langle hobbyking.com\rangle . . . 21

3.2 PixHawk flight control board. Taken from \langle pixhawk.org\rangle 22 3.3 BeagleBone Black. . . 23

3.4 ESP-01, a module from the ESP8266 family. . . 23

3.5 Ardupilot’ simulator map. . . 24

4.1 Architecture’s concept with multiple UAVs. . . 26

4.2 Information flow in the agent-hardware integration ar-chitecture. . . 26

4.3 Partial UML diagram of the Java classes that compose the head. . . . 27

4.4 Generic diagram of the body’s attributes and methods. . 31

5.1 Example of a simple scheme. . . 37

5.2 Scheme board creation by agent Alice. . . 38

5.3 Bob connects to a previously created scheme board. . . . 38

5.4 Execution of a simple scheme. . . 39

5.5 Coordination with Moise. . . 40

5.6 Scheme sharing and merging. . . 41

5.7 Scheme execution with distributed scheme boards. . . . 42

5.8 Manager agent starting an application. . . 43

6.1 Agent diagram for the single UAV experiment. . . 48

6.2 Route for the single UAV experiment. . . 49

6.3 UAV during autonomous agent-commanded flight. . . . 50

6.4 Scheme of the simple coordination experiment. . . 51

6.5 Agent diagram for the simple coordination experiment. . 52

6.6 Finite state machine for the imperative simple coordina-tion experiment. . . 53

6.7 Scheme of the search and rescue experiment. . . 54

7.1 Route performed by scout UAV and the virtual victims. . 63

(18)

(19)

LIST OF TABLES

2.1 Framework comparison table. . . 20 7.1 Simple Coordination Experiment - CPU usage . . . 62 7.2 Simple Coordination Experiment - Memory Usage and Size 62 7.3 Mock Search and Rescue Experiment - CPU usage . . . . 64 7.4 Mock Search and Rescue Experiment - Memory Usage

(20)

(21)

CONTENTS

1 Introduction 1

1.1 Objectives and Method . . . 2

1.1.1 Specific Goals . . . 2 1.1.2 Method . . . 3 2 Literature Review 5 2.1 Intelligent Agents . . . 5 2.1.1 BDI Agents . . . 6 2.1.2 AgentSpeak . . . 7

2.1.3 The Jason Interpreter . . . 9

2.1.4 Architecture Customization . . . 10

2.2 Coordination . . . 11

2.2.1 Commitments and Conventions . . . 13

2.2.2 Societies of Agents . . . 13 2.2.3 Organization-Based Coordination . . . 14 2.3 Related Works . . . 17 3 Equipment 21 3.1 UAV Assembly . . . 21 3.1.1 Frame . . . 21 3.1.2 Flight Controller . . . 22

3.2 Embedded Agent’s Hardware . . . 22

3.2.1 Single Board Computer . . . 23

3.2.2 Wireless Device . . . 23 3.3 Simulator . . . 24 4 Agent-Hardware Integration 25 4.1 Embedded Architecture . . . 25 4.2 Architecture’s Head . . . 27 4.2.1 Percept Handling . . . 28 4.2.2 Action Handling . . . 29 4.2.3 Message Handling . . . 30 4.3 Architecture’s Body . . . 30 4.3.1 Packet Handling . . . 32 4.3.2 MAVLINK Protocol . . . 33 4.3.3 Inter-Agent Communication . . . 34

5 Distributed Organization-Based Coordination 37 5.1 Distributed Coordination with Moise . . . 37

(22)

5.2.1 Mission Distribution . . . 42 5.2.2 Coordination Behaviors’ Implementation . . . 44

6 Experiments 47

6.1 Single Agent Flight . . . 47 6.1.1 Execution . . . 49 6.2 Coordination Experiments . . . 50 6.2.1 Simple Coordination . . . 51 6.2.1.1 Jason Implementation . . . 51 6.2.1.2 C Implementation . . . 53 6.2.2 Mock Search and Rescue . . . 54 6.2.2.1 Jason Implementation . . . 55 6.2.2.2 Cyclic Goals with JaCaMo . . . 57 6.2.2.3 Imperative Implementation . . . 58

7 Results and Evaluation 61

7.1 Simple Coordination . . . 61 7.1.1 Evaluation . . . 62 7.2 Mock Search and Rescue . . . 62 7.2.1 Evaluation . . . 64

8 Conclusions 67

8.1 Contribution . . . 67 8.2 Limitations and Future Works . . . 68

(23)

1 INTRODUCTION

Natural disasters have always been an ongoing threat to hu-manity. A UN report (WAHLSTROM; GUHA-SAPIR, 2015) estimates that “economic losses from disasters such as earthquakes, tsunamis, cyclones and flooding are now reaching an average of US$250 bil-lion to US$300 bilbil-lion each year”. These disasters cause not just material loss, such as destruction of buildings, but also the loss of many lives: around 65.000 each year.

Brazil is one of the countries with the highest number of plu-vial disasters, such as floods and mudslides. According to (CEPED, 2013), natural disasters in Brazil have been on the rise: from more than 38.000 occurrences that happened from 1991 to 2012, 22% happened in the 1990 decade, 56% happened in the 2000 decade and another 22% happened from 2010 to 2012 alone. When floods and mudslides occur, it is not uncommon for people to be left stranded and lost amidst the destruction caused, waiting for rescue.

Search and rescue operations on disaster struck environ-ments, such as flooded or tornado-struck areas, can be very chaotic and dangerous. Besides the lack of infrastructure, such as roads or electric power transmission lines destroyed, falling and shifting rub-ble and extreme weather conditions (strong winds, water currents, etc.) pose serious hazard to rescue teams. In these scenarios, the use o robotic rescue teams can spare human workers of the risk.

The adoption of UAVs (Unmanned Aerial Vehicles) in all sorts of fields has grown in the recent years. Applications range from military to commercial and educational (FAHLSTROM; GLEASON, 2012). As the area grows, more complex problems tend to be tack-led, such as search and rescue missions. In such applications, the use of multiple robots can yield better results. In order to do so, coordination and cooperation between the robots are necessary.

In order for robots to coordinate their actions, there are sev-eral mechanisms that need to be embedded into the system. In a general sense, robots need to interact with or sense others and act accordingly. Doing that might require many capabilities, such as: communicating directly to one another, planning one’s actions, pre-dicting other’s actions, following rules, etc. To address each of these problems, the multi-agent systems approach offers many solutions (WOOLDRIDGE, 2009; WEISS, 2000).

Coordinating a team of multiple agents might include: inter-preting a plan that a user requests to be executed, splitting such plan on multiple tasks, assigning tasks to each individual, and exe-cuting the tasks in a coordinated manner. Although many solutions

(24)

2 Chapter 1. Introduction

for coordinating robots with embedded agents are available on the MAS field, research on how to integrate those solutions with dif-ferent hardware is sometimes lacking (NAU et al., 2015). Moreover, there are not many publications that compare agent-based solutions with other more usual approaches. Santos and others (SANTOS, 2015) have already shown that the agent approach has some advan-tages over a more traditional approac, using imperative program-ming, when designing applications with a single robot. To evatuate the integration of multiple agents into a distributed coordinated sys-tem is a step forward on robot coordination using MAS.

With this context in mind, this work consists of comparing the development of an agent-based solution against an imperative solution in two distinct scenarios involving UAV coordination. The research question that motivates this research is whether it is pos-sible and, if so, what are the benefits and drawbacks, in terms of performance and flexibility, of using MAS technology to coordinate UAVs.

The comparison of the multi-agent approach against a more consolidated one is important to potentially drive the multi-robot systems scenario to new technologies. Without this comparison, it is hard to assess if these systems should be modeled and programmed as multi-agent systems or if it is better to keep using the current approaches.

1.1 OBJECTIVES AND METHOD

This work has as main objective evaluate the performance and flexibility of MAS solutions for coordination in a search and rescue scenario. In this context, performance refers to the consumption of computational resources, such as memory and processor, to perform the task. Flexibility is a subjective criterion that refers to the effort it takes to add new functionalities to or change an existing applica-tion.

1.1.1 Specific Goals

\bullet Verify the agent-based solution on a real UAV.

\bullet Evaluate agent-based solutions for coordination on perfor-mance and flexibility.

(25)

1.1. Objectives and Method 3

1.1.2 Method

To achieve the objectives, various activities are performed. First, to allow the agents to command a UAV, an architecture is developed (Chapter 4). Then, the coordination method is defined and adapted to run on a multi-robot system (Chapter 5).

Once this infrastructure is set up, applications are programmed in two different approaches: agent-based and another traditional (Chapter 6). For the traditional approach, an imperative program in the C programming language is developed, as this is a very consol-idated and broadly used language on embedded applications. The agent approach is developed using AgentSpeak, as this research is a continuation of previous work (SANTOS, 2015) that used this ap-proach.

Three applications are developed in total. The first one in-volves only one UAV and serves as a validation experiment on the architecture on a real UAV. The second is tested in simulation and has the main purpose of assessing wether the coordination mecha-nism has its basic functionality working. Then, a third more com-plex search-and-rescue application is tested to better asses the capa-bilities of the system. These experiments are described in Chapter 6. No measurements and comparisons were performed on the first ex-periment as it serves only as a validation step. It is only developed on the agent-based approach as well. The second and third though have their performances measured on both approaches and the flex-ibility, how difficult it is on each approach to go from a simple co-ordination experiment to a search-and-rescue mission, is assessed between these two applications (Chapter 7).

(26)

(27)

2 LITERATURE REVIEW

This chapter presents the necessary background on multi-agent systems and coordination in order to better understand the work developed in this dissertation. It is assumed that the reader has some previous knowledge of software programming.

The chapter also provides a brief review on embedded sys-tems and UAVs, since the evaluation performed in this work is geared towards embedded systems, and UAV applications were used as experiments.

2.1 INTELLIGENT AGENTS

There is no consensus on a universal definition for “intelli-gent a“intelli-gents”, but a good definition for the term is presented on (WOOLDRIDGE, 2009): “an agent is a computer system that is situ-ated in some environment and that is capable of autonomous action in this environment in order to meet its design objectives”.

According to (WOOLDRIDGE, 2009), agents have the ability to influence their environment by performing actions, through their effectors. Usually they have only a partial control of their environ-ment: the effect of their actions might not be guaranteed (hence the word “influence”). Agents also have percepts about their envi-ronment. As the environment changes, they should perceive at least some of those changes in order to react accordingly.

According Wooldridge and Jennings (WOOLDRIDGE; JEN-NINGS, 1995), rational agents posses the following characteristics:

\bullet Autonomy can be understood as: the ability to “act without the intervention of other humans or other systems” (WEISS, 2000). Autonomy is one of the central points regarding agents, and also one of its main advantages over other approaches to designing systems. Agents can reason by themselves (WEISS, 2000) and react accordingly to unexpected events without the programmer having to explicitly tell it what to do.

\bullet Social Ability refers to the capacity of an agent to “interact with other agents (and possibly humans) via some kind of [...] language” (WOOLDRIDGE; JENNINGS, 1995). Agents might communicate not only by exchanging raw data, but might also ask for information or even command other agents to act in a certain way. This emcompasses the “ability of agents to

(28)

6 Chapter 2. Literature Review

erate and coordinate activities with other agents” (BORDINI et al., 2007).

\bullet Reactivity refers to the capability of an agent to act according to what it perceives in the environment, to directly react to changes in the environment.

\bullet Pro-activeness, on the other hand, is what is behind the ac-tions that do not come from a direct mapping of the percepts. The ability to “exhibit goal-directed behavior” (BORDINI et al., 2007), to “take the initiative”.

According to Weiss (WEISS, 2000), there are many architec-tures that implement agents. Among them, there are:

\bullet Logic-based agents: decisions are derived through logical de-duction.

\bullet Reactive agents: decisions come from a direct mapping from situations to actions.

\bullet BDI agents: decision making depends upon manipulation of internal data structures that represents beliefs, desires and in-tentions of the agent.

2.1.1 BDI Agents

BDI agents have three main structures that are used to de-fine their program (or “mental state”): beliefs, desires and inten-tions. This software model is based on the works of Bratman on hu-man reasoning (BRATMAN, 1987). According to (RAO; GEORGEFF, 1995), each one of these can be described as follows:

\bullet Beliefs: representations of what the state of the world is to the agent, including itself. In other words, what the agent believes is true about the environment. It might be imprecise or even incorrect.

\bullet Desires: states that the agent might like to be in. The agent might not necessarily act upon all of these. In fact, some states might be mutually incompatible.

\bullet Intentions: states that the agent acts towards achieving. It might be a goal, delegated by an external source, or a result of the agent’s own choosing.

(29)

2.1. Intelligent Agents 7

To use these structures to perform actions, a BDI agent can use a model of decision making known as practical reasoning (BOR-DINI et al., 2007). Practical reasoning consists of two activities: de-liberation and reasoning. Dede-liberation is the process of choosing what objectives should be pursued. In the BDI architecture, this translates to selecting intentions to pursue. Reasoning (or means-end reasoning) is the process of figuring out how to achieve those objectives. It is also known as planning. To the agent, this means to select a set of actions that lead to the accomplishment of the inten-tion. The actions that lead to achieve an intention, that lead from the current state to the desired one, is known as a plan. The reason-ing process then selects plans that must be performed to achieve the agents intentions.

Thus, an implementation of a BDI agent is a program that continuously executes this practical reasoning. The steps executed by it can be roughly listed as follows:

1. Perceive the world and update the beliefs;

2. Select a suitable desire to “promote” to intention;

3. Use means-end reasoning to find a plan to achieve the inten-tion;

4. Execute the plan;

2.1.2 AgentSpeak

In order to program a software agent, a suitable language must be used. For BDI agents, there is AgentSpeak: an agent-oriented programming language (BORDINI et al., 2007). The main concepts pertaining to BDI agents, such as beliefs, can be expressed directly in the code with AgentSpeak.

In AgentSpeak, beliefs are represented as logical predicates, in a declarative manner. blue(car), for example, represents the belief that a car is blue, attached(uav, camera) can represent a belief that a UAV has a camera attached to it, and so on. It is worth reminding here that those do not represent the true state of affairs, only what an agent believes to be true.

Another essential concept for BDI agents are the goals. They represent the properties that the agent wishes to be accomplished. In AgentSpeak, there are two types of goals: achievement and test goals. Achievement goals are denoted by the “!” operator, and mean

(30)

that the agent work towards that state. E.g.: !put(book,table) represents the goal that the agent has of putting a book on a table. Test goals are denoted by the “?” operator and represent a condition that the agent should assert if it is true or not. E.g.:?apples(5) will

assert if the agent believes it has 5 apples or not.

Furthermore, test goals can be used to retrieve information from the agent’s belief base. apples(N) will check if there is any apples belief and will “assign” its value, enclosed in parenthesis, to the variable N. Just as in Prolog, lower-case identifiers (“apples”) and numbers are atoms, or logical “constants”. Upper-case identi-fiers (“N”) are “logical variables”. These variables are initialized without value. What “bounds” the variables to a particular value is the operation known as unification. Once unified, they keep that value throughout their scope.

Plans are also an essential concept for BDI agents. They rep-resent the agent’s know-how: what is it that the agent knows how to do. Changes in the agent’s beliefs or goals are what triggers them. A plan has a trigger, a context (together they are the head) and a body. The parts are separated by “:” and “<-”. E.g.: triggeringEvent : context <- body.

In short, triggering events are events that happened during an agent’s execution. Two of the most commonly used events are: belief addition (e.g.:+apples(2)) and achievement-goal addi-tion (e.g.:+!put(apple,table)). Whenever the specified belief is added to the belief base, or the specified goal is added, the plan should be executed.

The context defines a state of things that must be true for the plan to be triggered. For example, a plan that contains +!run : dangerwill only actually be executed if the agent believes it is in danger. Unification is possible in the context as well. +!buy(Item) : need(Item) would only allow the agent to buy an item if it believes some item is needed.

The body of a plan defines a sequence of actions that must be performed once a plan is selected for execution. A plan then can be something simple such as +!run : danger <-setSpeed(fast). Predicates in the body of a plan are inter-preted as actions. So, in the last example, setSpeed(fast) would be interpreted as the action of setting the speed to “fast”. Plans can also generate goals, much like a routine can call sub-routines in other languages: +!run : danger <- +running; !getToSafety would create another goal (getToSafety). In this case, the first goal would only be accomplished if its sub-goal is

(31)

2.1. Intelligent Agents 9

also accomplished.

There is much more on AgentSpeak than what is presented here, but this quick revision should provide a rough understanding to the reader of the most essential aspects of AgentSpeak program-ming. More about Jason agent’s programming can be found on the related literature (BORDINI et al., 2007).

2.1.3 The Jason Interpreter

After an agent’s program has been created using AgentSpeak, it should be possible for an interpreter to read the code and execute an agent with the behavior described. To do so, the interpreter run in an endless loop, also known as reasoning cycle. In the case of the Jason interpreter, its cycle is similar to the simple one already presented in 2.1.1, but more elaborate (BORDINI et al., 2007):

1. Perceive Environment: the agent gathers information about its environment, as specified in its architecture;

2. Update Belief Base: the agent updates its belief base by remov-ing the existremov-ing perceptual beliefs and addremov-ing the new ones; 3. Receive Messages: gets the messages that were sent by other

agents;

4. Select Message: selects one message to read; 5. Select Event: selects one event to process;

6. Retrieve Relevant Plans: retrieves the plans that are triggered by the event;

7. Determine Applicable Plans: checks which plans have a context in accordance with the agent’s beliefs.

8. Select Applicable Plan: from the applicable plans, select one to execute, i.e.: to become an intention;

9. Select Intention to Execute: from the active intentions, select one to execute;

10. Execute Intention’s Step: execute one step of the selected inten-tion.

(32)

After this cycle is executed, the interpreter does some “house-keeping” before proceeding to the next cycle. It checks if there is any kind of feedback pending (e.g.: actions usually wait for a feedback signal before they are considered “done”), removes completed inten-tions and re-instantiates those that were somehow on hold (such as waiting for a sub-goal to be achieved).

Although the list presented in this section describes the se-quence of steps performed by the interpreter to execute an agent’s program, it leaves a few “loose ends”. For example, there is no ex-planation on how an agent should get percepts from or exert actions onto an external world, be it physical or virtual. Those details are defined by an agent’s architecture: a customization of the agent’s functionality that lets it perform those interactions.

2.1.4 Architecture Customization

Every agent in Jason has an architecture lying underneath it. During the reasoning cycle, it calls the methods pertaining to this architecture to perform each step (e.g.: when it is time to perceive the environment, it calls the perceive method from the architecture). To run a Jason project, besides the agent’s program, a configura-tion file is also necessary. This file states which agents should be instantiated, in which environment (they can run on the same ma-chine but on different irtual environments, for example) and with which architecture. There is already a standard implementation of the architecture, but it can be customized to great extents. A simple explanation about this procedure is given in this section, but more information can be found on the book on the matter (BORDINI et al., 2007) an in Jason’s manual (BORDINI; H ¨UBNER, 2007).

To customize the agent’s architecture is to write a Java class (since Jason is built on top of Java) that overrides the standard methods. E.g.: the method perceive(), that does not do anything by default, can be overridden to read data from a sensor or to get information from the internet. Among what can be changed, there are methods responsible for perception, action, sending messages and receiving messages.

Besides the architecture’s methods, there is another very use-ful way of adding functionality to the agent: the internal actions. Internal actions are methods that can be used in the agent’s pro-gram to perform various tasks. A common example is to calculate the distance between two points in space. This is a kind of task that can be written in the declarative manner, using AgentSpeak, but

(33)

2.2. Coordination 11

other languages might provide a simpler way of doing the same. So, instead of writing a sequence of rules that eventually unify with the distance, a programmer might write a library with the method distance()and call it by using the name of the library, a point, and the name of the method. E.g.: math.distance(A,B,D), with “A” and “B” being the points and D the variable that will unify with

the distance in the program.

2.2 COORDINATION

In a general sense, “coordination is a property of a system of agents performing some activity in a shared environment” (WEISS, 2000). Whenever there are different entities working in a shared environment, the whole system might hold some degree of coordi-nation that is proportional to how much it reduces resource con-tention, avoids deadlocks, maintains safety conditions, etc. Accord-ing to O’Hare and JennAccord-ings (O’HARE; JENNINGS, 1996), there are three main reasons why the actions of multiple agents should be coordinated:

\bullet There are dependencies between agents. Decisions made by one agent have an impact on decisions of another community member;

\bullet There is a need to meet global constraints. The solution devel-oped by a group of agents must satisfy certain conditions to be deemed successful;

\bullet No individual has sufficient competence, resources or informa-tion to solve the entire problem.

One way of ensuring coordination between the parts of a team is to make them all subject to a single global controller. One individual receives information about the whole system and then commands the others. This simple approach introduces a few bottle-necks though: it might be too demanding on the controller’s comput-ing power to make decisions for others; the communication band-width might not be capable of sharing that much information; and it introduces a single point of failure: the controller (O’HARE; JEN-NINGS, 1996). To make the system more robust and less demanding on a single element, decentralized approaches are more appropri-ate.

(34)

If there is no single entity that commands every action or op-eration of the team, the system is called decentralized. When this is the case, the coordination mechanisms used cannot be exactly the same as in centralized cases. On decentralized applications, each in-dividual is responsible for the solution of its own local coordination problem. The individuals ponder and act based on an uncertain per-spective of the environment. Conflicts of interest sometimes arise from those uncertainties. To solve these problems there are vari-ous tools developed in the Multi-Agent field, such as task and re-sult sharing, commitments and conventions, social norms and laws (O’HARE; JENNINGS, 1996).

In order to solve possible conflicts between agents, it is useful to know how well their objectives are aligned. When those agents are non-antagonistic, coordination can be characterized as coopera-tion. When those agents act competitively, or in simple self interest, coordination becomes negotiation (WEISS, 2000). It might be the case that, on a given system, any agent might be in accordance with some number of agents and in conflict with others. For example, one agent might need to abort a task that another was depending on. Therefore, some amount of both cooperation and negotiation might be necessary for coordination.

With the need for coordinating task’s execution between agents, comes the need for communicating with each other. There are different aspects of sharing information. One of the basic clas-sifications is perhaps on what agents share. Two known classes are result sharing and task sharing (O’HARE; JENNINGS, 1996; WEISS, 2000). Result sharing basically means that states and results of ac-tions will be shared between agents. Task sharing, on the other hand, is the act of assigning (sub)tasks to other agents. Mechanisms or protocols for coordination might be focused on result sharing, on task sharing, or even have some of both (WOOLDRIDGE, 2009).

Some mechanisms are presented in this section. They do not represent an extensive list of coordination mechanisms, but repre-sent a selection of popular mechanisms for coordination in the field. Before advancing onto specific mechanisms, it is important to emphasize that there are other approaches to coordination that come from other fields, such as control theory and optimization based ones (GAZI; PASSINO, 2004; OGREN et al., 2004). But this work focused on coordination from the perspective of multi-agent systems, and only a few mechanisms from this field are exposed.

(35)

2.2.1 Commitments and Conventions

One of the cornerstones of coordination are commitments and conventions. “Commitments are viewed as pledges to under-take a specified course of action, while conventions provide means of managing commitments in changing circumstances” (WEISS, 2000). When those conventions refer to other agent’s commitments, they can be called social conventions. In other words, commitments are representations of what actors intend to do in the future. Conven-tions are rules that determine how actors should treat those com-mitments. For example, when is it allowed to drop or disregard a commitment.

By exchanging commitments and managing them with con-ventions, agents gather the much needed information that they need about the world that cannot be directly perceived by them. Information about if or when an activity will be performed is very important when tasks have interdependency. For example, if the prerequisite of an action needs to be provided by another agent, but there is no commitment that indicates that such action will be performed, an agent can drop the execution of said action, instead of waiting indefinitely for something that may never happen.

Commitments can also be joint commitments. When two or more actors pursue the same objective, they can make a joint com-mitment on it (O’HARE; JENNINGS, 1996). If agents are in a joint commitment, keeping them all with a common, more accurate, vi-sion of the world is useful. They can share information between themselves to help one another, but without broadcasting to non-participating agents.

Commitments and conventions provide a powerful infrastruc-ture for coordination. In fact, some argue that all coordination mechanisms can be reduced to or interpreted as a version of this model (Centrality of Commitments and Conventions Hypothesis) (O’HARE; JENNINGS, 1996).

2.2.2 Societies of Agents

Groups or teams of agents are similar to human societies in the sense that each individual has to interact with an environment in order to achieve one’s objectives. By doing so, it interacts with other individuals and becomes part of the environment. Therefore, it makes sense to model multiple agents as a society (WEISS, 2000). On these societies, agent are associated with social groups

(36)

that determine a social context, with possible roles associated with them. Roles are parts that agents play on a social context. With a role, comes permissions and restrictions on actions, as well as re-sponsibilities: actions that the agent is obligated to perform. These rules are sometimes known as social laws. They help to avoid unde-sirable states of the environment (WEISS, 2000), and can be role-specific or general to a whole social context. This way, a social be-havior is associated with each role in a society (WEISS, 2000).

Roles can have a relationship of hierarchy. Some roles can be superior to others and directly designate them to perform tasks (O’HARE; JENNINGS, 1996). When the actions of a role depend on the actions of another role, they have a social dependence. Cooper-ation in this context becomes a relCooper-ationship of mutual dependency (WEISS, 2000).

Besides offering a more familiar context, modeling teams with multiple individuals as societies helps to define the desirable behaviors of a system. When multiple programmers work together, if all of the designed agents follow the same laws, it becomes easier to predict what others may or may not do, simplifying the system design (O’HARE; JENNINGS, 1996).

2.2.3 Organization-Based Coordination

The concept of the organization derives from the society of agents. An organization in the real world is very much like one in the MAS world: “a set of entities regulated by mechanisms of social order and created by more or less autonomous actors to achieve common goals” (DIGNUM; DIGNUM, 2001). It is a set of behavioral constraints that, if correctly designed, should allow the agents to coordinate themselves.

\scrM oise+_{is an organizational model that considers the}

struc-tural, functional and deontic relations between agents. With \scrM oise+_,

an organization can be specified independently from the individual behaviors of the agents (H ¨UBNER et al., 2002).

A software that implements this organizational model is \scrS - \scrM oise+ _{(or simply Moise) (H ¨}_{UBNER et al., 2005). In it, the}

organizational specification is defined by three sub-specifications: structural, functional and normative. Each of them serves a specific purpose:

\bullet Structural specification - describes roles that agents might have in the organization and how they relate to one another: if they

(37)

are divided in sub-groups, which roles have authority over which roles, how many agents with each role can exist, and more.

\bullet Functional specification - defines plans and goals that the agent should try to achieve, as well as their characteristics: how many agents each goal takes, in what order they need to be performed, if the goals can be divided into missions, etc. \bullet Normative specification - describes norms and rules that the

agents should adhere to: which missions needs to be formed and by which agent, what are the obligations and per-missions that they have, and others.

In this work, the functional specification is much more rel-evant than the others. Therefore, it shall be explained more thor-oughly. More information about the other specifications can be found in (H ¨UBNER et al., 2005).

The functional specification defines plans and goals that the agent should try to achieve, as well as their characteristics: how many agents each goal takes, in what order they need to be per-formed, if the goals can be divided into missions etc.

According to (H ¨UBNER et al., 2002), the functional specifica-tion is composed of a collective global goal and “how these goals are decomposed (by plans) and distributed to the agents (by missions)”. In this context, plans are descriptors of relationships between sub-goals. If certain goals need to be accomplished in a sequence, for example, this is described by a plan. Missions are sub-sets of goals. The agent that commits to a given mission should pursue all of the goals in that mission.

Moise can be used in conjunction with Jason so that agents that join an organization follows these specifications and are able to work in coordination. A framework that groups both os these soft-ware is JaCaMo (BOISSIER et al., 2013). “JaCaMo is a framework for Multi-Agent Programming that combines three separate tech-nologies, each of them being well-known on its own and developed for a number of years so they are fairly robust and fully-fledged” (BOISSIER et al., 2013). It is a combination of Jason (BORDINI et al., 2007), CArtAgO (RICCI et al., 2009) and Moise (H ¨UBNER et al., 2005). This combination of tools extends the functionality of the Jason interpreter. CArtAgO allows Jason agents to interact with artifacts (software objects with less autonomy than an agent) and

(38)

Figure 2.1: A simple scheme and its basic elements.

Moise provides a coordination insfrastructure based on organiza-tions.

To make use of Moise in JaCaMo, the organizational specifi-cation has to be provided: a file that describes the intricacies of the organization that the agents will try to conform to. If it is provided, whenever an application is launched in JaCaMo, any agent can create an organization by providing the specification as argument. Then, they can create an artifact called scheme board, which can be used by the agents to monitor the organization and the progress of the missions.

During execution, agents can try to commit to missions de-scribed in the organization. The scheme board checks if the mis-sion is compatible with the request (e.g. if there is still room for more agents), and then registers the commitment or fails the ac-tion. The scheme board then will track the progress of the missions and warn agents when they should start pursuing each goal, en-suring the plans are correctly executed in the pre-defined order. In short, this is how agents interact with Moise to perform missions in a coordinated fashion.

As an example, consider the scheme shown in figure 2.1. In it, to accomplish the global goal execute, all goals, from G1 to G4, have to be accomplished. Those goals have a dependence relationship among themselves: G4 depends on G3, that depends on G2, that depends on G1. This means that each cannot begin if the previous was not accomplished previously. At last, there are two mission: Ma, with G1 and G3, and Mb, with G2 and G4.

Consider now two agents that are supposed to accomplish this global goal. As agents A and B start their execution, an orga-nization is created using the scheme in figure 2.1. Both join the

(39)

2.3. Related Works 17

organization and each commit to missions Ma and Mb, respectively. Agent A should be free to start pursuing G1, but agent B should not be able to start any of its objectives without A accomplishing at least G1. A cannot start pursuing G3 without B accomplishing G2 first. What makes sure that they follow the proper sequence is the scheme board, that checks if the conditions for pursuing each objec-tive are met and triggers desires on the committed agents. This way, the agents do not have to know before-hand when each objective should be executed: the scheme board decides what should be done and coordinates the agents.

2.3 RELATED WORKS

Before exploring works related to what this work is, it is also important to show what this work is not, as coordination is a broad subject. As Doriya, Yan and their colleagues show in their respective surveys on multi-robot coordination (DORIYA et al., 2015; YAN et al., 2013), the subject has many different dimensions to it. To name a few of those dimensions:

\bullet Cooperation X Competition: whether the robots in the applica-tion all cooperate among themselves to accomplish a common goal or compete for resources in a shared environment; \bullet Explicit X Implicit communication: whether the robots

explic-itly exchange messages among themselves or search for clues and signs that others leave on the environment to acquire in-formation about the state of others.

\bullet Centralized X Decentralized: whether all robots depend upon a central authority to make decisions and enforce coordination or if each individual makes its own decisions.

\bullet Static X Dynamic: whether rules or conventions for coordina-tion are defined prior to execucoordina-tion or are generated during ex-ecution, from information gathered through communication.

This work explores cooperative applications, the communica-tion used is explicit and the coordinacommunica-tion strategy is decentralized and static. It is not about any of those characteristics though. This work is focused on the programming paradigm used to implement a coordinated application. In a way, these dimensions are orthogonal,

(40)

as the same paradigm can be used to implement different varia-tions of the other characteristics. Therefore, this work is parallel to a plethora of works on multi-robot coordination, that approach this same subject from other directions.

On the MAS field, the adoption of agent-oriented program-ming on autonomous robots is not a novel idea. The paradigm has been applied in many different ways. Some have modelled single robots as multi-agent systems (MAS). Others have proposed frame-works and tools for programming robotics systems. Most of the works were tested on simulation only, while some others have been validated with a hardware-in-the-loop approach.

There are many tools and frameworks that enable a com-puter agent to command or interact with robotics systems. Wei and Hindriks (WEI; HINDRIKS, 2013) have developed an architec-ture that allows GOAL (HINDRIKS, 2011) agents to interact with a robotic hardware. This research was focused on mobile robots, and thus had features specific to mobile vehicles’ applications, such as navigation-related modules. This architecture mainly consisted of a middle-ware that connects the high-level layer (the agent) with a low-level (hardware interface). This modularity is a design pattern that has been repeated itself in other frameworks. The architecture was then tested on the humanoid robot Nao (GOUAILLIER et al., 2009) in an indoor environment.

For the AgentSpeak interpreter Jason (BORDINI et al., 2007), there is an architecture called ARGO (PANTOJA et al., 2016). It consists of a middle-ware that enables Jason’s agents to commu-nicate with low-level hardware (micro-controllers) by using a li-brary called Javino (PANTOJA; LAZARIN, 2015). In this context, the agent is assumed to be executing on a machine with considerable performance, such as a personal computer, and performing actions or receiving percepts through another lower-level hardware, such as an Arduino, which is actually in charge of the actuator or sensor. While ARGO assumes that the low-level control happens in a separate, lower-level hardware, JaCaROS (WESZ, 2015) is a tool that connects the cognitive agents of Jason to the Robot Operating System. ROS is a very versatile tool for robot control and simulation. JaCaROS then provides a framework that connects agent to either a real environment (through hardware control) or a simulated one, as long as the machine is capable of running ROS.

Fichera and colleagues described PROFETA (Python RObotic Framework for dEsigning sTrAtegies) (FICHERA et al., 2017). The framework uses the cognitive agent model and implements

(41)

AgentS-2.3. Related Works 19

peak(L) in Python. This framework, being developed in a mono-lithic platform, does not have the same layered architecture as the other works presented here so far. The author argues that there is one big advantage in using an integrated framework: the developer does not have to deal with different execution environments, one for low and other for high-level logic, making the system easier do debug and maintain.

SOIFRA (AROKIASAMI et al., 2016) is a framework for mo-bile robot autonomy. It allows for JADE (BELLIFEMINE et al., 2001) agents to control the robot on a behavioral level. The framework already has object detection and collision avoidance built into it and accepts other independent algorithms. The software provides modularity through a service-oriented architecture.

Santos and others (SANTOS, 2015) have shown that it is possible to use Jason on an embedded system (BeagleBone Black board) in a UAV context. The result of this work has shown that there are advantages of using an agent oriented programming lan-guage to develop UAV applications when compared to, for example, the imperative paradigm (using C), a more traditional approach. Namely, the main advantages were on the size of the program (size in Kbytes of the source files) and on how easy it is to imbue pro-activity into the program and to modify its behavior later on. Among the disadvantages, there is an increase in CPU usage for the agent-oriented approach, when compared to a traditional one.

The agent’s framework publications presented so far are clas-sified here according to four criteria, identified by the following questions:

\bullet Are they tested on embedded systems?

\bullet Do they use an agent oriented programming (AOP) language? \bullet Do they cope with more than one agent?

\bullet Do they provide a coordination tool or strategy?

Table 2.1 presents the classification results. As the table shows, none of the aforementioned publications address the issue of coordination between multiple agents by default. Although the solutions proposed could perhaps work in a coordinated fashion, there are no infra-structure already developed to address this issue. There are no ready-made tools for coordination. This aspect, if im-plemented, would rely on an ad-hoc solution. In this dissertation, all of the criteria presented are met. This project was developed

(42)

20 Chapter 2. Literature Review Framework Embedded? AOP Language? Multi-Agent? _{Coordination?}

GOAL yes yes no no

ARGO yes yes no no

JaCaROS no yes yes no

PROFETA no no no no

SOIFRA no no yes no

Jason, by Santos yes yes no no

Table 2.1: Framework comparison table.

and tested with hardware-in-the-loop, uses an agent oriented pro-gramming language, and addresses multi-agent coordination.

It is worth noting that there is a lack of works in the literature that compares the AOP approach to the traditional approach for em-bedded systems. One of the reasons is that any work that proposes to make such evaluation needs to implement a solution in both paradigms. Implementing a solution in two different paradigms might be too cumbersome.

(43)

3 EQUIPMENT

This project involves embedded hardware on UAVs. The equipment used was already pre-defined since previous works (SANTOS, 2015). They involve a single-board computer, to execute the agent’s program and architecture, and a fully assmenbled UAV. A brief description of what was used is given in this chapter.

3.1 UAV ASSEMBLY

A common hobby UAV, available online for purchase, was used. It is usually controlled with an RF controller, but it is also pos-sible to perform autonomous flight with it, including take-off and landing, if a GNSS receiver is available. It runs on Li-Po batteries that can give it an endurance of aproximately one hour, depending on the battery’s capacity.

3.1.1 Frame

The frame used is commonly known as the SK450 quad copter. It comes with four motors, one on each arm, as well as one electronic speed control (ESC) device for each. Propellers and RF receiver are also provided. A picture of the assembled UAV can seen on Figure 3.1. The UAV has already a flight controller attached to it in the center. For this project, a different flight controller board was used.

Figure 3.1: SK450 UAV assembled. Image from \langle hobbyking.com\rangle .

(44)

22 Chapter 3. Equipment

3.1.2 Flight Controller

The flight control board (FC) is the “brain” of the common UAV. It is responsible for interpreting the signals from the RF re-ceiver and determining the speed of each motor to achieve the atti-tude requested. Depending on the mode of flight, it might also be responsible for stabilizing the UAV on air. With a GNSS device at-tached, it can use global coordinates as reference, and fly to specific positions pre-determined.

The FC used in this project is the PixHawk (Figure 3.2). De-vices can interface with it by exchanging MAVLINK1packages. With this technology, it is possible to give commands (e.g.: take-off, go to a specific point, land) to the FC and extract information from it (e.g.: position, battery level). This allows another entity to command the UAV without having to worry about its dynamics and control.

Figure 3.2: PixHawk flight control board. Taken from \langle pixhawk.org\rangle

3.2 EMBEDDED AGENT’S HARDWARE

To embed the agent in the UAV, a companion embedded com-puter (EC) was used. EC and FC communicate via serial interface. To provide communication between agents (between ECs) for the multi-agent scenarios, a wireless device was considered. Since the coordination scenarios were tested in simulation, the device was not used. Its behavior was simulated though, as will be shown in Chapter 4, to facilitate compatibility of systems developed with the device in the future.

1_{More about the MAVLINK protocol can be found at:} _{\langle http://}

(45)

3.2. Embedded Agent’s Hardware 23

3.2.1 Single Board Computer

For the EC, the BeagleBone Black 2 _{single board computer}

was used. It has 512MB of RAM and a 1GHz processor. Debian Wheezy was the operating system chosen, installed on an 8GB mi-croSD card, used for storage memory. A picture of the board can be seen on Figure 3.3. The board can be accessed by a common desktop computer via secure shell over USB.

Figure 3.3: BeagleBone Black.

3.2.2 Wireless Device

The wireless device considered for inter-agent communica-tion was the ESP8266 3. It consists of a 32-bit microcontroler (80MHz), with 64KBytes and 96KBytes of RAM for instructions and data, respectively. It is capable of sending and receiving information via Wi-Fi. The device also has a serial interface. A picture of one of its variant can be seen on Figure 3.4.

The device allows the creation of ad-hoc mesh networks among other Wi-Fi devices, which makes it a good candidate for applications with no central point of access defined.

Figure 3.4: ESP-01, a module from the ESP8266 family.

2_{More about the BeagleBone Black can be found at:} _{\langle http://}

beagleboard.org/\rangle

3_{A datasheet with more information about the ESP8266}

can be found at: \langle https://www.espressif.com/sites/default/files/ documentation/0a-esp8266ex datasheet en.pdf\rangle

(46)

24 Chapter 3. Equipment

3.3 SIMULATOR

In order to simulate physical UAVs on the coordination exper-iments, the Ardupilot’ Simulator4_{was used. It simulates the logical}

and kinematic behavior of an aircraft. By establishing a socket con-nection on the right port, any software using the MAVLINK protocol can communicate with it, sending command and acquiring informa-tion, such as position and battery level, just like a commercial FC. It only simulates one aircraft at a time, but multiple instances of the program can be launched with different access ports, so many agents can connect with a different simulated aircraft on the same machine. Figure 3.5 shows the map generated by the simulator.

Figure 3.5: Ardupilot’ simulator map.

4_{Documentation about the Ardupilot Software-In-The-Loop}

simulator can be found at: \langle http://ardupilot.org/dev/docs/ sitl-simulator-software-in-the-loop.html\rangle

(47)

4 AGENT-HARDWARE INTEGRATION

As shown in Chapter 2, agents are entities that perceive and act upon their environment. Jason (BORDINI; H ¨UBNER, 2007) and JaCaMo (BOISSIER et al., 2013) assume that an agent might need to perform these activities, but how they perform them might change drastically among different applications. For example, while a mobile robot agent might perceive its position through different sensors and act upon motors to move around, a stock trading agent might need to acquire asset’s prices and generate purchase orders on the stock market’s web interface. Therefore, the framework al-lows the customization of architectures that integrate the agent with its environment.

In this project, the agent had to be integrated with the hard-ware used on the UAV: the flight control board (FC) and the Wi-Fi module. In this chapter, the architecture conceived for this purpose is presented in detail.

4.1 EMBEDDED ARCHITECTURE

As Jason’s manual (BORDINI; H¨UBNER, 2007) explains, the agent’s architecture is a Java class that defines new methods to be called by the interpreter during execution. E.g.: to perceive, the in-terpreter calls the architecture’s perceive() method during its reasoning cycle, and whatever is defined within that method should gather the apropriate information to be presented to the agent.

In general, an agent architecture for embedded systems must be able to provide access to hardware devices for the agent. These devices include sensors (to gather percepts) and actuators (to inter-act with the environment). For multi-agent applications, it is neces-sary to have access to communication devices as well. The concept of how the agents interact with each UAV is depicted in Figure 4.1. To attend these requirements, it is desirable that the programming language in which the architecture is written has libraries to access the devices used in the project. In special, for this application, the agent has to handle MAVLINK packages to interact with the UAV’s FC.

Even though Java is a well consolidated programming lan-guage, there are other languages that are better suited to interact with hardware in an embedded scenario. E.g.: there are libraries (or modules) for handling MAVLINK packages in Python that do not ex-ist in Java.

(48)

26 Chapter 4. Agent-Hardware Integration

Figure 4.1: Architecture’s concept with multiple UAVs.

Figure 4.2: Information flow in the agent-hardware integration ar-chitecture.

To try to take advantage of the power of different program-ming languages, the whole agent integration middleware was split in two parts: The head, written in Java, that customizes the required Java methods, and the body, that does the interface with the hard-ware. The head itself is divided in two layers: cortex and bulb.

The information flow between the parts is depicted in Fig-ure 4.2: information comes as percepts from the environment or as a message from another agent, goes through the body to the cortex via the bulb; Then, after the agent’s reasoning cycle, it flows back to the body as a message do be sent to another agent or as an action to be performed. The body then executes the will of the agent, either by sending the message or executing the action. More details about how these interactions work are presented in this chapter.

(49)

4.2. Architecture’s Head 27

4.2 ARCHITECTURE’S HEAD

The architecture’s head itself is divided in two pieces: the cor-tex, a Java class called EJasonArch that extends and overwrites

some Jason agent’s methods, and the bulb, a Java class called Jason-Bulb that handles the interface with the body. A class diagram with

the most relevant attributes and methods is shown in Figure 4.3.

Figure 4.3: Partial UML diagram of the Java classes that compose the head.

The cortex extends the AgArch class, as specified in the Ja-son manual (BORDINI; H ¨UBNER, 2007). In this architecture, the agent launches the bulb as a thread during initialization, that con-nects itself via socket, as a client, on a predefined IP and port. The body, which is run beforehand, is presumed to be functioning as a server and waiting to accept this connection. The bulb is constantly waiting for an incoming packet from the body. Whenever a packet arrives, it gets passed to the cortex. Whenever the cortex needs to communicate with the body, is calls a method from the bulb to do so. This leaves the cortex free to run its reasoning cycles, while the bulb handles the reception of packets, which is usually a blocking process.

In other words, the cortex handles the extended functionali-ties of the agent while the bulb manages the connections and packet exchange with the body. The separation in cortex and bulb is impor-tant because it helps the agent to save processing power. On some systems, due to the nature of the application, information updates

(50)

28 Chapter 4. Agent-Hardware Integration

come at a low rate. It would be a waste of computing power and en-ergy to leave the agent going through reasoning cycles with nothing new to consider. Therefore, when there is nothing new to consider, the Jason agent “sleeps”: its execution is interrupted. To keep the agent responsive, able to immediately process new information, in-stead of waiting for some sort of “sleep cycle” to finish, the bulb, which is always busy waiting for new messages, “wakes” the cortex up whenever there is new information to be processed. This allows the program to consume less energy when there is nothing to be done, but able to quickly cope with new information whenever it arrives.

4.2.1 Percept Handling

The body runs on a timed cycle. Whenever the body receives information from a sensor, it builds a packet containing the beliefs associated with that data. It then sends a list of updated percepts at cycle. The bulb identifies the percepts and stores them on a list, which is an attribute of the cortex class. When the cortex updates its percepts, it reads this list and register them as new beliefs. Old per-cepts are always thrown away, as the body always send a complete list of percepts. This prevents the belief base of growing indefinitely due to the percepts. E.g.: for percepts such as room temperature and position, the agent could receive the following packet:

temperature(25);position(10,20);

which would be identified by the bulb as a percept packet, separated into beliefs and stored on the cortex’s percepts list. On the next rea-soning cycle, the cortex would erase the existing percepts off the belief base and add the most recent ones, from the list.

As (STABILE M.F., 2015) puts it, when an agent gathers per-cepts, if a system is not properly designed, there is a possibility of overwhelming the agent with too many percepts. If the agent has too many percepts to deal with, or if it receives percepts at a very fast rate, it might spend too much processing power and memory managing them. This problem can be mitigated by percept filters.

There are two ways to implement them in these filters in this architecture. One of them, and perhaps the easiest way, is to limit percepts at the body. This approach will be better explained later. But another possibility at the head level is to customize a class called “PerceptFilter”. This class has a method filter() that takes two lists

(51)

4.2. Architecture’s Head 29

of percepts. When the bulb interprets the percepts, it passes both the last percepts and the new ones through this method. The standard implementation offilter() is to simply return the newer percepts. If

the programmer wishes, the class can be modified to implement any sort of filter. E.g.: only accept a newposition(X,Y) if the position

has changed more than Z meters.

4.2.2 Action Handling

The architecture over-writes the agent’s actions. For the agent to be able to execute both actions in and out of the framework, a distinction between them was created. For actions that are executed within the virtual environment of JaCaMo (such as using CArtAgO artefacts), no change is required. Actions that are directed towards hardware, external to the framework, should use the name space “ext”. E.g.: “ext::takeOff” is interpreted on the architecture as an

external action, and it is different from “takeOff”, which would try

to execute an action inside the virtual environment of the frame-work.

Whenever the agent performs an external action, a packet is sent to the body with an identifier and that action’s name. E.g.:

!takeOff, with “!” being the identifier. The body is supposed to read

the packet, perform the action and send the same packet back if it was performed successfully. Notice that this way, there is no need to describe what an action does on the agent-level. Every action becomes a packet. Its description, the code that describes what per-forming that action means, needs only to be written where it is needed, in the body’s program.

An important aspect of handling actions this way is that there is no guarantee of the action’s effects. The reception ofext::move’s

confirmation only mean that a package was sent to the body, dealt with somehow, and sent back to the agent. It does not mean, for example, that the agent actually moved. The expected effects of an action are not guaranteed by its execution. To program an agent that expects a result from an action, that behavior should be explic-itly programmed. For example, if it is necessary to know whether an agent has moved, the agent could have a behavior that verifies if its position has changed.