D
A Middleware for
Mobile Edge-Clouds
João Filipe Rodrigues
Doutoramento em Ciência de Computadores Departamento de Ciência de Computadores
2019
Orientador
Luís Miguel Barros Lopes, Professor Associado, Faculdade de Ciências
Coorientador
I am deeply indebted to my advisors Profs. Fernando Silva and Lu´ıs Lopes and also to Prof. Eduardo Marques for their fundamental role during my doctoral work. They provided me with every bit of guidance, assistance, and expertise that I needed. I would like to thank, again, my advisors for offering me a research grant and a special thanks for their effort to provide all the material required to the success of this project. Also, I would like to thank, Prof. Rolando Martins for his assistance and guidance at the beginning of my work. I simple cannot imagine better advisors and colleagues to work with. It is a honour to work with such good people.
I am thankful to the Funda¸c˜ao para a Ciˆencia e Tecnologia (FCT) for funding my
research grant in the context of project HYRAX (CMUP-ERI/FIA/0048/2013), which supported this work for 48 months, and to the Centre for Research and Advanced Computing Systems (CRACS) for the travel funding through projects financed by ERDF (COMPETE 2020, POCI-01-0145- FEDER-006961), FCT (CMUP-ERI/FIA/-0048/2013 and UID/EEA/ 50014/2013), and FEDER (NORTE 2020, NORTE-01-0145-FEDER- 000020).
To all my fellow colleagues from DCC-FCUP, specially F´abio Domingues, Sylwia Bugla
and Joaquim Silva, I wish you much success for your professional and personal future. Finally, I would like to thank my family and friends for all the support during the last years, specially my mother and sister, that believed, trusted and supported me in every way they could. I am also specially grateful to my grandparents, Sildina and Manuel, whose way to life is really inspiring.
Durante a ´ultima d´ecada, os dispositivos m´oveis tornaram-se ub´ıquos. Avan¸cos no processo de fabrico fizeram com que os pre¸cos de tais dispositivos baixassem significati-vamente enquanto aumentava a sua capacidade de processamento e de armazenamento. Tradicionalmente, esses dispositivos eram vistos como thin clients. No entanto, como
anteriormente afirmado, os smartphones e tablets de hoje contˆem recursos de hardware
que permitem que software mais sofisticado seja instalado, permitindo a sua utiliza¸c˜ao
como thick clients ou mesmo pequenos servidores. Simultaneamente, novos standards e protocolos, como WiFi-Direct, foram desenvolvidos para permitir que os dispositivos
m´oveis pudessem comunicar directamente entre si, em vez de ser atrav´es da Internet
ou de pontos de acesso WiFi. Al´em disso, esta tecnologia permite a comunica¸c˜ao entre
os dispositivos com alta largura de banda e ao mesmo tempo baixa latˆencia. Por fim,
os dispositivos tˆem agora incorporados v´arios sensores que lhes permitem ”sentir” o
ambiente, em redor, e alimentar aplica¸c˜oes e servi¸cos mais sofisticados.
Todos estes desenvolvimentos est˜ao na base de um interesse renovado na ´area de
edge-networks. Tais redes s˜ao compostos por dispositivos m´oveis que formam entre si
edge-clouds, trazendo parte da funcionalidade da cloud para a periferia da Internet. As
edge-clouds s˜ao portanto formadas por conjuntos de dispositivos m´oveis em estreita
proximidade que fornecem servi¸cos que se baseiam na agrega¸c˜ao dos seus recursos
computacionais e de armazenamento. O desenvolvimento de tais aplica¸c˜oes e servi¸cos
´
e, contudo, dificultada pela pela complexidade da forma¸c˜ao e manuten¸c˜ao das redes,
pela instabilidade intr´ınseca das liga¸c˜oes sem fio e pela heterogeneidade do hardware
e dos sistemas operativos dos dispositivos.
Nesta tese, apresentamos o desenho e a implementa¸c˜ao de um middleware gen´erico
para edge-clouds com a finalidade de fornecer aos programadores os procedimentos
b´asicos para implementa¸c˜ao de aplica¸c˜oes m´oveis para estas plataformas. Com esse
objectivo, o middleware oferece ao programador uma API que lida com a complexidade de baixo n´ıvel tal como forma¸c˜ao e gest˜ao de redes e a agrega¸c˜ao de recursos, de forma
caso contr´ario seria inutiliz´avel no mundo real.
Al´em da arquitectura e dos detalhes de implementa¸c˜ao, fazemos uma avalia¸c˜ao do
desempenho, da escalabilidade do middleware e tamb´em uma breve discuss˜ao sobre
as in´umeras aplica¸c˜oes e servi¸cos que tˆem vindo a ser implementados recorrendo ao
middleware. Al´em disso, desenvolvemos e estudamos o comportamento de aplica¸c˜oes
reais para a distribui¸c˜ao e partilha de v´ıdeo em infraestruturas desportivas, com ou
sem infraestrutura de rede.
In the last decade, mobile devices have become ubiquitous. Advances in manufacturing processes have significantly dropped the price tag of such devices whilst augmenting their storage and computational capabilities. Traditionally these devices were viewed as simple clients. However, as stated, smartphones and tablets today have hardware resources that allow more sophisticated software to be installed, allowing for their use as thick clients or even thin servers. Simultaneously, new standards and protocols, such as WiFi-Direct, have been developed that allow mobile devices to communicate directly with each other, as opposed to over the Internet or across WiFi access points. This technology enables low latency, high bandwidth device-to-device (D2D) commu-nication. Finally, the devices have now embedded multiple sensors that allow them to feel the environment and feed more sophisticated applications and services.
These developments fostered the research on edge-networks, composed of such devices, and on mobile edge-clouds, where some traditional cloud computing functionality is provided by at the edge. Mobile edge-clouds are thus formed by sets of mobile de-vices in close proximity providing serde-vices that crowd-source their computational and storage resources. The development of such crowd-sourcing applications and services is, however, hampered by the complexity of network formation and maintenance, the intrinsic instability of wireless links and the heterogeneity of the hardware and operating systems in the devices.
In this thesis, we present the design and implementation of a general purpose mid-dleware for edge-clouds that provides programmers with the basic building blocks for implementing mobile crowd-sourcing applications. Towards this goal, the middleware provides the programmer with an API that handles the low-level complexity of net-work formation and management and crowd-sourcing of resources, whilst handling problematic issues such as intermittent communication, device churn and untethered execution. All this is provided without the need for ”rooting” the device, which would automatically devoid the approach of its applicability in the real world.
applications and services that have been implemented on top of it, including real-world apps for content sharing in sports venues or in infrastructure-deprived environments and general services for computation and storage.
3G/4G Third/Fourth generation of wireless mobile telecommunications technology
AP Access Point
AODV Ad-hoc On-demand Distance Vector
API Application Programming Interface
Bluetooth LE Bluetooth Low Energy
CC Cloud Computing
D2D Device to Device
DSDV Destination Sequence Distance Vector
DSR Dynamic Source Routing
GB Gigabyte
Gbps Gigabit per second
GHz Gigahertz
GPS Global Positioning System
IaaS Infrastructure as a Service
iOS iPhone Operation System
IoT Internet of Things
IP Internet Protocol
ISM Industrial, Scientific, and Medical radio band
MAC Media Access Control MANET Mobile Ad-hoc Network
Mbit Megabit
Mbps Megabit per second
MEC Mobile Edge Cloud
NFC Near Field Communication
MHz Megahertz
OLSR Optimised Link State Routing Protocol
OS Operation System
OSI Open Systems Interconnection
P2P Peer to Peer
PaaS Platform as a Service
RAM Random Access Memory
RTC Real Time Communications
SaaS Software as a Service
SDK Software Development Kit
SSD Solid State Disk
TDLS Tunneled Direct Link Setup
WPA WiFi Protected Access
ZHLS Zone-based Hierarchical Link State Routing Protocol
ZRP Zone Routing Protocol
Resumo v
Abstract vii
Acronyms ix
List of Tables xv
List of Figures xviii
1 Introduction 1
1.1 Motivation . . . 2
1.2 Problem statement . . . 3
1.3 Contributions . . . 5
1.4 Thesis layout . . . 6
2 State of the art 7 2.1 Networking . . . 8 2.1.1 Technologies . . . 8 2.1.2 Network formation . . . 10 2.1.3 Routing . . . 14 2.2 Middleware . . . 17 xi
2.2.2 Special purpose . . . 23
2.3 Crowd-sourcing applications . . . 25
2.4 Discussion . . . 26
3 A Middleware for Edge-Clouds 29 3.1 Overview . . . 29 3.2 Link Layer . . . 31 3.2.1 Architecture . . . 32 3.2.2 Example . . . 34 3.2.3 API . . . 39 3.2.4 Implementation . . . 41 3.3 Network layer . . . 43 3.3.1 Architecture . . . 44 3.3.2 Examples . . . 46 3.3.3 API . . . 48 3.3.4 Implementation . . . 50 3.4 Discussion . . . 62 4 Middleware evaluation 63 4.1 Link layer . . . 63 4.1.1 Evaluation setup . . . 63
4.1.2 Latency of link actions . . . 64
4.1.3 Bandwidth measurements . . . 66
4.1.4 Resource consumption . . . 66
4.2 Network Layer . . . 70 xii
4.2.2 Packet routing . . . 71
4.2.3 Network Formation . . . 87
4.3 Discussion . . . 94
5 Other Contributions 97 5.1 Wireless Technology Assessment . . . 97
5.2 User Generated Replays: Part I . . . 99
5.3 User Generated Replays: Part II . . . 100
5.4 Edge-Cloud Services and Apps . . . 102
5.4.1 Distributed Computing . . . 103 5.4.2 Distributed Storage . . . 104 5.4.3 Publish/Subscribe . . . 104 6 Conclusions 107 References 110 xiii
2.1 Frameworks/SDKs available. . . 27
3.1 Supported Logic Actions per Wireless Technology. . . 34
4.1 Link layer - bandwidth. . . 66
1.1 Proposed Scenarios. . . 4
2.1 WiFi-TDLS. . . 8
2.2 WiFi-Direct. . . 8
3.1 Hyrax Middleware. . . 31
3.2 The link layer internal structure. . . 32
3.3 Link Layer API. . . 40
3.4 Link Layer Flow. . . 41
3.5 A Logical Network. . . 44
3.6 The network layer internal structure. . . 45
3.7 Network Layer API. . . 48
3.8 Network Layer Flow. . . 50
3.9 Star Network. . . 57
3.10 Mesh Network. . . 57
4.1 Link layer - action latency. . . 65
4.2 Link layer - CPU usage. . . 67
4.3 Link layer - memory usage. . . 68
4.4 Link layer - battery usage. . . 69
4.5 Bluetooth star . . . 74 xvii
4.7 WiFi direct star . . . 77
4.8 WiFi direct star traffic. . . 78
4.9 WiFi star . . . 80
4.10 WiFi traffic. . . 81
4.11 WiFi/WiFi-Direct with up to 3 devices per group. . . 84
4.12 WiFi/WiFi-Direct with up to 6 devices per group. . . 86
4.13 Bluetooth star formation. . . 89
4.14 WiFi-Direct star formation. . . 91
4.15 WiFi/WiFi-Direct star formation. . . 93
5.1 Benchmark Scenarios (from [85]). . . 98
5.2 Cloud to Edge Architecture (from [93]). . . 100
5.3 The Edge Cloud Architecture (from [84]). . . 101
5.4 Deployment at Nave Desportiva de Espinho (from [84]). . . 102
Introduction
In the last two decades, the domestic and industrial computing paradigm has drasti-cally changed with a new, cost effective, programming paradigm called Cloud Com-puting (CC). The providers of CC solutions offer an illusion of an “infinite” processing power and storage pool supported by a vast infrastructure of powerful, server calibre, machines. Traditionally, such computing power was available only to large companies, institutions and governments. Smaller companies and businesses had to resort to their own information systems for providing online services. The introduction of CC allowed the big companies to monetise their often huge computational resources whilst providing a cost effective, highly flexible, way for smaller companies to deploy and maintain their business services on the Internet.
More recently, the exponential increase in the number of mobile devices has changed the way information is consumed. Whereas before most contents were provided by mostly static, physically connected machines on the Internet, later most of the Internet traffic is already produced by mobile devices and “things”, forming highly dynamic wireless networks. The computational and storage resources of these devices, however, still did not allow complex tasks to be performed “in loco”. Mobile devices were, at most, thin clients. In this setting that CC was given a new twist called Mobile Cloud Computing (MCC). The new paradigm allowed mobile devices to offload complex tasks and data onto a traditional CC infrastructure. The results were then sent to the devices provided they have an Internet connection.
Mobile devices, however, have since evolved considerably in terms of processing power, memory and storage space, some rivalling desktop systems only a decade old. These resources can be exploited to perform complex tasks locally, something unthinkable
only a few years ago. Mobile devices today have the resources to provide thin services. Moreover, many application scenarios never really benefited from the MCC paradigm as they rely heavily on local data, generated and stored in the devices, require high bandwidths and low communication latency. This realisation, the availability of stock hardware supporting high performance wireless connections, and the ubiquity of the devices set the stage for Mobile Edge Computing.
1.1
Motivation
The number of mobile devices have been steadily increasing. It is predicted that by 2020 there will be more than 11.6 billion connected devices, exceeding the world’s projected population at that time (7.8 billion) [21]. The same study suggests that the mobile global traffic will reach 35 exabytes per month in 2020 (contrasting with 11 exabytes in 2017), 75% of which will be video content. Handling such a high traffic will be really challenging for the communication infrastructures (e.g. 3G/4G and WiFi) that must evolve in order to support it (e.g, 5G products are hitting the market now). Nowadays, a typical mobile device features like computing, memory and storage resources are equivalent to laptop computers only a decade old and more and more beefy models are announced at a very fast pace. Such resources did not go unnoticed by researchers who envisioned using the devices for more challenging tasks, from traditional thin clients up to (thin) service providers. Moreover, new standards and protocols, such as WiFi-Direct, allow device-to-device (D2D) communication with low latency and high bandwidth. Taken together, these advances allowed researchers to propose a computational model similar to Cloud Computing except that the resources are provided by devices at the edge of the Internet, connected via wireless links in highly dynamic network configurations. These “edge clouds” provide the backbone upon which multiple crowd-sourcing applications supporting unthetered execution can be implemented to explore data locality and fast D2D communication [26, 29, 32, 44]. Despite the growing interest in edge-clouds, relatively few crowd-sourcing applications have been proposed, either as research prototypes or commercial products. One of the reasons for this meagre output is, we believe, the lack of adequate middleware to support the development of such applications, allowing programmers to abstract away from the intricacies of D2D communication using multiple protocols, from building and maintaining a mesh network of devices, from moving and storing data between devices, from scheduling computations over the network, and other complex operations. A
middleware capable of providing an API and core services with such functionality would go a long way to make application development more agile.
1.2
Problem statement
This work was developed in the context of the Hyrax project1 in which three
appli-cation scenarios were envisioned to motivate and to provide initial experience before and during the design and implementation of the middleware:
• Crowded Venues: in crowded scenarios the problem arises when loads of people want to access to contents having as consequence the stress of the communication
infrastructures, radio spectrum and central servers. Some of that traffic is
repeated, representing some kind of context (same interests), of which can be handled more efficiently. For these scenarios, new types of mobile applications may emerge by taking advantage of content locality and the new communication technologies (D2D), like WiFi-Direct, Bluetooth LE, WiFi-TDLS in order to offload some of the traffic in alternative ways. Traditionally, these systems are pure client-server models that must be adapted to work on P2P fashion and to deal with intermittent connectivity and mobility.
• Amber alert: a crowd-sourcing application for distributed face recognition to deal with emergencies like Amber alerts, used to disseminate information about missing people. We consider crowded scenarios such as stadiums or shopping malls, where it is common for instance to have children getting lost from their parents. By using computer vision on edge clouds, it is possible to minimise, and potentially eliminate, the dependency on infrastructural clouds, by doing the search of the missing person’s photo on the local storage of each mobile device without having to disclose them to a central cloud provider. The objective is to gather positive identifications of a missing person, along with important time and spatial information.
• Disasters: several crowd-sourcing applications emerged to deal with emergency and disaster scenarios making use of mobile devices. Most of these applications, though, rely on standard communications and a web/cloud-based infrastructure, and are focused on mobile data collection and dissemination. In disaster sce-narios, infrastructural communications (WiFi, GSM/3G/4G) may be severely
(a) User Generated Replay.
Missing
(b) Distributed Photo Search.
(c) Environmental Disasters. Figure 1.1: Proposed Scenarios.
impaired due to adverse environmental conditions, and people (e.g., a rescue team worker or a person in distress) find it hard or impossible to communicate. The Hyrax application would let users find others that may be near the same physical vicinity, and establish peer-to-peer communication for the exchange of text messages or other media like audio/video, geo-tags, etc.
Based on these motivational examples, for which simple prototypes were implemented from scratch using Android’s API, we hoped to gain an understanding of the technical requisites and issues associated with the main goal of this work: to design and implement a middleware that seamlessly, through a rich API, allowed programmers to form and manage mobile edge-clouds and provide basic computational and storage services for crowd-sourcing applications. We had some generic requisites when we started. We envisioned that the middleware should handle the intricacies of D2D communication using a choice of technologies (e.g., WiFi-Direct, Bluetooth). This includes device discovery, network formation and maintenance and message routing. On top of this communication backbone, several low-level services should be provided
to support distributed computation of tasks, shared storage, streaming and publish/-subscribe operations on data. A major requisite was that the middleware should also be developed without requiring root-level access to the device (“rooted” device) as it would render it useless for general use.
1.3
Contributions
The work started with an evaluation of the available wireless technologies and an assessment of their performance with a small prototype of a video dissemination app for crowded venues. Further development of this app and related services allowed us to make the first real world experiments devices controlled by volunteers during a Champions League game, whose stream we fed into the system. The experience gained allowed us to design the bottom layers of the middleware, accordingly called “Link” and “Network”. We then moved to refactor the video dissemination application using the middleware as the backbone, ditching the Google API. A full evaluation of the middleware then followed and major test of the app in a real world scenario during a professional league volleyball game “in situ”.
The main contributions of this work were published in several conferences and work-shops as listed here:
• assessment of the latency and bandwidth provided by available wireless tech-nologies using an app for the context of the crowded venues scenario, towards their use in the middleware (published at [85]);
• design and development of an crowd-sourcing app for video dissemination in crowded venues; assessment of the app’s performance based on a real-world experiment at the faculty and comparison with results obtained in the previous work (published at [93]);
• design and implementation of the link and network layers of the Hyrax middle-ware (published at [83]);
• integration of the middleware in the crowded venues application; assessment of the app’s performance based on a real world experiment during a professional volleyball league game at Nave Desportiva de Espinho (published at [84]). Besides this core work, we also provided continuous support for the development of other services on top of the middleware, namely:
• P3Mobile, a distributing computing service (published at [89]);
• Ephesus, a distributed storage system based on DHTs (published at [91]); • Thyme, a publish/subscribe system (published at [17]);
• Panoptic, a distributed face recognition system (published at [36]).
1.4
Thesis layout
This thesis is organised as follows. Chapter 2 describes the wireless technologies
available on Android operation system (SO) and the network formation and routing algorithms, in the literature, for mobile ad-hoc network (MANETS). Furthermore a few generic and specific middleware are shown, for edge-clouds, and some existing applica-tions that already take advantage of device-to-device (D2D) technologies. Chapter 3 presents a general view of the Hyrax architecture as well as a detailed description of each architecture layer. Chapter 4 mention how the middleware was implemented and a discussion is made about the results that were gathered for each middleware’s layer. Chapter 5 refer the work that was performed, since the beginning, that originate the Hyrax middleware, and also a brief description is done of the applications that were/are being built on top of the Hyrax middleware. To finalise, the Chapter 6 shows the conclusions, of this work, and also the next steps.
State of the art
The growth and proliferation of mobile devices, opens the possibility to create new applications that take advantage of their aggregate capabilities and of data locality. To that end, edge-clouds have being introduced so that the nearby devices may accomplish something bigger by cooperating with each other. This notion of proximity arises from the requirement to reduce the latency of the communication by performing some of the work, like storage and processing, at the edge of the network avoiding the path to the central servers. Such configuration alleviates the stress on the servers as well the infrastructure, which in turn decreases maintenance costs like hardware and administration teams. Furthermore, mobile edge-clouds can play a major role, in scenarios, where there is no infrastructure available, using exclusively device-to-device (D2D) communication.
No only the number of mobile devices is growing, the technology is also improving. Mobile devices are becoming smaller, computationally powerful and with plenty of storage capacity. Besides the computational resources, these new devices support a diversity of communication technologies such as Bluetooth, Bluetooth Low Energy, WiFi ad-hoc, WiFi-Direct and NFC. These new communicating features are essential to create pure ad-hoc networks bypassing traditional infrastructure.
In this chapter we go through to the most relevant developments on edge-clouds as well as the technologies and algorithms that support it.
2.1
Networking
WiFi and cellular networks are becoming faster, but at same time bandwidth demand increases even faster because of the fast growth in users and contents. These constrains inevitably lead to the degradation of the link quality in dense scenarios and to face such traffic demand, new technologies are required that support reasonable bandwidth over short distances. One potential solution is the utilisation of D2D technologies that use the wireless spectrum more efficiently, enabling parallel communications. In this section, a few D2D technologies are described and also some of the most influential network formation and routing algorithms for mobile ad-hoc networks (MANETS).
2.1.1
Technologies
Currently, the mobile devices come with very rich wireless hardware resources. Some of this technologies are very common like Bluetooth and WiFi and other are relatively new such as Bluetooth Low Energy, NFC (Near Field Communication), WiFi-TDLS (Tunnelled Direct Link Setup) and WiFi-Direct. These new technologies are promising since they allow to dynamically create ad-hoc networks and also to optimise the spectrum utilisation. We now briefly discuss these technologies.
TDLS TDLS
Server
Client Client
Figure 2.1: WiFi-TDLS.
Server and Soft Ap
Client Client Client
Figure 2.2: WiFi-Direct.
Bluetooth is a well studied and stable wireless technology standard for short range
distance communication (10-30m). It operates on the 2.4-2.485 GHz ISM (industrial, scientific and medical radio band) band and jumps continuously on different channels in order to listen and communicate (frequency hopping spread spectrum). In older versions there are 79 distinct channels with 1 MHz width, but in newer versions the number of channels is just 40 with 2 MHz width [11], achieving speeds of 1–3 Mbit/s. The current version is 4.2 and Bluetooth 5, has been announced that will further increase the speed, the range and broadcast capacity [12].
Bluetooth Low Energy is a power-aware version of Bluetooth that was designed for the Internet of Things (IoT) and the maximum speed is 1 Mbit/s. Essentially this version consists on beaconing information that is collected by a set of subscribers. The range and speed of this version is lower than the original, in contrast it increases by far the energy capacity. A coin sized device could last months or even years beaconing information without a recharge [13].
NFC is a communication protocol that enables two devices to communicate when
they are really close (≈10cm) [69]. This technology cannot be used to form edge clouds (short range and low data rates), however it can be useful to complement the edge by adding some location aware features. For instance exchange WiFi credentials, notify a position of a certain place in a venue.
WiFi is a stable long range (50-100m) wireless technology that allows high speed
communication. It operates on the 2.4 and 5 GHz ISM bands and the new versions of WiFi allow gigabit speeds (802.11ac). The 2.4 GHz ISM band has just 3 non-overlapping channels (20 MHz width), while the 5 GHz ISM band has more channels available, 19 in Europe (20 MHz width). Traditional routers are just able to use one channel at the time (one antenna), but the newer versions can use MIMO (multiple-input and multiple-output) thanks to the multiple antennas available capable of send-ing and receivsend-ing data ussend-ing multiple channels at same time [102].
WiFi-Direct is a standard built on top of the WiFi stack and has the ability to
create a soft-AP, on demand, providing WiFi signal to other devices in the area (Figure 2.2). Thus devices in the neighbourhood are able to connect to the WiFi network under the normal circumstances. This standard offers security as well, WPA2, and essentially allows to create small WiFi networks dynamically everywhere [103]. Note this is similar to traditional mobile access point, but with an extra feature in which the devices can belong to an ordinary WiFi network while being part of a WiFi-Direct network at same time.
WiFi-TDLS (802.11z) is a WiFi feature that allows two devices under the same
AP to communicate directly to each other (avoiding the AP path) (Figure 2.1). To use this feature two devices, under the same AP, negotiate a different channel to shift the communication and, if they succeed, all the data between the two devices are moved to the new channel - direct link. Otherwise, if the negotiation fails, then they
will continue to communicate in a normal way - using the AP. This feature allows to reduce the traffic handled by the AP and also to use the spectrum more efficiently [95]. Note the devices may negotiate a channel from a different band. For example a router can operate on the 2.4 GHz ISM band and their clients may create direct links using a channel from the 5 GHz ISM band.
Is expected, that in the near future, new wireless standards (e.g. 5G) will arise that utilises the spectrum more efficiently, more bandwidth, less energy consumption, and in some cases higher communication range. At same time the cellular networks are evolving to provide more bandwidth, less latency and more coverage. To fill the gaps where these technologies do not operate efficiently, and also to create alternatives, the D2D technologies will play a fundamental role to form edge-clouds where the communication will be driven to a certain context in order to operate either on dense (scalability) or in infrastructureless environments. Some of these technologies are new and must be tested in real environments and improved. The traditional WiFi offers high speeds, however in dense environments may perform poorly due the properly channel management and coverage. The new WiFi extensions or even Bluetooth can help to re-utilise the spare channels in wireless networks in order to improve the communication performance.
2.1.2
Network formation
The most widely form of wireless communication is WiFi (less coverage, higher speeds)
and 3G/G networks (high coverage, smaller speeds). These technologies are very
similar: a specialised static hardware (router, antenna) emitting a signal and a group of clients connecting to the network when in range - star network. The problem arises in dense areas scenarios, where a large amount of people want to communicate (e.g. stadium), raising scalability issues and in scenarios where there is no communication infrastructure. Thus, to tackle these niche new wireless technologies have risen, like as we describe in the section before 2.1.1, and the next challenge is how we take advantage of these technologies to form edge clouds. Seen this, new and more dynamic formation algorithms must be employed in order to answer to the new set of requirements. Consequently, in order to form edge clouds the devices, that are near to each other, must be organised somehow using these D2D technologies like Bluetooth, WiFi-Direct and WiFi ad-hoc. Note the WiFi-TDLS is a feature and cannot coexist without a WiFi network previously formed.
other over short range and provides low bandwidth communication, when compared with WiFi. The network formation algorithms theme over Bluetooth is not new and in fact there is plenty of research done on this field. Bluetooth uses a specific topology - piconets and scatternets - to organise the network. A piconet is a star network where a node acts as master and the other nodes act as slaves (connected to the master), in which all share the same communication channel. The problem with piconets is that only eight devices can be part of the network. Moreover to overcome this limitation the notion of scatternet was introduced in order to interconnect the piconets by allowing some nodes to belong to more than one piconet at same time (bridge nodes). The piconets can be connected into three ways:
• Link master-to-master: In terms of number of hops this solution is the most efficient since it only needs one hop to reach another piconet, but in contrast the master nodes that belong to several piconets will drain the battery faster and will be the bottleneck of network since it needs to jump constantly over several channels and is less tolerant to churn - tree based.
• Link slave-to-slave: This solution is less efficient in terms of number hops, but allows to build larger networks and also to distribute the efforts by more nodes (not only the masters). In this case there are less nodes jumping on different channels which is good for network performance and is more tolerant to churn -mesh based.
• Link master-to-slave: This solution sits right in the middle of the previous two. The problems are similar as the first one however it allows larger networks. Regarding the piconets and scatternets formation the authors in [64] present some guidelines to be considered and discuss the performance aspects about this types of networks. In the literature there are plenty of formation algorithms and they can be divided into two major groups, tree based and mesh based. The tree based algorithms enable structured networks which is good for routing but in contrast is less tolerant to churn and have some crucial points of failure. The next, mesh based, is less structured and adapts easily to changes, but the routing becomes a bit more complex [10]. In the following list are the most influential Bluetooth formation algorithms.
• Bluetrees: is a tree-based formation algorithm where each node belongs at most two piconets at same time. This restriction reduces the channel switching overhead. Essentially the algorithm starts with just one node (blueroot) and the remaining nodes will be added to the network sequentially, forming a tree [106];
• Bluenet: is a distributed version of Bluetrees in which the formation phase is divided into three main sub phases 1) visibility graph, knowledge about the whole network, 2) independent piconets formation and 3) piconet interconnection form-ing a scatternet. The advantage is the creation of balanced networks, however the algorithm lay on assumptions not feasible on real world scenarios [100]; • Bluestars: is similar to the previous one, however the visibility phase is replaced
by an alternating state discovering and discoverable, which allows to form net-works with fewer information [76]. This algorithm formation is costly which compromises the implementation in dynamic environments;
• Others: the core of the algorithms presented in [57] and [86] are very similar to the ones shown before. The major difference is that these new ones have procedures to join/split piconets, nodes move and migration. For higher churn networks these procedures can be tricky because the cost of energy and time, for these changes, is high. Although for stable networks those procedures are good to minimise the network overhead in regard of scatternet properties. These algorithms follows the guidelines present in [64].
WiFi-Direct [104] is a D2D technology that allows direct communication between nodes in range. It has similar bandwidth than the legacy WiFi networks and costs less in terms of energy consumption. Comparing with Bluetooth, the bandwidth is by far higher but in contrast it consumes more energy. Depending on the scenario the technology must be chosen accordingly. For example for a chat application, Bluetooth is more than enough since it does not requires much bandwidth, but in case of a video sharing application WiFi speeds is required inasmuch as it is a bandwidth demanding application. For network formation the algorithm is simpler than Bluetooth since the Android implementation only allows to create isolated stars. To form a WiFi-Direct network one device must be elected Group Owner (GO) and the other devices connect to it using the legacy WiFi interface or WiFi P2P interface.
The Android implementation of WiFi-Direct does not allow multi-group communi-cation, however in the specification [104] this ability is an optional feature. In the literature there is some research done, in this area, that attempts to implement the multi-group communication using the two possible modes of WiFi-Direct: legacy and non-legacy. Note that a client device on non-legacy mode is able to keep a connection to an AP and be part of WiFi-Direct group at same time contrasting with legacy mode. In both cases the GO is always capable of keep a WiFi connection and participate on the WiFi-Direct group at same time. Another restriction is that distinct WiFi-Direct
groups have always the same network mask (192.168.49.0/24) which is a challenge for communication among groups, e.g to establish a socket. The following work has addressed this issue:
• The authors in [16, 30] were the first to propose the multi-group communication using WiFi-Direct on non-routed devices. The authors propose a solution that uses both legacy and non-legacy interfaces to achieve multi-group communication using a relay node. As the devices belong to same IP network (192.168.49.0/24) they resort to the network broadcast capabilities to provide communication outside the group. Note that in Java the developers may select the network interface to broadcast messages. Android devices, by default, choose the legacy WiFi interface to redirect the traffic since the GO is connected to two networks with the same IP address range, which makes impossible to establish a socket, for example. The intrinsic limitation of this solution is the available bandwidth for communication outside the group since the maximum broadcast speed allowed is 6Mbps, contrasting the unicast speeds of 54Mbps. Another problem is that a TCP socket cannot be established outside the group;
• The authors in [80] present an early solution where the communication is per-formed just inside the group, using WiFi AP mode. They could use WiFi-Direct as communication layer that the system would work without any changes. Next the authors in [96] propose a new architecture that allows the WiFi-Direct groups to communicate to each other using bidirectional links - no need for broadcast. The authors do this by proposing two relay nodes, instead of just one, to connect two WiFi-Direct groups and provide bidirectional links. For example Relay1 is connected to GO1 using the WiFi P2P interface (non-legacy) and connected to GO2 using the normal WiFi interface (legacy), while the Replay2 is connected to GO1 using the normal interface and connected to GO2 with the P2P interface; • The last work in [37] evaluates the different ways of multi-group communication using WiFi-Direct. The authors propose two alternatives: time sharing and simultaneous connections. For time sharing the idea is that one of the devices switches among groups in a time slicing fashion. The switching process is done by disconnection from one group and connect to another. The authors evaluated the time required for the switching process and the energy consumption for different scenarios. In the second solution the authors keep the group formed and achieve connectivity with other groups using multicast sockets, very similar to the work in [16] or changing the WiFi Direct (requires root) implementation
to assign distinct IP address ranges to different groups and to instantiate a specific network interface. They measure the amount of time to transfer a certain quantity of data and the energy consumed by it.
There are other D2D technologies available such as Bluetooth LE (Low Energy) and WiFi ad-hoc. For Bluetooth LE the network formation is pretty much the same as the legacy Bluetooth systems. In fact this lower power version of Bluetooth has extra features that could optimise the network formation since it allows the devices in range (neighbourhood) to know something about a device (before connecting), enabling decision-making (services). Moreover this version of Bluetooth consumes significantly less power than it’s predecessor, with bandwidth penalty. Regarding WiFi ad-hoc and at this level there is no network formation issues since those are strongly coupled with routing algorithms for MANETs.
2.1.3
Routing
After the network formation the messages must be delivered somehow. Thus the next step is to study the mechanisms available to route messages in the network. In the literature there are plenty of routing algorithms for MANETS and all the existing algorithms assume unrestricted access to low level messages. In mobile devices this is possible only if the user has root permissions. Although, as we are developing mid-dleware that should run on non-rooted devices we cannot resort on such assumptions. In that case, this section will describe the most known routing algorithms and their categories in order to built an application level routing algorithm based on those. The routing algorithms are divided into two major groups: proactive and reactive algorithms. In proactive routing nodes keep a table of routes, where the route to a specific destination can be found. These algorithms exchange periodic messages in order to keep all the routing tables up to date. One advantage of proactive routing algorithms is the ability to know, previously, the route to a destination node avoiding to discover the route on demand diminishing the number of exchanged messages. On the other hand to keep the routing tables up to date can be a challenge for larger networks or high mobility networks, because the number of messages and the latency grows rapidly. The following are the best known proactive routing algorithms:
• Optimised Link State Routing Protocol (OLSR): The OLSR [22] algorithm exchanges Hello messages among the direct neighbours allowing the discovery of
it’s neighbours and two-hop neighbours information. After that it will select the best Multipoint Relays the best fits to communicate with two-hop neighbours. • Destination Sequence Distance Vector (DSDV): The DSDV [75] is based on
Bellman-Ford algorithm that gives the best, and only route to a destination. It basically uses two types of messages: fulldump and incremental packages. The fulldump package carries all available routing information. The incremental package carries just the updates since the last fulldump package. The incremental packages are exchanged more frequently than the fulldump packages.
The behaviour of reactive routing protocols is quite opposite, because they search for a route on-demand, in other words, they try to find a route just when it’s needed. It resorts to flooding to find the route to the destination. The main advantage of this kind of algorithms is that it does not need to exchange any control messages because there is no routing table to maintain. One obvious disadvantage is that to find a route, the network must be flooded first which, if not done carefully, will lead to network clogging and high latencies. The best known proactive algorithms are the follow:
• Dynamic Source Routing (DSR): DSR [53] is a reactive routing protocol, that requires to cache all device addresses from the source to destination to route a message. Thus, the routing information will be part of the message header. Whenever a node needs to send a message to a destination it first sends a RouteRequest package to find the destination. Then the destination will answer with a RouteReply packet in which will carry all the devices addresses transversed by the RouteRequest packet. The advantage is does not need to keep a routing table, in contrast the disadvantages are the overhead of finding a new route and also the cache of routes that can be big (for large networks). Additionally the information can be easily outdated (mobility) which may lead to inconsistencies. • ad-hoc On-demand Distance Vector(AODV): The AODV [74] is very similar to DSR, the big difference being that AODV does not need the whole path during the RouteRequest or RouteReply, just the source or the destination node. The major advantage is that is able to adapt easily to highly dynamic networks, but in another way introduces more delays at route discovering.
The aforementioned routing protocols are the most popular and the reader may find more detailed information about them and performance comparisons in [1, 51, 62]. Be-sides the proactive routing protocols and reactive routing protocols there are protocols
that join the best of the two worlds, called hybrid routing protocols. Generally, the hybrid routing protocols have the proactive behaviour in the proximity neighbourhood and reactive behaviour for further nodes. The two best known hybrid routing protocols are:
• Zone Routing Protocol (ZRP): The ZRP is divided into two main parts: Intra-zone Routing Protocol (IARP) and Inter-Intra-zone Routing Protocol (IERP). The IARP protocol defines a radius within which the algorithms has a proactive behaviour (table-driven). When the destination node is outside the zone, in this case it uses the IERP that has a reactive behaviour (on-demand) to find the routes in others routing zones. More detailed information and specification about it in [43].
• Zone-based Hierarchical Link State Routing Protocol (ZHLS): The ZHLS is very similar to ZRP the difference here is that a node can just to belong to one zone. To have notion of zone the ZHLS uses Global Positioning System (GPS) to define non-overlapping zones. More detailed information about ZHLS in [46, 79].
We have seen the major categories of routing protocols which are proactive, reactive and hybrid routing protocols. In the literature there are dozens more routing protocols, however most of them derive from the aforementioned routing protocols.
Apart from routing protocols there are other protocols that are useful to exchange messages in network, namely flooding and gossip protocols. Flooding protocols are used frequently by ad-hoc wireless networks and they simply flood the network with packages so that they could reach the destination/s. They are really simple to imple-ment and naturally use the shortest path to reach the destination. In contrast it adds more duplicated packets into the network which in turn introduces more latencies
because of the increasing bandwidth requirements. Additionally the packets may
live forever if some precautions are not taken [2, 3]. Gossip protocols are another set of communication protocols that disseminate information across the network in
more controlled fashion. The data is propagated thought the nodes like a virus
and eventually, with high probability, it will reach every node in the network. The advantage is the number of messages exchanged, but on the other hand is always the risk the messages do not reach (low probability) all the desired nodes [4, 58, 98]. Aside from, the aforementioned routing protocols, a new set of routing protocols are emerging which are the routing protocols geographically-aware. To provide some insight the authors in [90] suggest a geographical routing algorithm for MANETs using
ad-hoc networks combined with infrastructure access for opportunistic routing. Thus, either nodes with infrastructure access (WiFi or 3G/4G) or without any infrastructure access may coexist together in order to create a larger network and to take advantage of D2D communication protocols. The nodes are organised into cells where the routing process is geographical location-based instead of ordinary networking address. The routing process attempts to select the best path to a destination based on some heuristics, like energy consumption, the number of hops or latency. This determines if the message is going be routed through ad-hoc network or over the infrastructure, if available.
2.2
Middleware
Using edge networks to perform computation or to store information, as alternative to the cloud infrastructure is a hot top topic of research. The work in [68] is a good motivation for this kind of scenarios where they use people’s nearby devices to achieve large-scale distributed computing. In the experiment, they took real world traces of human encounter at the university campus and at a conference. After that, the authors analyse the utility of the computation and task farming efficiency on those different scenarios. Additionally they study the average utility of computation in different periods (1 hour interval) of the day which lead them to conclude that the amount of achievable computing (parallelism) is far greater during the student working hours, when there is more people gathering. Other on this area can be found on [28] the authors try to study the trade-offs between offloading computation to a cloud versus on the mobile edge-cloud using the Hyrax framework [61]. Is important to note that the clouds work on top of Gigabit networks while the mobile edge-cloud use WiFi 802.11n as communication relay - Megabit network, which have a big impact on system performance. To compare the trade-offs they use two different scenarios, Panorama Construction and Person Finder. On these scenarios they evaluated the total execution time, the energy consumed and the amount of traffic produced. The authors were able to show that there are applications that could benefit using mobile edge-clouds, for example the Person Finder only if it does not requires to much communication among tasks. These previous work suggest that the mobile devices have a lot of potential by cooperating together.
However, implementation of cloud-like services at the edge involves a considerable effort and implementation is not generally reliable unless a software backbone, a middleware that handles the low level details of networking, components, consistencies,
is provided to the developer. We will now describe some work that has been done towards this goal.
2.2.1
Generic Middlewares
Currently, there are a few middleware implementation available. Most require root privileges to function while a few work completely at the application level. The generic middleware is a kind of software architecture that can be adapted to new situations and is not tailored to solve a specific problem.
Hyrax
This work proposes [61] a platform derived from Hadoop MadReduce that supports cloud computing capabilities on Android mobile devices. This platform is able to perform jobs in heterogeneous networks joining together mobile devices and static servers nodes so that they could extend the application to be executed on the edge. Hadoop is a software library that allows to distribute large data sets among a cluster of devices [45]. This software package supports hardware failure, however it was not built for high churn environments, e.g. mobility scenarios. To port Hadoop to the Android system rooting all the mobile devices and making some difficult configuration was required. The port allowed distributed computation to be performed on the mobile devices using WiFi as communication mechanism. The author showed that the system tolerates node-departure keeping the system working with a reasonable performance.
Fram
The authors in this paper [49] present a content distribution middleware architecture for Android based mobile devices. The middleware is able to work without infrastruc-ture support and content dissemination is achieved using WiFi ad-hoc mode (IPv4 statically configured) which requires “root” privileges on the mobile devices. The nodes communicate in Peer-to-Peer (P2P) fashion using publish/subscribe message pattern so that to abstract the formation network issues. One interesting feature is the ability to work in intermittent scenarios, such as nodes leaving/joining the network. For example, a download is divided into fixed chunk sizes which allows incomplete downloads to be resumed later. As a proof of concept the authors implemented several applications to illustrate the Fram middleware work: Opportunistic Media Blog [50];
Personal profile sharing [56]; Collaborative music sharing [19]; and Participating light show [60].
C3PO
The C3PO project proposes a middleware [14, 94] framework that exploits different wireless interfaces (WiFi, WiFi-Direct, Bluetooth) of the mobile devices (non-rooted) to form a different type of network. They provide multi-hop networks by structuring the devices on a collection of micronets, grouped together in independent macronets. A micronet is essentially a star network (like Bluetooth piconets) where one device is responsible for the group and the remainder may communicate to each other directly. A macronet is a group of micronets interconnected through gateway devices (a device
that belongs to more than one micronet ), similarly as Bluetooth scatternets. As
message forwarding mechanism they use flooding and gossip-based routing protocols, which are configurable and extendable (custom routing protocol allowed). To finalise they permit two communication paradigms: publish/subscribe and P2P. In the pub-lish/subscribe model the users may publish multimedia content of a specific topic and subscribe in order to receive content related to a given topic. In the case of P2P communication model they use a concept of channel that allows the users to communicate directly to a specific address.
7DS
The 7DS [73] system was originally intended for extending web browsing and e-mailing of mobile nodes beyond the WiFi network coverage. A more recent work [65] is extending the original architecture to provide a mobile generic platform to de-velop disruption-tolerant applications. The 7DS platform is intended for working on intermittent infrastructure WiFi networks (not ad-hoc networks) and provides two main modules: discovery and data sharing. The discovery module is built on top of mDNS [20] that allows announcement, discovery and services resolution in close proximity. The data sharing module permits the users, when in physical range, to synchronise their shared contents (group of interest) transparently, e.g, a group of users editing one document. They only synchronise the content differences, which is a cost efficient mechanism to save bandwidth, using rsync delta encoding algorithm [97]. Applications like Gnutella [54] and BitTorrent [23] provide file sharing in always connected environments. Similar systems, iClouds [48] and Clique [82] implements file sharing mechanisms but the authors in [65] claim they are less efficient. There
are other, like Hayes [47], Proem [55] and Peer2Me [99] , systems that take advantage of Bluetooth to form ad-hoc networks for file sharing however they do not provide automatic service discovery and they are also limited by bandwidth.
Multipeer
The Multipeer Connectivity framework [9] is developed by Apple and enables services to be advertised and discovered between nearby iOS devices using different wireless protocols. The wireless protocols that can be used are WiFi, WiFi P2P and Bluetooth. The connected peers are able to transmit messages and files securely, without any web infrastructure. This framework can be divided into three main parts, ”advertising & discovering”, ”session” and ”sending & receiving information”. In advertising & discovering, the peers need to be aware of the services available, therefore the later need to be advertised. After that, a discovery process must be performed in order to find which services are available. Secondly, in order to communicate among devices
a session needs to be created. Sessions allow identity verification, through X.509
certificates, and encrypted communication, which significantly reduces the transfer rates. Once a Session is created the devices are ready to exchange messages/data. The Multipeer Connectivity framework has three different forms of data: messages (e.g. short text, serialised objects), streams (e.g. audio, video, real-time data) and resources (e.g. images, movies, documents).
Open Peer
Open Peer [71] is a P2P signalling protocol whose goal is to enable any kind of device to communicate in a P2P fashion. It is implemented in C++ and it has support for language bindings, like Objective C and Java, allowing this to function in iOS and Android devices. Open peer is an open source thin layer that works upon the webRTC [101] standard and is able to work fully in P2P manner offering security (privacy, identity validation and cryptography), network resilience, scalability and federation. In terms of security, it permits the users to exchange information in a free way, without firewalls restrictions of centralised clouds or data centres. WebRTC is an open project, supported by Google, Mozilla and Opera, aiming to provide a way of browsers and/or mobile applications to communicate directly to each other. For this, the devices must support real time communication capabilities [101].
P2pKit
The P2pkit [72] is a proximity-aware SDK that enables nearby search of devices and users. Basically, the p2pkit seeks to sense which devices are in the neighbourhood interchanging information, even in places without any type of traditional communica-tion infrastructure. It provides some features such as fast nearby discovery, range estimation (tell us how close someone is), dynamic content exchange (broadcasts information even in places without connectivity), multi platform support, background compatible (works even when the users withdraw their apps from foreground) and is energy efficient. The P2pkit is a commercial, developed by a Swiss company, Ueppa, and it uses Bluetooth low energy to announce services discovering who is nearby and resorts to WiFi-Direct or ad-hoc to transfer content. It supports iOS and Android.
Nearby
Nearby [42] is an API developed by Google, that provides publish and subscribe methods based on proximity. This API is cross-platform which can be used either by Android or iOS devices. This allows to create real-time connections between nearby devices and share messages. Nearby is divided into two distinct APIs: the ”nearby messages” API and the ”nearby connections” API. The nearby messages API is a publish-subscribe API that allows a set of interconnected devices to exchanged small amounts of data. The devices do not need to be in the same network but they must be connected to the Internet. This API uses a combination of Bluetooth, Bluetooth Low Energy and WiFi to establish communication between devices. The nearby connection API allows apps to discover others devices in the neighbourhood, forming a local network, which in turn keeps permanent connections so that to exchange messages in real-time. With this API, one device will become a host and the others must to connect it to form such local network.
AllJoyn
AllJoyn [5] is an open source, agnostic network framework that allows communication
among several devices. It has been developed by a consortium, e.g. Microsoft1,
Qualcomm2, Sony3, Sharp4and it is natively implemented in C++, but has the support
of bindings to other languages like, C, C#, Objective C, Java and JavaScript. Besides that, it runs on different operation systems like Windows, Linux, OS X, iOS and Android (>= 2.1). The AllJoyn framework abstracts the way the devices communi-cate, hiding the complexity of distinct network protocols and hardware, leaving only application details for the users. The communication is performed using two main objects: BusAttachment and BusObject. The BusAttachment is an abstract message bus, while the BusObject is the messages/data exchanged. The AllJoyn framework supports distinct network technologies as well as to run in different hardware devices
(e.g. smartphones, laptops, desktops). To the best of our knowledge only WiFi
infrastructure is actually implemented, and the peers are organised as mesh of stars, where the leaf nodes are connected to router nodes and these act as bridges to the others router nodes [6].
Serval Project
Serval [87] is a project with the objective of reducing the load of cellular networks, making possible to perform phone calls using only WiFi infrastructure (not resorting to cellular infrastructure). They developed a hardware [88], called the Serval Mesh Extender, aiming to communicate even in areas without any WiFi or cellular infras-tructure. Serval Mesh Extender hardware, is a peripheral that allows to transform a smartphone into an access point, enabling other devices to connect to it. Additionally, this hardware, is able to cover a wider range than a common mobile WiFi chip and consumes substantially less energy. Another motivation for them to build such hardware is that the android does not allow WiFi ad-hoc mode connection without requiring a custom ROM, rooting the device. Essentially, they are able to make phone calls resorting only to WiFi, forming a mesh network where the several ad-hoc access points are able to communicate with each other. More information may be found in [39–41, 88].
Open Garden - MeshKit
The Meshkit [63] is a proprietary software module that enables P2P communication among nearby mobile nodes. It uses different types of wireless communications, like
2https://www.qualcomm.com/ 3http://www.sony.com/ 4http://www.sharp-world.com/
Bluetooth, Bluetooth LE, WiFi Direct, to form a mesh network in order to deliver messages or share Internet connection to peers that are part of the mesh network. Thus every node, in such mesh network serves as relay of messages and allows to form dynamic networks taking into account node mobility. The mesh connectivity grants the possibility to function on intermittent connectivity scenarios or even in scenarios without any communication infrastructure.
2.2.2
Special purpose
Besides the general purpose middleware architectures there some more specialised projects that solve very specific problems like distributed task management or low level services management.
mCrowd
The mCrowd [105] is a mobile crowdsourcing platform that uses the sensor-rich iOS mobiles devices to execute large scale tasks. The general idea is that users can post sensing tasks to be further explored by mobile workers who can decide to answer to the posted tasks using their mobile phones. Additionally, mCrowd, exploits the existing crowdsourcing services like Amazon Mechanical Turk (MTurk) [67] or ChaCha [18]. MTurk and ChaCha are services that empower the human intelligence to perform tasks that computers cannot usually do. For example answering to the question “which is the best restaurant of a certain area?” is not an easy task for a computer. Thus the solution goes by asking around through a mobile app that question, for example using D2D technologies to take advantage of the crowd. The authors claim mCrowd supports four type of tasks: image tagging; image data collection; textual queries from users; and GPS location-based queries.
Femto Clouds
Habak et all [44] present the design and architecture of a distributed computing system. Their system uses a central controller (cloudlet), which provides WiFi signal, to which the mobile devices in range may connect to it and provide the available resources to perform parallel tasks. The core of the FemtoClouds system is on the central controller since it implements a sophisticated scheduling mechanism to assign tasks to the mobile devices and is also able to control when the devices must communicate in order to avoid
the WiFi congestion problem. The scheduling algorithm assigns tasks based on some metrics that are dynamically inferred, e.g., heart-bit, WiFi strength, and others that are provided by the user’s profiles that dictates some rules to join a FemtoCloud or not, e.g, minimum battery level, percentage of CPU usage. Additionally the FemtoCloud system has churn handling in mind which makes dynamic adaptations to the tasks assignment as the users leave or join the system.
MMPI - Mobile Message Passing Interface
MPI [66] is a programming model used by the High Performance Computing com-munity to perform distributed/parallel jobs on a cluster of static machines. With the proliferation of Bluetooth capable mobile devices the authors in [25] thought to take advantage of that by proving a MMPI library that could execute MPI jobs on Bluetooth capable devices such as laptops, smartphones. The authors were able to perform an experiment on Nokia devices, distributed matrix multiplication, showing execution speedups. Two years later, the authors [24, 27] showed promising results on calculation the Mandelbrot set fractal using Bluetooth piconets and Bluetooth scatternets, respectively, using the MMPI (embarrassingly parallel only).
Honeybee
The Honeybee [31] is a distributed computing framework that employs “work stealing” to dynamically load balance tasks across mobile devices. Those tasks can either be automatic CPU intensive tasks, e.g., image processing, or tasks that require human aid, e.g, image tagging, surveys. The authors in [32] developed three Android applications, on top of Honeybee framework, using different real mobile devices. Those applications are: distributed face detection; distributed Mandelbrot set; and collaborative pho-tography. The computation is fully done by mobile devices which are interconnected to each other using Bluetooth. The papers shows that collaborative work may be beneficial since the tasks do not introduce much communication overhead (dependen-cies). In [33] they were able to conduct a similar experiment but using the WiFi-Direct as communication mode among the mobile devices. The results suggests that simply moving the communication technology, to WiFi Direct, the speedup increases significantly since the WiFi based technologies have far more bandwidth than the Bluetooth based.
Bonjour
Bonjour Services [8] are a zero-configuration networking technology developed by Ap-ple. A zero-configuration networking is a network that is automatically formed without any manual intervention [81]. This implementation is open source, and supports OS X, iOS, Linux and Windows. The services work at the IP level, and are divided into three main components: ”Addressing”; ”Naming”; and ”Service Discovery”. The Addressing component permits to allocate different IP addresses to distinct devices, while Naming creates an alias for each device. The Service Discovery works similarly to a publish-subscribe service, where the peers advertise their services so that the other peers on the network may listen and connect to them. In order to the devices in a network to be discovered, upon turning on the Bonjour service, they announce themselves to the network. At the same time, other nodes in the network, already running a Bonjour Service, periodically ask the network what devices are available. As an optimisation, these requests are sent to the network using a exponential back-off and service announcement optimisation, thus decreasing the bandwidth use. Other optimisations used are DNS caching and suppression of duplicate messages.
2.3
Crowd-sourcing applications
In the literature there are already a few real world applications that take advantage of the device proximity by enabling P2P communication. On this section a few such applications are described.
FireChat
Firechat [35] is a proprietary mobile app, based on Firebase [34] and developed by Open
Garden5. The Firechat app offers a secure multi-user and multi-room chat with flexible
features, such as authentication, moderator capabilities, user presence and search, private messaging and chat invitations. This mobile app uses an underlying network
abstraction framework called Open Garden (the same name of the company). It
allows the formation of mesh networks and communication without having an Internet connection. To achieve this, it uses distinct wireless technologies, e.g., Bluetooth, WiFi and Apple’s Multipeer Connectivity Framework (only supported on Apple devices).
ZombieChat
Like Firechat, ZombieChat [107] seeks to build a mesh network for users to exchange messages without the need of an infrastructure. It relies on Bluetooth, WiFi P2P and WiFi to form a mesh network allowing to communicate in a vicinity or beyond that. This app is available for iOS only.
2.4
Discussion
So far, some middlewares (generic and specific) were described. In this section a discussion will be made in order to justify the reason why we did not choose any the options available. For the concept of edge clouds to properly work, generally, ad-hoc networks formation is required using dynamic configurations, e.g., D2D communication technologies. For that reason we hoped to build a middleware that is technology agnostic, i.e, it allows communication between nearby devices using any of technologies available. The middleware must allow the creation and connection of multiple ad-hoc networks (for larger coverage) and route messages in a transparently way. We identified the following set of requirements that must be addressed:
• Non-rooted: The middleware must work on non-root devices since the target audience is common people. Is not feasible to oblige the users to root their devices since most of them do not possess the knowledge to do so and also they may lose the device’s warranty. Another problem of routing devices to perform communication is the lack of control for energy consumption which is a precious resource in mobile environments.
• Distinct wireless support: Every wireless technology has its own limitations and for that is useful to provide support to multiple technologies. Another reason is that WiFi networks are not available in every place so other communication means must be employed. As such, the middleware must support different wire-less technologies such as Bluetooth, Bluetooth LE, WiFi, WiFi-Direct, 3G/4G. • Ad-hoc networking: Pure ad-hoc network connections, i.e, to be able to perform
communication without assistance of pre-installed infrastructure. The infras-tructure may not be available at a certain time or may be overloaded, in that case alternatives are required to perform communication.
• App-level Routing: As multiple technologies are being used, every message must be able to traverse distinct network interfaces, transparently. To do so, an app-level routing mechanism is needed. Note that the devices are non-rooted which is another motivation to do the routing at app-level. For some technologies, like WiFi the routing can be a mix of network-level and app-level since sockets already allows to pass through several nodes.
• Abstract communication: The middleware must be technology agnostic which means that does not matter which technology is being used to route a specific
message. From the developer’s perspective the only interface available is a
send/receive of bytes. Another motivation for this abstraction is to hide the complexity of traversing distinct wireless technologies.
• Basic Services: As a general purpose middleware it must provide a few basic services like computing, storage, publish/subscribe (P/S) and streaming. The services are important since they allow to discover peers that are interested on the same subject e.g. a video. Besides being technology agnostic, the services are a simpler form to listen to changes (P/S) or to distribute work, like computing and/or storage.
• Open Source: The framework/SDK must be open. It’s easier to expand/change an open source project in order to fix bugs, to adapt to new situations and also to add new features.
Open Garden Serval P. AllJoyn MultiPeer Open Peer P2pKit Nearby
Open Source 7 X X 7 X 7 X
Non-rooted X 7 X X X X X
Multiple Technologies X 7 7 X 7 X X
Ad-hoc Network X X 7 X 7 X 7
Multi Hop Network X X X X X X 7
App Level Routing X 7 X X 7 7 7
Abstract Comm 7 7 7 X 7 7 7
Multi platform X 7 X 7 X X X
Basic Services 7 7 7 7 7 7 7
Table 2.1: Frameworks/SDKs available.
Table 2.1 depicts a summary about the requirements, we wish, against the frameworks available. First of all, we eliminate the frameworks that are not open source, Open
