F
ACULDADE DEE
NGENHARIA DAU
NIVERSIDADE DOP
ORTOMulti-Technology Router for Mobile
Networks: Layer 2 Overlay Network
over Private and Public Wireless Links
Helder Martins Fontes
Report of Dissertation
Master in Informatics and Computing Engineering Supervisor: Prof. Manuel Alberto Pereira Ricardo (PhD) Second Supervisor: Tânia Cláudia dos Santos Calçada (MSc)
Overlay Network over Private and Public Wireless Links
Helder Martins Fontes
Report of Dissertation
Master in Informatics and Computing Engineering
Approved in oral examination by the committee:
Chair: António Augusto de Sousa (PhD)
External Examiner: José Manuel Torres (PhD)
Internal Examiner: Manuel Alberto Pereira Ricardo (PhD)
Abstract
In the last decade we have assisted to a significant evolution on the mobile wireless net-work technologies; they have become faster and more economically accessible. Nowa-days, multiple wireless network technologies co-exist in the same area; some used in public networks, provided by ISPs, and others used in private networks. There are in-creasingly more sites providing public wireless Internet access. Terminal equipments are also becoming smaller, cheaper and having applications that take advantage of the network access. The number of wireless clients is increasing along with the usage of pro-vided network services. The idea of providing Internet access and network based services to buses and their passengers has come up in this context and is the basis of the QREN Project SitMe.
This dissertation is focused on designing the architecture of the SitMe project’s com-munications system and evaluating implementation strategies for the multi-technology routers to be installed on buses. The architecture has to deal with multiple wireless net-work technologies, switches between them, terminals mobility, and support cover the entire public transportation fleet. An architecture with similar requirements was found, the WiMetroNet, but does not support public links. This dissertation specifies an com-munication architecture based on WiMetroNet, identifying the changes to implement to support the project requirements and modules to develop. Was developed a prototype, for proof of concept, implementing the most relevant module concerning control plane adap-tation for public links support. To make a decision on the best implemenadap-tation strategy for the run-time environment of the router machine, on which video services will also run in the bus, were studied virtualization and simulation technologies for fast-prototyping.
The goals of this dissertation were achieved with results that validate the concept of the innovative architecture components that allows to support public and private links. On the implementation strategy, the use of virtualization and simulation technologies has been proven to be efficient, allowing to run different operating systems and also take advantage of fast-prototyping techniques.
Na última década temos assistido a uma evolução significativa nas tecnologias de redes móveis sem fios; tornaram-se mais rápidas e mais economicamente acessíveis. Nos dias que correm, múltiplas tecnologias de redes sem fios coexistem na mesma área; algumas são usadas em redes públicas, fornecidas por ISPs, e outras usadas em redes privadas. Existem cada vez mais sitios a dar acesso público sem fios à Internet. Os equipamentos terminais estão também a ficar mais pequenos, baratos e possuindo aplicações que tiram partido do acesso à rede. O número de clientes sem fios está a aumentar conjuntamente com a utilização de serviços de rede disponibilizados. A ideia de disponibilizar acesso à Internet e serviços de rede em autocarros e aos seus passareiros surgiu neste contexto e é a base do projecto SitMe do QREN.
Esta dissertação está focada em projectar a arquitectura do sistema de comunicações do projecto SitMe e avaliar estratégias de implementação para os routers multi-tecnologia a serem instalados nos autocarros. A arquitectura tem que lidar com múltiplas tecnolo-gias de redes sem fios, mudanças entre elas, mobilidade de terminais e suportar cobrir toda a frota de transportes públicos. Uma arquitectura com requisitos idênticos foi en-contrada, a WiMetroNet, mas não suporta ligações públicas. Esta dissertação especifica uma arquitectura de comunicações baseada na WiMetroNet, identificando as alterações a implementar para suportar os requisitos do projecto e os módulos a desenvolver. Foi de-senvolvido um protótipo, para prova de conceito, implementando o módulo mais relevante que trata da adaptação do plano de controlo para suportar ligações públicas. Para tomar uma decisão acerca da melhor estratégia de implementação para o ambiente de execução da máquina do router, no qual correrão também serviços de video dentro do autocarro, foram estudadas tecnologias de virtualização e de simulação para rápida prototipagem.
Os objectivos desta dissertação foram alcançados com resultados que validam o con-ceito de componentes inovadores da arquitectura que permitem suportar ligações privadas e públicas. Quanto à estratégia de implementação, o uso de tecnologias de virtualização e simulação foi provado ser eficiente, permitindo correr diferentes sistemas operativos e também tirar partido de técnicas de rápida prototipagem.
Acknowledgements
I would like to express my total gratitude to all the extraordinary people which made possible the development of this dissertation in the investigation environment of INESC Porto.
I feel pleased to have integrated the WiN (Wireless Networks) group of the Telecom-munications and Multimedia Unit at INESC Porto, whose elements unconditionally helped and supported the various steps of this dissertation. They would be too many to reference separately in this short part, but their contribution is present all over this work.
I would especially like to express my large gratefulness towards my supervisors Tâ-nia Calçada and Prof. Manuel Ricardo for their friendliness and absolutely outstanding support. Thanks to them, and the other elements, work had always been an object of personal interest, reflection, and constructive discussion. I owe them the wireless net-works knowledge they transmitted during this months and the patience to answer all my questions.
As a last point, I would like to acknowledge all my family, friends and specially my parents who, in some way, made it possible for me to be here through their unconditional support in all situations of my life.
1 Introduction 1
1.1 Contextualization . . . 1
1.2 The SitMe Project . . . 2
1.3 Motivation and Goals . . . 2
1.4 Results . . . 3 1.5 Document structure . . . 3 2 The Project 5 2.1 Project description . . . 5 2.2 Requirements . . . 6 2.2.1 Connectivity requirements . . . 6 2.2.2 Security requirements . . . 7 2.2.3 Mobility requirements . . . 8
2.3 Scenarios and Use Cases . . . 8
2.3.1 Passenger connects to the bus network . . . 8
2.3.2 Passenger changes bus, maintaining connectivity . . . 9
2.3.3 Passenger accesses Internet contents . . . 9
2.3.4 Passengers communicate between them . . . 10
3 State of the art 11 3.1 Related Work . . . 11
3.1.1 Icomera Moovbox . . . 11
3.1.2 OpenVPN . . . 12
3.1.3 TRILL . . . 13
3.1.4 WiMetroNet . . . 14
3.1.5 Comparison and Conclusions . . . 18
3.2 Network Protocols for over IP Data Transportation . . . 18
3.2.1 RAW IP . . . 19
3.2.2 TCP . . . 19
3.2.3 UDP . . . 20
3.2.4 Comparison and Conclusions . . . 20
4 Proposed Architecture 22 4.1 Overview . . . 22
4.2 RBridge’s architecture . . . 24
4.2.1 Core RBridge . . . 24
CONTENTS
4.2.3 Bus RBridge . . . 25
4.3 Data Plane . . . 26
4.4 Control Plane . . . 28
4.4.1 RBridge Operation Modes . . . 29
4.4.2 Centralized L3 HELLO System . . . 31
4.4.3 Centralized L3 Flooding System . . . 32
4.4.4 TC Messages’ Link Weight field . . . 33
4.4.5 Detailed RBridge operation mode behavior . . . 34
5 Architecture Validation 37 5.1 Centralized L3 HELLO System . . . 37
5.1.1 Exchanged Messages Format and Protocol . . . 38
5.1.2 Server Module . . . 41
5.1.3 Client Module . . . 44
5.1.4 Tests and Results . . . 45
5.2 Conclusions . . . 52
6 RBridge implementation strategy 54 6.1 Constraints . . . 54
6.2 Decision on the run-time environment . . . 55
6.3 Virtualization scenarios . . . 55
6.3.1 USB pass-through . . . 56
6.4 Validation . . . 56
6.4.1 Pre-established QoS requirements . . . 56
6.4.2 Test scenarios . . . 57
6.4.3 Testbed definition . . . 58
6.5 Results analysis and Conclusions . . . 60
7 Conclusions and Future Work 62 References 65 A Minimum required Internet access speed 67 A.1 Bus capacity and percentage of passengers using the Internet service . . . 67
A.2 Internet usage profile per passenger . . . 67
A.3 Internet bandwidth usage during a day . . . 67
A.4 Internet traffic . . . 68
B Centralized L3 HELLO System execution log 72 B.1 Server . . . 72
B.2 Client 1 . . . 74
B.3 Client 2 . . . 75
C NS3 Node performing routing operations 77
2.1 SitMe Communication System’s Simplified Architecture . . . 6
3.1 Icomera Moovbox M200 Ruggedized Equipment . . . 12
3.2 OpenVPN possible implementation scenario . . . 13
3.3 The WiMetroNet reference network . . . 15
4.1 SitMe Communication System’s Overview . . . 23
4.2 Core RBridge connections to Internet, WiMax base station and the servers of network centralized services. . . 24
4.3 Bus RBridge . . . 25
4.4 Communication stack used on (a) terminal communications, (b) private links, and (c) public links. . . 27
4.5 SitMe Data Plane. Internal and Internet access communications, using different technology interfaces available on BUS RBridges. . . 28
4.6 RBridges Operation Modes: L2, L3, Hybrid and:(a) Island L2, or (b) Hybrid Island. Private links are represented in blue and public in brown. . 30
4.7 State Machine Diagram for all bus RBridges’ Operation Modes . . . 31
4.8 L3 Mode behaviors (a) - Reception and retransmission; (b) transmission . 34 4.9 Island L2 mode, L2 mode and Hybrid mode behaviors. (a) Reception and retransmission; (b) transmission. . . 34
4.10 Island Hybrid Mode behaviors. (a) Reception and retransmission; (b) transmission. . . 35
4.11 core RBridge behaviors. (a) Reception and retransmission; (b) transmission. 36 5.1 Centralized L3 HELLO System’s Overview Diagram . . . 38
5.2 Centralized L3 HELLO System - Server Software Architecture . . . 41
5.3 Centralized L3 HELLO System - Client Software Architecture . . . 44
5.4 Server is started with an empty database of neighbors. . . 48
5.5 Client 1 is started and registers its public link. . . 48
5.6 Server receives the Link Registration message. . . 49
5.7 Client 1 received the Link Syncronization with its own link, registers it locally, and sent an acknowledgment with the same time-stamp. . . 49
5.8 Server receives the acknowledgment. . . 49
5.9 Client 1 remains updating its link periodically. . . 49
5.10 Client 2 registers its two links, one per thread, sending Link Registration messages to the server . . . 50
LIST OF FIGURES
5.11 Server receives the interfaces registration and now sends the Links Full Syncronization message with all public links to client 2 (using its less
weighted link), and only the incremental updates to client 1 (figure X). . . 50
5.12 The clients receive the updates, acknowledges the reception to the server, and the databases are synchronized again. . . 51
5.13 The two clients continue to register their links, not allowing them to expire in the server database. . . 51
5.14 Client 1 is closed, simulating that its public link is unavailable. Server detects that client closed and the server entries expire and are sent incre-mental updates with this deleted link, for the less weighted link of the client 2. . . 52
5.15 Client 2 receives that incremental update, acknowledging it to the server. . 52
6.1 Two possible virtualization scenarios for implementing Communication and Video modules in the same hardware; Windows Host (a) and Ubuntu Host (b). . . 56
6.2 Network Flows to calculate bandwidth requirements . . . 57
6.3 Testbed overview . . . 58
6.4 Communication Module Kernel (a) and NS3 (b) routing. . . 59
6.5 Aggregated goodput results for each test scenario on the Communication Module . . . 60
3.1 Similar Solutions Comparison table . . . 18
3.2 RAW IP encapsulation stack . . . 19
3.3 TCP encapsulation stack . . . 20
3.4 UDP encapsulation stack . . . 20
3.5 RBridges’ public links socket comparison . . . 21
5.1 Upstream network traffic per public link . . . 50
A.1 Bus capacity and percentage of passenger using the Internet service. . . . 67
A.2 Different Internet traffic flows, required bandwidth and usage frequency. . 68
A.3 Average upstream and downstream bandwidth used by each passenger accessing Internet. . . 68
A.4 Number of bus passengers during a day . . . 69
A.5 Number of bus passengers accessing Internet during a day. . . 70
A.6 Required Internet bandwidth during a day. . . 71
A.7 Internet traffic per day. . . 71
Abbreviations
AAA Authentication, Authorization and Accounting ARP Address Resolution Protocol
Bluetooth Wireless technology by IEEE 802.15.1 DHCP Dynamic Host Configuration Protocol
DNS Domain Name System
DoS Denial of Service
EUI-48 Ethernet Uniq Identifier 48bits EV-DO Evolution-Data Optimized
FEUP Faculdade de Engenharia da Universidade do Porto GPRS General Packet Radio Service
GPS Global Positioning System
GSM Global System for Mobile Communications HSDPA High-Speed Downlink Packet Access IETF Internet Engineering Task Force
INESC Porto Instituto de Engenharia e Sistemas de Computadores do Porto
IP Internet Protocol
IPTV Internet Protocol television ISP Internet Service Provider
LAN Local Area Network
MPLS Multi Protocol Label Switching
NAT Network Address Translation
OLSR Optimized Link State Routing Protocol OpenVPN Open Source Virtual Private Network
OSI model Open System Interconnection Reference Model OSPF Open Shortest Path First
PAT Port Address Translation PLFF Public Links Forwarding Flag
QoS Quality of Service
QREN Quadro de Referência Estratratégico Nacional RBridge Routing bridge
SitMe Serviços Integrados para Transportes Metropolitanos STCP Sociedade de Transportes Colectivos do Porto, SA
STP Spanning Tree Protocol
TAP Virtual interface that can access layer 2 data TCP Transmission Control Protocol
TTL Time to Live
TUN IP Tunnel virtual interface for IP over IP encapsulation UDP User Datagram Protocol
UMTS Universal Mobile Telecommunications System WAN Wide Area Network
WiMax Worldwide Interoperability for Microwave Access, by IEEE 802.16 WLAN Wireless LAN by IEEE 802.11
Chapter 1
Introduction
1.1
Contextualization
In the last decade we have assisted to a significant evolution on the mobile wireless net-work technologies. They are becoming faster and more economically accessible to imple-ment and use. Nowadays, multiple wireless network technologies are available and co-exist in the same area. Examples of those technologies are GSM, GPRS, UMTS/HSDPA, EV-DO, WLAN, WiMax and Bluetooth.
There are many wireless network technologies available, although they are not used in the same context. On one hand we have public networks, such as UMTS, EV-DO and WiMax, provided by telecom operators, operating on a licensed radio spectrum. On the other hand, we have private networks, such as WLAN, Bluetooth and WiMax operat-ing on unlicensed radio spectrum. Public network technologies like UMTS have higher geographic coverage, higher implementation and maintenance cost and lower bit-rates than technologies such as WLAN, used normally to implement private networks. WLAN [IEE07] is the reference technology to implement wireless local networks. There are increasingly more sites providing public Internet access through WLAN Access Points. Terminal equipments that access these networks are also becoming smaller, cheaper and having applications that take advantage of the wireless network access. The number of wireless clients is increasing along with the usage of network services provided by those connections.
The idea of providing Internet access and network based services to metropolitan buses and their passengers, improving its traveling experience, has come up in the con-text described above and is the basis of the QREN Project SitMe where this dissertation is integrated. Internet access on buses makes possible to support dynamic advertisement and information, updated in real time through the Internet. The buses are inter-connected through a network and also their passengers’ terminals connected to it.
1.2
The SitMe Project
This main goal of the QREN SitMe project is to develop and supply information services to passengers traveling in buses. These services include (1) display news and entertain-ment information, (2) supply inter-modality information (passenger travel assistance), (3) interactive services, (4) supply Internet access to passengers using the bus internal wireless network. For that purpose, a multi-technology communications system is being developed. The project has secondary goals such as (1) to deploy a pilot in 10 STCP buses operating in a bus line near FEUP, (2) run tests in the pilot with bus passengers, (3) evaluate the technical and financial viability of the developed solutions. The partners of this project are: FEUP1, INESC Porto2, FEP3, Xarevision4and STCP5.
FEUP and INESC Porto tasks are related to the multi-technology communications system design, development, and test. These tasks deal with a scenario with multiple wireless network access technologies (UMTS, WLAN and WiMax), switches between them, and terminals mobility. The project architecture has to support future network expansion to cover the entire public transportation fleet.
This dissertation is focused on designing the architecture of the communications sys-tem and evaluating implementation strategies for the network access equipment (multi-technology router) to be installed on buses.
1.3
Motivation and Goals
After doing an extended research in the Internet and consulting the mobile networks area specialists at INESC Porto, the conclusion was that it didn’t exist, at the time of start of this dissertation, any project or technology already implemented and running satisfying all the requirements of the SitMe project. Nevertheless, a further study in this matter is addressed on Chapter3, where similar solutions are introduced.
The objective of this dissertation is to study the available technologies and imple-mentations and specify a solution that can be used in this project. By the end of this dissertation, the architecture of a communications system is expected to be available as well as a strategy defined for implementing communications routing equipments. During the validation process, the necessary tests should be carried out, yielding scientific results that evaluates the liability of the decisions made.
1http://www.fe.up.pt/ 2http://www.inescporto.pt/ 3http://www.fep.up.pt/ 4http://www.xarevision.pt/ 5http://www.stcp.pt/
Introduction
1.4
Results
This dissertation specifies the architecture of a communication system that supports (1) multi-technology access links (UMTS, private/public WLAN and private/public WiMax), (2) ad-hoc WLAN links, (3) seamless inter-technology and intra-technology handovers provided by layer 2 overlay networking, (4) mesh topology between network access equipments, (5) terminal and network mobility, (6) Internet access provisioning, (7) ter-minal auto-configuration, (8) of-the-shelf networking hardware and unchanged terter-minals, (9) network element authentication, and (10) thousands of nodes scalability. The pro-posed architecture intends to be used on a vehicular network which will be deployed by installing communication equipment on buses and other vehicles.
The proposed architecture is based on WiMetroNet which is the architecture described in [RCF+10,CFDR10]. This dissertation identifies the required changes to support pub-lic links (e.g. UMTS) as part of the network architecture; in addition to the private link support defined by WiMetroNet. Two innovative architecture components that allows to support public links are defined. Was developed a prototype, for proof of concept, imple-menting the most relevant module concerning the adaptation for public links support.
This dissertation studies the run-time environment for the communication equipment to be installed on buses. To make a decision on the best implementation strategy for the run-time environment of the router machine, on which video services will also run in the bus, were studied virtualization and simulation technologies for fast-prototyping.
The goals of this dissertation were achieved with results that validate the concept of the innovative architecture components that allows to support public and private links. On the implementation strategy, the use of virtualization and simulation technologies has been proven to be efficient, allowing to run different operating systems and also take advantage of fast-prototyping techniques.
1.5
Document structure
The document structure is the following. Chapter2introduces the SitMe project charac-teristics describing the problem addressed by this dissertation; connectivity, security and mobility requirements are here listed, followed by the use cases identified for each com-munication scenario. Chapter 3addresses the state-of-the-art study; a study of network protocols for over IP data transportation is also presented. Chapter 4 presents detailed information about the communications system architecture. On Chapter 5a implemen-tation of Centralized L3 HELLO System is presented as a validation of the proposed architecture; evaluation of control data overhead and their impact on public links are pre-sented. Chapter6addresses RBridge implementation strategy identifying and evaluating
constraints of the SitMe project for the on board equipment, running inside the bus. The document is concluded on Chapter7presenting conclusions and future work.
Chapter 2
The Project
In order to better introduce the SitMe project and understand what are the goals of this dis-sertation, in this chapter is presented a detailed description on the SitMe Communication System.
2.1
Project description
This dissertation was developed within the QREN project SitMe1. The SitMe project aims to develop a solution to provide information services access to passengers travel-ing on buses and implement it on a demonstrative pilot in 10 STCP’s buses. Several services are offered to passengers including access to updated news and entertainment in-formation, access to inter-modality inin-formation, interactive services and WLAN Internet access. Most of these services will be provided by an equipment named Xarepoint Mobile which includes a network access equipment and the Xarepoint equipment which provides video services and has an attached LCD panel. On figure 2.1 the project’s simplified architecture is shown.
Each bus uses heterogeneous wireless network technologies to external communi-cations including UMTS, WiMax, infra-structured WLAN and WLAN mesh networks. Intelligent technology selection mechanisms are provided based on economical and tech-nical aspects. Technology and interface switching, due to bus mobility, economical re-strictions or link quality, is seamless to terminals inside the bus, i.e. their communication sessions, are not affected.
The network architecture aims to place all buses inside the same virtual private net-work with LAN characteristics building a layer 2 overlay netnet-work available to terminals inside the bus. Through the SitMe overlay network, terminals can establish local and In-ternet communications without specific software requirements. Xarepoint connection to SitMe virtual private network is not affected by local traffic on the bus WLAN access.
Figure 2.1: SitMe Communication System’s Simplified Architecture
SitMe provides a private WiMax base station, centralized network services like DNS, DHCP and AAA Servers, and Internet access using FEUP NET.
2.2
Requirements
The communication system has connectivity, security and mobility requirements. The communication solution may be inspired on ad-hoc/mesh networking techniques being each BUS a network node. The network will be built with available wireless network technologies (WLAN, UMTS and WiMax). The communications layer should guarantee an ‘Always on’ service supporting system management information, broadcast of audio-visual contents, inter-modality information and data communications to passengers (In-ternet access and interactive services).
To implement a communication system that integrates all the features presented on the previous section, it has to obey to the following identified requirements.
2.2.1 Connectivity requirements
• SitMe should be a private IP network (private IP range);
• Each bus will have a network access equipment with a unique identifier;
• Inside each bus should be supplied an WLAN network (WLAN Access Point) and an Ethernet interface, which will be used, respectively, by passengers’ terminals and by Xarepoint to access SitMe network. The Xarepoint should be able to access SitMe network independently from the WLAN access provided for the passengers;
The Project
• Network access equipment should support on its access interfaces multi-technology wireless networks: UMTS, WiMax and WLAN. While the UMTS interfaces will be always connected to a public ISP network, the WiMax and WLAN can be connected to private networks, owned and managed by SitMe;
• The WiMax wireless network will be provided by a base station IEEE 802.16e that will be connected to the SitMe private network;
• Network access equipment should support intelligent interface selection, based on metrics like bandwidth and operation financial cost;
• The network formed by buses should have mesh topology;
• Every network access equipments must be in the same overlay network, indepen-dently from their active interfaces;
• One or more bus stops and even city areas may have SitMe network access through WLAN coverage, which will give network access to the terminals similar to the coverage existing on buses;
• Terminals feel like they are in the same LAN, one hop away from each other; • Every entities inside SitMe network should have Internet access, where NAT can be
used;
• WLAN access interfaces should support ad-hoc mode between buses and station mode to access WLAN hot spots;
• No additional software should be necessary on terminals. They need to support only the IP protocol stack;
• Should not be necessary to manually configure the IP address, network mask, DNS server and default gateway for each terminal (network auto-configuration);
• Expected usage values on Internet access require a minimum bandwidth of 76 kb/s for upstream and 1225 kb/s for downstream;
• The expected daily network traffic is 360 MByte for upstream and 5866 MByte for downstream, a total of 6226 MByte.
In AppendixAare presented, in detail, all the parameters used for this estimation.
2.2.2 Security requirements
• System should support user identification and authentication to access SitMe net-work if the business model and legal restrictions require that control;
• There exist equipments connected to the SitMe network without the need for au-thentication before the network (i.e. Xarepoint);
• If authentication is used, the content of WLAN connections should be encrypted, helping the users privacy;
• The WiMax base station must support authentication and data encryption mecha-nisms.
2.2.3 Mobility requirements 2.2.3.1 Network mobility
During the mobility of SitMe’s network access equipments, it may be necessary to switch the wireless technology used based on the coverage that each one has. Seamless network technology switching must be supported so the terminals connectivity to the SitMe/Internet network is not affected.
2.2.3.2 Terminals mobility
Communication system must support terminals mobility between buses and between a bus and a bus stop covered by SitMe WLAN. The terminals’ communication sessions must be maintained. Mobility scenarios between buses may occur while a passenger change between buses and there are other buses near enough to not loose SitMe network coverage.
2.3
Scenarios and Use Cases
In this Section are described the scenarios and use cases identified for this project. 2.3.1 Passenger connects to the bus network
Actors
Passenger; Bus network equipment; Central network equipment. Description
A passenger, traveling inside a bus, can connect to SitMe network, available on the bus, through a simple search of wireless network available and request of connection to the SitMe. The network will detect the terminal and configure it automatically allowing network communications.
The Project
Use Cases
• Bus network equipment announces the network; • Passenger requests connection to the network;
• Bus network equipment forwards the connection request to the central network equipment;
• Central network equipment authenticates and registers the terminal;
• Central network equipment sends the network configuration to the terminal. 2.3.2 Passenger changes bus, maintaining connectivity
Actors
Passenger; Bus network equipment.
Description
After being connected, the network configurations given to the terminal will be valid and sufficient so that it continues to enjoy the network services without any connectivity loss, as long as it remains under the coverage of, at least, one SitMe network equipment. The network will begin handover processes when necessary, communicating between net-work equipments, making this process seamless to the terminal.
Use Cases
• Passenger changes location for other also covered by SitMe network;
• New bus network equipment detects the new terminal and announces the new ter-minal location to the network;
• All network equipments are aware of the new terminal location. 2.3.3 Passenger accesses Internet contents
Actors
Description
Probably the most common scenario will be the bus passenger accessing Internet con-tents, that may be their e-mail, web pages, multimedia, etc. The network will forward its communications, independently from the network technology used, to the central network equipment that acts as an Internet Gateway.
Use Cases
• Passenger accesses Internet contents, being able to interact whit them;
• Bus network equipment forwards communications through the best available net-work paths until they reach the central netnet-work equipment;
• Central network equipment acts as an Internet Gateway. 2.3.4 Passengers communicate between them
Actors
Passenger; Bus network equipment; Central network equipment. Description
The SitMe network, being an layer 2 overlay network, helps to establish private com-munication between terminals, like peer-to-peer (P2P) and client-server. This function-ality can be used to deploy intranet pages and interactive contents. The layer 2 overlay concept brings the ability to establish local communications, taking advantage of local connections with more available bandwidth.
Use Cases
• Passenger uses a network service whose content is requested/provided by a local network terminal;
• Bus network equipment forwards communications through the best path available to reach the other bus network equipment where is the destination terminal;
• Destination terminal communicates with the passenger’s terminal as they were di-rectly connected, in the same LAN segment.
Chapter 3
State of the art
Chapter 2 introduced the SitMe project and its requirements. Now, there is the need to study other similar technological solutions to obtain an overview of what exists, to better understand the problem and get some useful ideas, and to help determine the best candidate base architecture for SitMe network. This chapter also addresses the study of network protocols for over IP data transportation, selecting what is the best to use in SitMe.
3.1
Related Work
This section addresses the analysis of technological solutions sharing characteristics with SitMe’s communication system. The main characteristics include supporting (1) multi-technology access links (UMTS, private/public WLAN and private/public WiMax), (2) ad-hoc WLAN links, (3) seamless inter-technology and intra-technology handovers pro-vided by layer 2 overlay networking, (4) mesh topology between network access equip-ments, (5) terminal and network mobility, (6) Internet access provisioning, (7) terminal auto-configuration, (8) of-the-shelf networking hardware and unchanged terminals, (9) network element authentication, and (10) thousands of nodes scalability.
The scope and depth of this analysis are oriented by previously identified require-ments. This section is concluded with a comparison of available solutions and the selec-tion of the best approach to follow and evolve to project architecture.
3.1.1 Icomera Moovbox
The Icomera Moovbox M200 [Ico], presented on figure3.1, is an WLAN router with NAT technology, using two mobile networks as WAN interfaces. It is a ruggedized1hardware
1Ruggedized is a term used to describe a electronic device specially designed to be robust on a environment with
Figure 3.1: Icomera Moovbox M200 Ruggedized Equipment
equipment plus proprietary software solution designed for mounting inside vehicles, in-cluding public transportation. As WAN interfaces, it supports UMTS/HSDPA, EV-DO and WiMax. It has an integrated GPS, used to keep track of the vehicles current location. It is compatible with in vehicle power supplies and it supports connection to an IPTV to show, for example, streamed video commercials.
Icomera solution has also a centralized information system to keep track of bandwidth utilization, number of users, vehicles location, configure a landing page2, push advertise-ment to passengers browser and keep track of clicked ones.
3.1.1.1 Comparing with SitMe
Although it seems a complete solution, it does not support seamless interface switching, mobility of terminals, ad-hoc WLAN networks, mesh topology networks, establishment of the overlay L2 network and also LAN application between two or more distinct equip-ments.
3.1.2 OpenVPN
OpenVPN [Ope] is a free and open source virtual private network solution that allows client-to-client and server-to-multi-client connections, placing them into the same over-lay network. It supports the establishment of virtual Ethernet (over-layer 2) or IP (over-layer 3) links, through the use of TUN or TAP virtual interfaces, allowing the exchange of Eth-ernet frames or IP packets. This is achieved by the encapsulation of the real EthEth-ernet or
State of the art
Figure 3.2: OpenVPN possible implementation scenario
IP traffic inside TCP or UDP packets that, in conjunction with other techniques, makes the traffic pass through routers, firewalls and NAT, being compatible with Internet links. Note that communication between clients is only possible through the server. OpenVPN also supports client authentication and data encryption mechanisms which ensures their identity and data confidentiality. To make use of this solution it is necessary to install the OpenVPN software on both client and server machines.
3.1.2.1 Comparing with SitMe
The OpenVPN implementation scenario presented on figure 3.2 uses virtual Ethernet links, a OpenVPN client per bus equipment interface, and a centralized OpenVPN server with access to the Internet and the WiMax base station. This OpenVPN implementation scenario would lead to the creation of the layer 2 overlay network needed in SitMe. Al-though, redundant layer 2 links and consequent loops would appear. The activation of the STP solves the loops problem but, with a very dynamic network topology and a stan-dard STP convergence time of fifty seconds [IEE98] would lead to intermittent network connection and clients sessions disruption. Ad-hoc links are not covered by OpenVPN solution, and mesh topology would be unexploited with the impossibility of direct com-munication between buses.
3.1.3 TRILL
TRILL [TPPT09] is an IETF protocol developed to address the limitations of very large LANs requiring STP to avoid layer 2 loops. The STP has problems [Dan00] like high
convergence time and the use of inefficient paths, which lead to intermittent layer 2 con-nectivity and bandwidth waste, respectively. TRILL presents a solution to this problem applying the concepts of the layer 3 routing protocols to the layer 2 maintaining the back-wards compatibility of the existing layer 2 network elements. It uses the IS-IS [Ora90], a modern link state routing protocol3 similar to OSPF [Moy98], to propagate the network topology and calculate the best layer 2 paths for each Ethernet frame destination. The TRILL is implemented by network equipments called RBridges.
3.1.3.1 RBridge concept
On OSI model [Zim80], RBridges [Per04] operate between layer 3 as routers and layer 2 as bridges. An RBridge connects multiple network segments at the layer 2 and performs routing functions using RBridge’s identification tags, instead of IP addresses. Similarly to routers, RBridges terminate STP trees. When calculating the best route for each desti-nation, the RBridge selects the interface to use for each route based on its cost.
There are three kinds of network traffic operations performed by the RBridges: ingress, routing and egress. Ingress occurs when an Ethernet frame comes from a standard layer 2 link and reaches an RBridge; the RBridge encapsulates the frame and sends it to the next RBridge on the path to the destination. Routing occurs when an RBridge receives an already encapsulated frame and forwards it to the next RBridge, according to the pro-tocol routing table. Egress operation happens when the encapsulated frame reaches the destination RBridge which decapsulates and injects it on the layer 2 link.
3.1.3.2 Comparing with SitMe
This protocol was developed to work on very large LAN environments where the links are private and layer 2 is accessible. TRILL is not prepared to run over public Internet links like UMTS, where layer 2 is controlled and accessible only by the ISP. This means that it could not create the needed layer 2 overlay network over private and public links. Also, the IS-IS protocol is based on reliable topology propagation, which is not possible in the wireless ad-hoc scenarios of the SitMe project.
3.1.4 WiMetroNet
WiMetroNet [RCF+10] presents an architecture for a large mesh network of moving RBridges operating over heterogeneous wireless technologies to provide broadband ac-cess to vehicles, stops, and passengers of a public transportation system. Figure 3.3
presents the WiMetroNet reference network diagram.Each vehicle and stop are equipped
3A link state routing protocol is one where each router announces its network topology to all the other routers and,
at a given time, it can calculate the best path to reach other router, applying a minimum path algorithm (like Dijkstra) to the topology database.
State of the art
Figure 3.3: The WiMetroNet reference network
with an RBridge. RBridges form a mesh network using wired links, WiMax and WLAN. UMTS is also considered as a backup access technology. Fixed RBridges (e.g. RBridges on stops) may be connected using wired links and may provide WLAN connections to moving RBridges. Terminals access the network through WLAN or Bluetooth. Internet access is provided centrally by Internet Gateway using NAT.
Mobility of terminals between different RBridges and seamless switching between network technologies are supported by WiMetroNet, allowing terminals to keep their connections when they move through the network and when RBridges change their ac-cess technology. These features are supported without changing the network software on the terminals. All the intelligence is on the network side, terminals only need to support WLAN interfaces and a bare IP communications stack.
WiMetroNet architecture assumes that most of the metropolitan area has private WiMax coverage. WLAN is used, when possible, to achieve higher bit-rates and free WiMax bandwidth to the network equipments without WLAN coverage. In WiMetroNet all core communications are assumed to be carried out always through private links (wired, WiMax or WLAN) provided internally. UMTS is considered as backup solution to areas without WiMax coverage, however this possibility is not described nor implemented.
WiMetroNet uses the foundation concepts of TRILL, using also RBridge equipments, but running WMRP [RCF+10] routing protocol designed to mobile networks.
WiMetroNet architecture defines a data plane and a control plane. Data plane speci-fies how frames are switched based on a forwarding table defining packet encapsulation and decapsulation. Control Plane exchanges network topology information in order to calculate and obtain the best paths to other nodes and compute the forwarding table. A
detailed behavior of data and control planes are presented next. 3.1.4.1 Data plane
WiMetroNet data plane introduces a layer 2.5 header to carry a TTL field required to avoid loops. This new header should be small to avoid excessive overhead, which led to the adoption of the standard MPLS header for WiMetroNet data plane. On top of the MPLS layer the original layer 2 frame is encapsulated.
When a ingress operation takes place, the user data frame is encapsulated in an MPLS header containing the egress RBridge 20bit ID of and non zero TTL value, and transmitted through the outgoing interface. When the MPLS frame arrives at an egress RBridge it is “decapsulated” and the original user frame transmitted to the destination station.
The original destination MAC address is used to look-up the egress RBridge in a local terminal location database, which is a table mapping MAC addresses of known terminals to RBridges. Knowing the egress RBridge, the routing table is then used to look-up a label and an outgoing interface. [RCF+10] details the behavior of WiMetroNet data plane.
The use of a MPLS header carrying the destination RBridge label results in a layer 2 overlay network built on top of dynamic and multi-technology network links existent between RBridges. Original layer 2 frames are routed by RBridges until they reach the destination terminal, which is layer 2 attached to the egress RBridge.
3.1.4.2 Control plane
For the control plane, WiMetroNet runs a routing protocol, the Wireless Metropolitan Routing Protocol (WMRP). WMRP is a proactive link state routing protocol for ad-hoc networks, based on OLSR [CJ03] which uses techniques of neighbor discovery and topol-ogy propagation in a unreliable link environment. WMRP runs over layer 2, instead of operating at layer 3 like OLSR; RBridges IDs are used to map network topology instead of IP addresses.
WMRP introduces changes to the OLSR related to network scalability, making pos-sible to support large networks [RCF+10]. In WMRP the location of terminals and of RBridges is treated separately; different databases and different routing messages are em-ployed for each one. HELLO and Tra f f ic Control messages are used to maintain the RBridges location; MAC Control and IP Control are used to fill terminals’ databases. HELLO messages are used for link sensing, i.e. to allow nodes to be discovered by
their neighbors. HELLO messages are broadcasted to each link, periodically (2 seconds, by default) by each node, but are never forwarded. Through this process, each RBridge knows their local network topology.
State of the art
Traffic Control Messages (TC) are used by each node to advertise through the network the list of links to neighbors it has discovered, along with metrics associated with those links. TC messages’ contents is a vector of 32-bit fields; 20 of those bits represent the node id of a neighbor that has been found, while the remaining 12 bits store the link weight. Neighbor node IDs are discovered by listening to HELLO messages, while link costs is a linear combination of factors such as bandwidth, delay, link usage price, and stability. TC messages are generated periodically by each node and retransmitted once by other nodes, until reaches every node in the network. At a given moment, each RBridge has a TC per each existing RBridge and uses this information to calculate the best path to reach every other RBridge using Dijkstra algorithm.
MAC Control Messages (MC) are similar in purpose to TC, but instead of advertising other RBridges it advertises a list of attached end-user terminals, each terminal rep-resented by its MAC identifier (EUI-48). Like TC, MC messages are periodically generated and forwarded by all the other nodes. At a given moment, each RBridge has a MC per each existing RBridge and uses this information to know which RBridge has the destination MAC. Once the destination RBridge ID is known, the original Ethernet frame is encapsulated and a MPLS [RVC01] header is added con-taining the 20bit id of the destination RBridge.
IP Control Messages (IC) are used to disseminate IP-MAC associations. Typically, IC messages are generated only by RBridges directly attached to a DHCP server, using the information contained in DHCP leases. As the overlay network operates over layer 2 and all the terminals have the same sub-net IP addresses, they think that all other terminals are locally accessible. Terminals use ARP [Plu82] protocol to discover what MAC address has the destination terminal IP address. RBridges in-tercept ARP requests and forge a ARP reply using IC messages information. This mechanism is an optimization to avoid ARP broadcasts in the network increasing the architecture scalability.
3.1.4.3 Comparing with SitMe
The basic characteristics of terminal mobility and overlay network over heterogeneous wireless technologies are similar in SitMe and WiMetroNet. However, SitMe project has some limitations regarding private network coverage; WiMax area coverage is expected to be few hundred meters rather then the full metropolitan WiMax coverage considered by WiMetroNet and few bus stops may be equipped with RBridges. Due to this lack of private network coverage, the WiMetroNet backup plan of using UMTS access has become the main solution to get coverage along the bus line.
Characteristics OpenVPN TRILL WiMetroNet Moovbox SitMe
MultiTech Priv/Pub links V X X V V
Ad-hoc WLAN links X X V X V
Seamless Net handovers X V V X V
Mesh L2 topology V V V X V
Term/Network mobility V V V X V
Internet access V V V V V
Term auto-config V V V V V
Unchanged HW and Term V V V V V
Net Elements Authen. V X X V V
Scalability X X V V V
Table 3.1: Similar Solutions Comparison table
3.1.5 Comparison and Conclusions
In this section the main characteristics of the SitMe project were identified and used as a guidebook when evaluating the different similar solutions and comparing them indi-vidually with the SitMe project. This characteristics are presented in table 3.1, which summarizes the comparison of studied solutions with the SitMe project.
WiMetroNet architecture is the solution with more common characteristics with SitMe architecture when compared with other architectures analyzed. However, WiMetroNet do not support authentication of network elements and the use public links to form its network; this limitation exists because its data plane and control plane was designed to run on a metropolitan area with full private WiMax coverage, not having to deal with public links and their security risks. SitMe architecture is based on the WiMetroNet architecture since it scales to few thousands of RBridges, allowing the coverage of an entire public transportation fleet, maintaining connectivity and good convergence times. SitMe aims to build a 10 bus pilot but in a real commercial application the number of nodes can grow significantly and take full advantage of all functionalities.
3.2
Network Protocols for over IP Data Transportation
It is clear that SitMe project will need to establish core network links over operator net-works (e.g. UMTS), where layer 2 is not accessible. This mean that data encapsulated with the layer 2.5 MPLS header can not be transmitted directly through the link due to restrictions imposed by ISP’s network elements. Data must be delivered to ISP links with a standard IP header and in some cases a standard layer 4 header is also required to exist when firewalls4 or QoS policies are implemented in the ISP network; thus, the MPLS packet must be encapsulated using layer 3 and/or layer 4 protocols.
4Firewall is a technological barrier designed to prevent unauthorized or unwanted communications between sections
State of the art DATA Overlay L2 MPLS header Public IP Public L2
Table 3.2: RAW IP encapsulation stack
This section addresses the study of network protocols to encapsulate the layer 2 over-lay network data and transport it over IP, analyzing (1) network overhead, (2) ease of implementation and (3) NAT/Firewall compatibility characteristics. A comparison of en-capsulation solutions is presented as well as a selection of the best approach regarding this project requirements.
3.2.1 RAW IP
IP [Pos81a] protocol allow, as payload, very well known protocols like TCP or UDP, but also the free use of its payload. The kind of sockets that can override the TCP/UDP encapsulation and write all the IP packet payload with custom data are called RAW IP sockets. The use of raw IP sockets would lead to the use of the protocol stack presented in table3.2.
In table 3.2 is visible the direct MPLS encapsulation of the layer 2 overlay network over IP. This solution would have an extra overhead of 20 bytes [Pos81a] per packet, the size of the IP header. RAW IP sockets are oriented to the IP packets, allowing multiple IP addresses as destinations or sources per socket, for the outgoing or incoming traffic, respectively. The RAW IP socket solution would be easy to implement, as only one socket per RBridge public interface would be needed, being able to send/receive RAW IP packets from/to all other RBridges. Although it seems a good solution, RAW IP may be blocked by ISPs and it is not compatible with NAT mechanisms implementing PAT, because there is no protocol port to map in the NAT with PAT table, not allowing to distinguish more than one RBridge behind the same NAT table.
3.2.2 TCP
TCP [Pos81b] protocol works over IP and is connection oriented using streams of infor-mation exchanged between well defined endpoints. Communication using this protocol is achieved by using TCP sockets. The use of this socket type would lead to the network protocol stack presented in table3.3.
In table3.3is visible the MPLS encapsulation over TCP protocol. This solution would have an extra overhead of 40 bytes (20 for IP header + 20 for TCP header [Pos81b]) per
DATA Overlay L2 MPLS header TCP Public IP Public L2
Table 3.3: TCP encapsulation stack
packet. TCP is connection oriented, therefore each RBridge would have a socket/con-nection per neighbor using public links, multiplied by the number of public links. This is a very complex solution considering the scalability requirements previously presented. TCP protocol is compatible with NAT technology, even when used with PAT.
3.2.3 UDP
UDP [Pos80] protocol, like TCP, works also over IP but is datagram oriented. Communi-cation using this protocol is achieved by using UDP sockets. The use of this socket type would lead to the protocol stack presented in table3.4.
In table 3.4 is visible the MPLS encapsulation over UDP protocol. This solution would have an extra overhead of 28 bytes (20 for IP header + 8 for UDP header [Pos80]) per packet. UDP is not connection oriented, therefore each RBridge requires a single UDP socket per public link, independently of the number of neighbors using public links. This characteristic turns the solution scalable and easy to implement. UDP protocol is compatible with NAT technology, even when PAT is used.
3.2.4 Comparison and Conclusions
The key characteristics identified to evaluate the different encapsulation possibilities are used at table3.5to compare the previously presented solutions.
RAW IP variant is the one that offers less overhead but was discarded because RAW IP sockets are not compatible with NAT tables operating with PAT, as there are no protocol port associated. Also, a key factor to exclude this variant was the incompatibility with ISPs’ security or QoS restrictions.
DATA Overlay L2 MPLS header UDP Public IP Public L2
State of the art
Socket Type RAW IP TCP UDP
Introduced Overhead (Bytes) 20 40 28
Ease of implementation Easy Difficult Easy
NAT/Firewall Compatibility Incompatible Compatible Compatible Table 3.5: RBridges’ public links socket comparison
TCP variant is NAT/Firewall with PAT compatible but was also discarded due to its high complexity.
UDP socket is the most simple and easy solution to implement; a single socket is required per public link, to receive and send encapsulated data from/to other RBridges. The firewall compatibility should not be a problem as UDP packets are not blocked in the networks that will be used as public links.
Packet loss and order factors were not covered on the comparison because they are assured by the encapsulated TCP/IP protocol.
Proposed Architecture
This chapter presents detailed information about system architecture. An overview of the system is presented followed by communication architecture elements description. Next, RBridges types and internal architecture are presented. Finally, architecture’s data and control plane are detailed.
4.1
Overview
SitMe communication system is a large layer 2 overlay network, composed by RBridges as presented in Figure4.1.
Inside each bus, terminals are connected to the RBridge through local access networks. Fixed terminals (e.g. Xarepoint) are connected through Ethernet and mobile terminals (e.g. devices carried by passengers) are connected by WLAN provided by the RBridge’s access point. RBridges are transparent network elements from terminals’ point of view.
Wireless links used to connect RBridges can be public or private regarding how they are provided. Public links are Internet connections provided by ISPs; UMTS and public WLAN Hotspot connections are considered public links. Private links are provided by the system’s equipment; WiMax provided by the system base station, WLAN ad-hoc links between RBridges and WLAN provided by RBridges on bus stops are considered private links. A MPLS header is used as encapsulation mechanism to transport the original layer 2 frames inside the network. This results in a layer 2 overlay network built over the dynamic and multi-technology network links existent between RBridges. Using private or public link affects how data is transmitted over the overlay network as explained in3.2. RBridges run an adapted version of the WMRP, the routing protocol presented in Sub-section 4.4. When calculating the best route for each destination, WMRP intelligently selects the interface and technology to use for each route; each RBridge can use multi-ple interfaces simultaneously but not more than one per destination. The interface and
Proposed Architecture
Figure 4.1: SitMe Communication System’s Overview
technology selection considers aspects like economical cost, link quality and network performance.
Despite the mesh topology requirement listed in chapter 2, most of the network ser-vices are centralized; mesh topology is useful to enable direct communications between RBridges. Centralized services includes DHCP, AAA, DNS and Internet access. A RBridge existent at system core aggregates the centralized services and a gigabit Ethernet connection to the WiMax base station. Terminals acquire a private IP address managed by DHCP and access the Internet through a Internet Gateway located in the system core.
While moving with bus, RBridges intelligently adapt to available wireless network technologies originating handovers between UMTS base stations, roaming between WLAN access points or network technology switching. While UMTS handover should not represent connectivity problems, network technology switching and WLAN roaming could lead to sessions loss.Centralizing the Internet access using NAT with PAT, connec-tions with Internet peers carry the IP address of the RBridge placed at system’s core which acts as the system Internet gateway. By using the centralized Internet access in the overlay network architecture, RBrigdes’ movement do not affect Internet connections. This func-tionality is very important because it enables quality perception, avoiding problems as VoIP calls going down, messenger services going offline, on-line radio streams hanging, etc.
Figure 4.2: Core RBridge connections to Internet, WiMax base station and the servers of network centralized services.
4.2
RBridge’s architecture
4.2.1 Core RBridge
Core RBridge shown in figure 4.2, is a fixed centralized equipment, situated at system core. This RBridge is permanently connected to the Internet through FEUP NET1. Core RBridge is the system’s Internet Gateway, having a static public IP address well known by all the other RBridges.
System’s WiMax base station is Gigabit Ethernet connected to core RBridge. All WiMax connections between RBridges on buses share this base station and have direct access to the core RBridge.
Having permanent Ethernet connection to WiMax private network and to Internet, core RBridge will play an important role on the centralized component of system’s routing solution. This RBridge has also the role to make centralized network services, mentioned on section4.1, accessible to the network. These services are the following.
DHCP Server provides auto-configuration information for each system terminal; infor-mation includes a private IP address, network mask, DNS servers and default gate-way. DHCP service is centralized assuring the nonexistence of replicated IP ad-dresses. Network management is easier when services are implemented on a single place.
A single DHCP server exists to prevent IP address conflicts in the network. An IP range could be reserved for each RBridge, with IP address association made locally and saving DHCP traffic on the overlay network; however, this approach would limit the possibility of terminal mobility between RBridges because IP address con-flicts could happen when a new terminal connects to the previous RBridge and get a conflicting IP address.
Proposed Architecture
Figure 4.3: Bus RBridge
DNS Server implements DNS cache, decreasing the name resolution time overhead for each name resolution requested by terminals. It can also be used as an authoritive server to implement name resolution to system’s domain.
AAA Server implements services of authentication, authorization and accounting for all entities inside system’s overlay network.
SitMe project pilot counts with a single core RBridge, but a generic solution could in-clude several DC RBridges, hierarchically or mesh connected in order to avoid network bottlenecks and single points of failure.
4.2.2 Fixed RBridge
Some bus stops may be equipped with a fixed RBridge connected to the core RBridge through metro-Ethernet links. Each RBridge installed on bus stops has an WLAN access point to connect passengers and buses. Passengers are provided with Internet access while waiting for the bus; nearby buses can access system’s network .
4.2.3 Bus RBridge
Bus RBridge, shown on figure4.3, is the mobile network access equipment installed in each bus. Bus RBridges share its multiple technology network access with terminals inside the bus Each bus RBridge provides 2 LAN segments to terminals inside the bus; one connects the Xarepoint and the other connects an WLAN access point to terminals carried by passengers inside the BUS.
Each bus RBridge is equipped with heterogeneous WAN access technology interfaces including UMTS, WiMax and WLAN provided internally by the network or by pub-lic telecom operators. The base station WiMax included in the system provides broad-band access to buses on a short distance range through the WiMax subscriber station existent on the bus RBridge; WiMax public networks provided by ISP may be used if available. WLAN ad-hoc connections are expected to exist between buses on a near dis-tance. WLAN infrastructure connections are provided, on some places, by public WLAN Hotspots like Porto Digital or by WLAN access points installed on bus stops. One or more UMTS connections can be used to provide broadband access in areas not covered by other technologies.
4.3
Data Plane
System’s data plane defines a layer 2 overlay network built over the dynamic and multi-technology network links existent between RBridges. The overlay network is achieved by using a MPLS header carrying the destination RBridge identifier. Original data frames are transmitted between terminals and RBridges using the communication stack presented in figure4.4(a)which represents the standard IP communication stack. RBridges encapsu-late data received from terminals with a MPLS header carrying the destination RBridge identifier as specified in section3.1.4.1. The destination and origin MAC addresses of the original frame remain unchanged. Terminals using the overlay network are unaware of the RBridges existence.
MPLS encapsulated data is then transported inside the overlay network which is build over the multiple technology links existent between RBridges. The network protocol stack used to transport data is different if either a link is private or public as described in section
4.1. On private links MPLS encapsulated data is sent directly over the link as described in
3.1.4.1; communications stack obtained is presented in figure4.4(b). When using public links data encapsulated with the MPLS header can not be transmitted directly through the link due to restrictions imposed by ISP as explained in section3.2; data is transmitted using UDP sockets using the communication stack presented in figure4.4(c).
Terminals layer 3 auto-configuration requirement is achieved using DHCP protocol. DHCP protocol is implemented by two elements; an DHCP Client present, by default, on the terminals that requests for layer 3 configuration and an DHCP Server that replies them with the configuration parameters they have to assume. When a terminal connects to bus WLAN , authenticates itself if required and acquires an IP address through DHCP (the DHCP broadcast request is tunneled to a well known DHCP server situated at the network core).
Before start a communication session, terminals use ARP to find out the MAC ad-dress of its destination; an ARP request is broadcasted requesting the MAC adad-dress of the
Proposed Architecture DATA (includes IP header) L2 (a) DATA (includes IP header) Original L2 MPLS header Private L2 (b) DATA (includes IP header) Original L2 MPLS header UDP Public IP Public L2 (c)
Figure 4.4: Communication stack used on (a) terminal communications, (b) private links, and (c) public links.
destination IP which is intercepted by the RBridge and an appropriate ARP reply is gen-erated based on the information gathered by the routing protocol as described in section
4.4. Once having the destination MAC address, communication is now possible through Ethernet frames using that destination MAC.
If a destination station is known only by name, a DNS question is generated to resolve the IP address associated with that name. The question is placed to the DNS server as-sumed by DHCP auto-configuration, the one located on SitMe network core. This server acts as a DNS cache server that recursively resolve the name and replies the station with the corresponding IP address. This server is used to reduce the name resolution delay overhead and save Internet traffic, as cache hits are expected to occur (i.e. terminals ac-cessing same websites).
Internet access is provided centrally on the network core using NAT. NAT with IP masquerading is the mechanism used so all private IP addresses of SitMe network can be mapped in the same public IP address. This mechanism maintains TCP and UDP sessions information so the return Internet traffic can reach back the entity that requested it. The Internet gateway that implements NAT runs on core RBridge.
System’s data communications may be internal or to Internet access; on internal com-munications sessions are established between terminals within the system; Internet access involves communications between a system terminals and external peers. The overview diagram of the SitMe network data plane, presented on figure4.5 contains some exam-ples of internal and Internet access communications, using different technology interfaces available on bus RBridges. The arrows represent the data paths used by these communi-cations. In figure4.5are also presented examples of communications using the different network technologies available. When terminals communicate among themselves in the same AP, on a bus RBridge, that network traffic is local and do not need to be encapsulated and routed by the RBridge. Local or Internet access that implicates destination stations not present in the same RBridge, implies local RBridge Ingress operations (MPLS tag-ging and encapsulation), intermediate RBridges packet switching (based on MPLS tag)
Figure 4.5: SitMe Data Plane. Internal and Internet access communications, using different tech-nology interfaces available on BUS RBridges.
and destination RBridge Egress operations (MPLS tag removal and decapsulation). On Internet communications the egress RBridge is the Internet gateway running on the Core RBridge.
4.4
Control Plane
The system’s control plane is based on the WMRP protocol described in section3.1.4.2. WMRP is a link state routing protocol, so connectivity information of each node is flooded to all other nodes. This information is gathered, maintained and distributed by the con-trol plane component of each RBridge routing module. Upon receiving this information each node computes pair-wise optimal paths for each other node on the network. This information is then used by data plane to forward packets.
WMRP routing messages were detailed on section 3.1.4.2. It is evident that there are two main behaviors assumed by an RBridge when handling control data; (1) layer 2 Broadcast for HELLOs and (2) layer 2 Flooding for TC, MC and IC messages. Broadcasts are not accessible nor make sense being used to discover RBridge neighbors over public
Proposed Architecture
links. All RBridges using public links are neighbors on the overlay network point of view, but they still need some way to know each other. Simply flood control data to all neigbours using public links could lead to control data repetition and to a lot of upstream bandwidth and traffic consumption. To overcome this problem WMRP was adapted to operate over public links creating the Centralized L3 HELLO System, and the Centralized L3 Control Flooding System. These adaptations are introduced in subsection4.4.2and4.4.3.
Before starting with the definition of the communication system architecture’s con-trol plane, it is necessary to discover and understand every possible RBridges’ opera-tion scenarios and their implicaopera-tions on its operaopera-tion behavior. To each different behav-ior/situation identified, was named an operation mode.
4.4.1 RBridge Operation Modes
Diagrams of figure 4.6, present all the possible network scenarios that a bus RBridge will encounter during its operation. On each operation scenario RBridges changes its operation behavior that is here descried as operation mode.
L2 Mode Connected to other RBridges only through private links, mesh topology. core RBridge is accessible through layer 2;
Island L2 Mode Connected to other RBridges only through private links, mesh topology, but core RBridge is not accessible;
L3 Mode Connected to other RBridges through Internet only (public link or layer 3 Link);
Hybrid Mode Connected to other RBridges through private links and Internet;
Island Hybrid Mode Like Hybrid Mode, but core RBridge is not accessible through pri-vate links;
Disconnected Mode Without connectivity to other RBridges.
Transitions between modes are determined by changes on RBridge connectivity as described next and represented on the state machine diagram presented on figure4.7.
Disconnected -> Island L2 Gained layer 2 connectivity to neighbor’s RBridges, but none of them is the core RBridge.
Disconnected -> L3 Gained layer 3 connectivity to core RBridge. Island L2 -> Disconnected Lost layer 2 connectivity.
Island L2 -> L2 Now, core RBridge is one of the layer 2 Neighbors. Island L2 -> Island Hybrid Gained layer 3 connectivity to core RBridge.
(a) (b)
Figure 4.6: RBridges Operation Modes: L2, L3, Hybrid and:(a) Island L2, or (b) Hybrid Island. Private links are represented in blue and public in brown.
L2 -> Disconnected Lost layer 2 connectivity.
L2 -> Island L2 Lost layer 2 connectivity to core RBridge. L2 -> Hybrid Gained layer 3 connectivity to core RBridge. L3 -> Disconnected Lost layer 3 connectivity to core RBridge.
L3 -> Hybrid Now, core RBridge is one of the layer 2 Neighbors and the control data from core RBridge was transported only through layer 2 Links.
L3 -> Island Hybrid Now, core RBridge is one of the layer 2 Neighbors and the control data from core RBridge was transported through layer 3 Links.
Hybrid -> L2 Lost layer 3 connectivity to core RBridge. Hybrid -> L3 Lost layer 2 connectivity.
Hybrid -> Island Hybrid Now, the control data from core RBridge was transported through layer 3 Links.
Island Hybrid -> Island L2 Lost layer 3 connectivity to core RBridge.
Island Hybrid -> Hybrid Now, the control data from core RBridge was transported only through layer 2 Links.
Proposed Architecture
Figure 4.7: State Machine Diagram for all bus RBridges’ Operation Modes
4.4.2 Centralized L3 HELLO System
In WMRP protocol RBridges discover neighbors spreading HELLO messages through layer 2 broadcasts. RBridges using public links have several problems discovering neigh-bors through the same method: layer 2 broadcasts cannot be used on public links since this layer is not accessible; RBridges using public links connect directly by using public IP addresses in the Internet, therefore, they are all layer 3 neighbors; due to bus mobil-ity, public links and IP address are dynamically changing; public links can have limited upstream bandwidth and traffic costs, replicated traffic should be avoided. Using a cen-tralized system to announce themselves is the solution to these problems.
Centralized L3 HELLO System on core RBridge gathers, maintains and distributes public IP addresses and UDP ports assigned to RBridges using public links. Centralized L3 HELLO System takes advantage of the high bandwidth available on the core to per-form all the needed unicasts. The Centralized L3 HELLO System operation is described in the following steps.
1. Each RBridges registers their public interfaces (IP Address and UDP Port to be reached, for each interface) and periodically refreshes this information; an Inter f ace Registrationmessage is unicasted to the core RBridge;
2. After registration, if it is a new RBridge, all information maintained by the central-ized system is pushed to it; an Inter f aces Full Synchronization message is unicas-ted to the new RBridge;
