Inter point-of-presence network service orchestration

(1)

Universidade de Aveiro Departamento deElectrónica, Telecomunica¸cões e Informática, 2019

Jorge Vicente

de Oliveira

Orquestra¸

c˜

ao de Servi¸

cos de Rede com m´

ultiplos

Pontos de Presen¸

ca

Inter Point-of-Presence Network Service

Orchestration

(2)

(3)

Universidade de Aveiro Departamento deElectrónica, Telecomunica¸cões e Informática, 2019

Jorge Vicente

de Oliveira

Orquestra¸

c˜

ao de Servi¸

cos de Rede com m´

ultiplos

Pontos de Presen¸

ca

Inter Point-of-Presence Network Service

Orchestration

Disserta¸cão apresentada à Universidade de Aveiro para cumprimento dos requisitos necessários à obten¸cão do grau de Mestre em Engenharia Eletró nica e de Telecomunica¸cões, realizada sob a orienta¸cão cient´ıfica do Doutor Diogo Nuno Pereira Gomes, Professor do Departamento de Eletrónica, Tele-comunica¸cões e Informática da Universidade de Aveiro e do Doutor Rui Lu´ıs Andrade Aguiar, Professor do Departamento de Eletrónica, Telecomu-nica¸cões e Informática da Universidade de Aveiro

(4)

(5)

o j´uri / the jury

presidente / president Professor Doutor Arnaldo Silva Rodrigues de Oliveira

Professor Auxiliar da Universidade de Aveiro

vogais / examiners committee Professor Doutor Paulo Alexandre Pereira Sim˜oes

Professor Auxiliar do Departmento de Engenharia Inform´atica da Faculdade de Ciˆencias e Tecnologia da Universidade de Coimbra

Professor Doutor Diogo Nuno Pereira Gomes

(6)

(7)

agradecimentos /

acknowledgements Para muitos, a conclusão do curso é um sinónimo de tristeza, do término da ”boa vida” e da entrada no mundo dos adultos onde o acto mensal de pagamento da renda deixa de ser uma trivialidade e passa a ser uma preocupa¸cão constante. Para mim, o fim desta jornada é equiparável ao... término de uma prova desportiva bastante invulgar, fruto da mistura de uma prova de endurance realizada numa montanha de estradas extremamente sinuosas com um desporto de equipa com regras muito manhosas onde por vezes até as próprias equipas mudam. Porém, para minha sorte, na reta final desta prova acabei por me colar a um equipa de indiv´ıuos que, apesar de ao pric´ıpio parecerem estran-hos e disfuncionais, acabaram por se tornal numa equipa de elite. Hoje, no dia de cruzar a meta, aproveito para agrede¸cer não só a esta equipa mas também à minha claque pessoal que me apoiou incessantemente ao longo de toda a prova. Come¸carei primeiro por agradecer à equipa. Em primeiro lugar agrade¸co ao Ricardo Enes, ao Pedro Rodrigues, ao João Resende e ao Miguel Carretos, Miguel Nunes de nascen¸ca, os quais me ensinaram que tudo está ao nosso alcance caso tenhamos paixão, esfor¸co e dedica¸cão suficientes. Agrade¸co-lhes pois todos eles trabalharam incessantemente a meu lado para nos pudessemos superar a nós próprios por forma a conseguirmos realizar o nosso desejo co-mum de, no fim, não acabarmos por ser apenas mais algumas gotas no oceano. Parabéns equipa. Conseguimos. Em segundo lugar, agrade¸co de novo ao Ricardo Enes, embora noutro contexto, e ao Diogo Craveiro, amigos do peito, colegas de casa e bffs que se mantenveram a meu lado na saúde e na doen¸ca. O meu muito obrigado por se terem tornado a minha fam´ılia de Aveiro. Agradecerei agora à claque. Em primeiro lugar agrade¸co à minha namorada, Tatiana Antunes, a minha PIC que me apoiou de todas as maneira poss´ıveis e imaginárias quando não conseguia ver a luz ao fundo do túnel. Que esta seja apenas mais uma de muitas histórias na qual serás também uma importante pro-tagonista. Agrade¸co também ao meu pai, Fernando Oliveira, e irmão, José Oliveira, que, na sua boa disposi¸cão, sempre acreditaram que teria um desfecho positivo, mesmo quando nem eu próprio acreditava. Por ´

ultimo, agrade¸co à minha mãe, Edite Vicente, a pessoa que provavel-mente mais sofreu ao ver-me realizar esta prova. A ti, mãe, tenho apenas a dizer isto: se algum dia tiveres em mim metade do orgulho que eu tenho por ti morrerei um filho feliz. Por último e como não poderia deixar de ser, agrade¸co ao meu orientador, Diogo Gomes, e

(8)

(9)

Palavras Chave Redes 5G, Virtualiza¸cao, Fun¸cões de Rede Virtualizadas, Encadea-mento de Fun¸cões de Rede, Orquestra¸cão, Redes Softwarizadas, Open-source MANO

Resumo Ao longo dos últimos anos, as operadoras de Servi¸cos de Telecomu-nica¸cões mudaram os seus modelos de negocio baseados em Servi¸cos de Rede f´ısicos alojados em hardware especializado para ambientes vir-tualizados, os quais oferecem vantagens em termos de rapidez de pro-visionamento de recursos, maior flexibilidade e redu¸cões de custos de capital e operacionais. Este novo paradigma levou a que se come¸assem também a virtualizar as funcionalidade dos Servi¸cos de Rede bem como os mecanismos de gestão e orquestra¸cão dos recursos computacionais e de rede. Esta mudan¸ca levou à necessidade da cria¸cão de plataformas capazes de gerir os ciclos de vida destes novos Servi¸cos de Rede vir-tualizados. Opensource MANO (OSM) é uma destas plataformas e é compat´ıvel com os standards produzidos pelo Electronics and Telecom-munications Standards Institute (ETSI), sendo capaz de fazer a ante-riormente referida gestão do ciclo de vida dos Servi¸cos de Rede virtu-alizados. Contudo, esta plataforma ainda não possui maturidade sufi-ciente para ser capaz de solucionar alguns cenários das futuras redes 5G, nomeadamente aqueles que requerem a orquestra¸cão de Servi¸cos de Rede com múltiplos pontos de presen¸ca. Este trabalho come¸ca por analisar o Estado da Arte em rela¸cão a solu¸cões de virtualiza¸cão de Servi¸cos de Rede e plataformas de gestão já existentes, bem como as `

areas tecnológicas abrangidas por este tema, seguindo-se uma contex-tualiza¸cão do problema que este trabalho pretende solucionar. Após a contextualiza¸co do problema são apresentadas duas solu¸cões, as suas correspondentes arquiteturas e os testes realizados para avaliar a vial-bilidade das mesmas. Finalmente são analisados os resultados produzi-dos pelos testes retirado-se as respetivas conclusões e apresentadas propostas para possiveis trabalhos futuros.

(10)

(11)

Keywords 5G Networking, Virtualization, Virtualized Network Functions, Ser-vice Function Chaining, Orhcestration, Software Defined Networking, Opensource MANO

Abstract Over the past few years, the Telecommunication Service Providers’s (TSPs) Network Service (NS)-based business model have drastically changed from traditional proprietary physical network appliances to-wards virtualized ecosystems which offer faster resource provisioning and improved flexibility while reducing both Capital expenditure and Operational expenditure. This new paradigm pushed both the physi-cal network appliances’ functionalities and physiphysi-cal and networking re-sources’ management and orchestration into the software plane, leading to the creation of several Management and Orchestratrion (MANO) frameworks tasked with managing the lifecycle of NSs. Opensource MANO (OSM) is one of said frameworks and it is an Eletronics and Telecomunnications Strandards Institute (ETSI) Network Function Vir-tualization (NFV) compliant MANO platform capable of managing the lifecycle of both NSs and their composing Virtual Network Functions (VNFs). While this framework already has an acceptable degree of maturity, it still cannot fulfill some of the 5G future networks’ use-cases, namely those regarding inter-Point-of-Presence (PoP) NS or-chestration. This work starts by analysing the current state of the art regarding NFV and NFV MANO solutions and research initiatives and technological areas encompassed by them, followed by a contex-tualization of the problem itself. Then, two solutions aiming to solve the problem of inter-PoP NS orchestration through OSM are presented followed by an overview a of their practical implementations. Finally, the results yielded by the tests performed are analysed, conclusions are drawn and possible future works are suggested.

(12)

(13)

List of Figures

2.1 3GPPP roadmap . . . 7

2.2 NFV architecture specifications map . . . 9

2.3 Example of L2 network extension across L3 connectivity [1] . . . 11

2.4 Comparison of SDN and traditional networking [2] . . . 12

2.5 SDN architecture and its fundamental abstractions [3] . . . 13

2.6 Packet flow through the processing pipeline. [4] . . . 15

2.7 Main components of an Openflow (OF) switch [4] . . . 16

2.8 OF packet processing pipeline . . . 17

2.9 SFC architecture without SDN (left) and with SDN (right) [5] . . . 19

2.10 Comparison of the main Management and Orchestration (MANO) platforms [6] . . . 22

3.2 Upstream Multihoming with OpenFlow [7] . . . 29

3.3 Network isolation use-case [7] . . . 30

3.4 MEC deployment with EPC and MEC application on the same NFVI plat-form [8] . . . 31

3.5 Proposed Distributed Virtualized MECs inspired by [8] . . . 31

3.6 5G URLLC use-cases . . . 33

3.7 Network Slicing use-case . . . 34

3.8 Architectural design of the 5TONIC MANO platform. [9] . . . 36

3.9 Network service overview [10] . . . 38

3.10 Openstack’s provider networks overview . . . 39

3.11 Openstack’s self-serving networks overview . . . 40

4.1 OSM configuration operations workflow through JuJu charms . . . 44

4.2 Conceptual architecture . . . 45

4.3 Use-Case 1: Tunneling as a standalone Network Service (NS) . . . 45

4.4 Use-Case 2: Integration of the solution as part of an already existing NS . 45 4.5 VNF configuration flow . . . 47

5.1 Open Source Mano (OSM)’s Resource Orchestrator (RO) current architecture 50 5.2 Target architecture . . . 54

(18)

5.4 ODL connector architecture 2 . . . 56

5.5 WAN link actions . . . 61

5.6 WAN link refresh . . . 62

5.7 QoS Diagram . . . 63

6.1 Experimental setup . . . 66

6.2 Charm setup time . . . 67

6.3 Experimental setup . . . 68

6.4 Graphical representation of the WAN link average throughput for different OF meters . . . 70

7.1 Floating IPs implementation . . . 74

(19)

List of Tables

2.1 Fields of research and associated technologies . . . 6

2.2 SDN Controller Comparison . . . 14

6.1 VNF primitives and their performance . . . 67

(20)

Acronyms

3GPPP Third Generation Partnership Project. 6, 7 5GPPP 5G Infrastructure Pulbic Private Partnership. 7 CAPEX Capital Expenditure. 1

DSCP DiffServ Code Points. 28 e2e end-to-end. 1, 19

EPC Evolver Packet Core. 27–31, 41

ETSI European Telecommunications Standards Institutes. 1, 2, 7, 8, 10, 19, 23–25, 37, 41, 73

ETSI ISG European Telecommunications Standards Institute Industry Specification Group. 1

IDS Intrusion Detection Service. 29 IPS Intrusion Protection Service. 29 ISP Internet Service Provider. 29 L2 Layer 2. 39, 75

L3 Layer 3. 39

LCM Lifecycle Management. 75 LTE Long Term Evolution. 27, 30, 41

MANO Management and Orchestration. v, 1–3, 8, 10, 19, 22–25, 30, 33–35, 37, 41, 73 MEC Mobile Edge Computing. 30–33, 41

(21)

NBI Northbound Interface. 43

NFV Network Functions Virtualization. 1–3, 5, 7–9, 11, 14, 19, 21–25, 30–35, 37, 41, 43, 49, 64, 73

NFVO Network Functions Virtualization Orchestrator. 9, 10, 22, 23, 25, 37, 38, 49, 50 NS Network Service. v, 1–3, 9, 10, 19, 21–25, 27, 28, 31, 35–37, 41, 43–45, 47–50, 53–55,

64, 69, 70, 73–75

NSD Network Service Descriptor. 38, 70 NSH Network Service Header. 19, 20

NSO Network Service Orchestration. 19, 27, 37 ODL Opendaylight. 57, 58, 68

ODL connector OpenDaylight Connector. v, vi, 56

OF Openflow. v, vi, 15–18, 28, 29, 54, 55, 57, 58, 60, 69–71, 75 ONAP Open Network Automation Platform. 1, 22, 25

ONF Open Networking Foundation. 15 OPEX Operational Expenditure. 1

OSM Open Source Mano. v, 1, 2, 22, 23, 25, 35, 41, 43, 44, 47–50, 52, 54, 64, 65, 68, 69, 71, 73, 75, 76

OVS OpenVswitch. 46

PoP Point of Presence. 2, 3, 33–36, 41, 44, 47–49, 53–55, 64, 65, 68–71, 73–75 QoS Quality of Service. 1, 16, 20, 27, 28, 36, 57, 64, 71, 74

RFC Request for Comments. 19

RO Resource Orchestrator. v, 44, 49, 50, 64, 75

SDN Software Defined Networking. 1–3, 8, 12–15, 19, 21, 22, 24, 27, 28, 30, 35–37, 41, 43, 53, 54, 57, 64, 68, 71, 75

SDN-C Software Defined Networking Controller. 12, 14–16, 19, 29, 37, 49–51, 55, 57, 60, 68, 69

(22)

SFC Service Function Chaining. 3, 19, 20, 24, 35, 41, 55 SFF Service Funtion Forwarder. 19, 20

SFP Service Function Path. 19, 20, 54, 55, 60

TSP Telecommunication Service Providers. 1, 2, 19, 22–24, 29–34, 48, 74 URLLC Ultra Reliable Low Latency Communications. 32

vB vBridge. 47

VCA VNF Configuration and Abstraction. 44, 75

VIM Virtualized Infrastructure Manager. 9, 10, 22, 23, 35, 37, 39, 49–51, 53, 68, 69, 71, 74, 75

VLD Virtual Link Descriptor. 38 VM Virtual Machine. 44, 49, 65

VNF Virtual Network Function. 1, 8–10, 14, 19, 22, 25, 30, 31, 37, 38, 43–48, 65, 69–71, 73, 75

VNFD Virtual Network Function Descriptor. 37, 38, 46, 70 VNFFG Virtual Network Function Forwarding Graph. 20, 38

VNFM Virtualized Network Functions Manager. 9, 22, 23, 37, 46–48 VPN Virtual Private Network. 36, 73

WAN Wide Area Network. ii, iii, 3, 35, 41, 43, 44, 48–55, 57, 60, 61, 63–65, 69–71, 73–75 WIM Wide-Area Infrastructure Management. 43, 49–55, 60, 64

(23)

Chapter 1 Introduction

Over the past few years, the exponential growth in connected devices, application diversity, Quality of Service (QoS) demand and the traffic generated by its ever increasingly number of users has forced Telecommunication Service Providers (TSP)s to develop new solutions in order to be able to provide better network-based services while achieving lower deployment and operation costs. Moreover, with the upcoming generation of mobile telecommunications standards, 5G, TSP have acknowledged that the previous generations’ technical stack is unable to fulfill the expected technical requirements for the new 5G futures networks use-cases and began to search for solutions that would enable them to accomplish these new use-cases as well as finding new opportunities and business models. However, recent advancements in other technical areas such as Virtualization, Cloud computing and Software Defined Networking (SDN) have created suitable conditions for a change in the service provisioning paradigm towards Network Functions Virtualization (NFV), a new network architecture based on modern virtualization technologies. NFV has the potential to reduce both Capital Expenditure (CAPEX) and Operational Expenditure (OPEX) while eliminating problems associated with proprietary-based network appliances due to the fact that European Telecommunications Standards Institute Industry Specifica-tion Group (ETSI ISG) is currently developing a standardized architecture for NFV known as European Telecommunications Standards Institutes (ETSI)’s NFV MANO [11]

NFV advocates the conversion of the typical physical network functions (such as Fire-walls, Wide Area Accelerators, Network Address Translators, etc) into Virtual Network Function (VNF)s which, when chained together, form NSs, an end-to-end (e2e) service sold to clients that performs a set of operations on the traffic that enters the said service. TSPs then manage the lifecycle of both VNFs and NSs through a MANO platform, an entity capable of managing both physical and network resources on the software plane, thus improving resource provisioning speed, flexibility and decreasing the complexity of lifecycle management operations.

While there are already several NFV MANO solutions such as OSM, Open Network Automation Platform (ONAP) or OpenBaton, most of them are not yet mature enough to be able to meet the technical requirements of the most complex 5G future networks’ use-cases. From the network engineer point of view, many those unsolved challenges are

(24)

associated with the degree of flexibility of how network links comprised by both virtual and physical networks can be transparently setup, managed and thorn down, a functionality that, if solved, would allow MANO platforms to meet the requirements of scenarios where subsets of the service’s functionalities need to be split across multiple Point of Presence (PoP)s.

1.1 Motivation

With the change in TSP’s production environments towards NFV architectures based on virtualization technologies, several entities all over the world began to research and standardize NFV in the context of the telecommunication industry. In Europe, ETSI is coordinating the NFV standardization initiatives and several documents including stan-dards, reference architectures and technical documents have already been produced. This work firstly analyzes the state of the art and current state and maturity of the existent NFV MANO frameworks comparing them and assessing the viability of multi-PoP NS which require NS’ functionalities to be split across different physical locations (i.e one part of the service in an edge cloud and another on the datacenter).

After this analysis, was pointed out that OSM, an ETSI reference architecture com-pliant orchestration framework still does not have the proper set of functionalities which would allow it to orchestratate the aforementioned NSs spawning multiple PoP. Since this inability to orchestrate these NS severely reduces the number of 5G’s use-cases that OSM can possible solve, finding solutions for this problem becomes a necessity and a suitable endeavor for this work’s objective.

1.2 Objectives

With the rise of SDN, many 5G networks’s use-cases assume the utilization of SDN hardware which potentially increases provision speed, flexibility and ease the management of network resources. While new deployments may start to be based on SDN hardware, much of the already in place infrastructure runs on IP network equipment which does not offer the same set of functionalities as their SDN counterparts. Also, NFV architectures require that the management of physical and network resources as well as the lifecycle management of NSs to be done through a MANO frameworks.

Thus, the objective of this work is to develop mechanisms which will allow the orches-tration of NS spawning multiple PoP for both SDN and non-SDN hardware, in the most automatized way possible and through OSM, an ETSI reference architecture compliant orchestration framework.

(25)

1.3 Dissertation Outline

The current chapter, Chapter 1, serves as an introductory chapter and defines this work’s scope. This introduction is followed by an overview of the work’s motivation and objectives. The rest of the work is organized as follows:

• Chapter 2 starts by analysing the proposed new technological stack upon which the new generation of mobile telecommunication, 5G, will be built on top of. After this analysis, it is studied how some state-of-the-art technologies such as NFV, SDN and Service Function Chaining (SFC) have enabled new architectures where it is possible to deploy virtual network functions on top of virtualized infrastructure, managing their lifecycle through MANO platforms. The chapter wraps up by evaluating the maturity of the said MANO platforms, comparing them according to different metrics and analysing their possible research avenues.

• In Chapter 3 are studied the application domains of the technologies presented in Chapter 2. This study yielded insights which indicate that NFV is able to enhance several use-cases such as the virtualization of legacy telecommunication cores as well as creating new business models. However, still during the same study, it is also found out that some of these use-cases’ technical requirements have not yet been fulfilled. This chapter wraps up by stating that finding solutions that will enable the aforementioned use-cases would be well within this work’s scope and thus, the development of such solutions became this work’s objective.

• In Chapter 4 it is presented a solution that enables the creation of inter-PoP NSs over IP WAN infrastructure. This chapter also analyses the solution’s architecture, its technical implementation and its limitations.

• In chapter 5 it is presented a solution that enables the creation of -PoP NSs over SDN WAN infrastructure. This chapter also analyses the solution’s architecture, its technical implementation and its limitations.

• Chapter 6 starts by presenting the setups used for the practical implementation which validate the solutions described in chapters 4 and 5 and the test performed used to evaluate the developed solutions’ viability. This chapter wraps up by analysing the results yielded by these tests, evaluating them and then drawing conclusions.

• Chapter 7 revisits this work’s key contributions and evaluates how they have improved the current state-of-the-art. Lastly, this work wraps up by identifying and explaining a subset of problems left out of its scope that are seen as viable future research avenues.

(26)

1.4 Contributions

The main contributions of this work can be summarized as follows:

• A paper published as a first author in the conference ConfTele2019’s proceedings with the title ”Inter-site Network Service Orchestration over SDN-enabled Transport Networks”.

• Collaboration in 5GinFire’s Deliverable D.4.3 with the ”Final Report on the MANO Platform” which is yet to be published.

(27)

Chapter 2 State of the Art

In this chapter will be evaluated the roles of Europe’s different standardization bod-ies in the standardization of the upcoming generation of mobile telecommunications. As standardization activities are divided in several different technological areas, it is given focus to the technological area in the scope of this work, the Core Architecture and & Core Networking followed by an analysis of its state-of-the-art.

2.1 5G - From Use-Cases to Standardization

5G be will the next generation of communication systems but, unlike previous genera-tions, 5G not only promises to bring the typical technological improvements such as higher bandwidth, lower latency, etc, but also aims to expand the current business perspectives by including new services and business models, specially those concerned with Verticals (e-Health, Smart Cities, Energy, Automotive, etc).

While 5G use-cases vary from Vertical to Vertical, 5G networks promises to offer un-limited mobile broadband experience, providing massive connectivity for everything from human-held smart devices to sensors and machines, and most importantly, it has the ability to support critical machine communications with instant action and ultra-high reliability [12] while also supporting back-compatibility with already existing Gs. In order to achieve all this, the whole telecommunications technological stack needs to be redefined and sev-eral new key technologies must be developed to accommodate these challenging constraints [13].

These new technologies can be broadly divided into the following categories: • Wireless software-defined networking

• NFV

• Millimeter wave • Massive MIMO

(28)

• Ultra-densification

• Big data and mobile cloud computing • Radio access techniques

In Europe several entities are leading the development, experimentation and standard-ization of emerging 5G technologies which can be divided into fields of research. Figure 6.4 provides an overview over these research fields and the entities associated with them.

Category Organization 3GPP 1

NGMN 2

Network Architecture & ETSI3 ISG NFV Core Networking ESTI3 _{ISG MEC}

OPNFV 4

IETF 5

ITU6_{-T SG13 IMT2020 FG}

Small Cell Forum

Network TMF 7

Management NGNM 2

3GPP Industry Alignment ITU6-R

& Adoption GSMA8

NGNM2

Access Technology 3GPP 1

IEEE 802.11 ETSI3 ISG mWT

ETSI3 _RRS

Table 2.1: Fields of research and associated technologies

The scope of this document is well defined within the Networks Architecture & Core Network category and, as such, this work will be focusing on the organizations working within its scope and their developed works.

The Third Generation Partnership Project (3GPPP) consortium is an organization mainly focused on aspects related to mobile telecommunication standards and was the en-tity responsible for the standardization process of technologies of previous generations such

1_{https://www.3gpp.org/} 2_{https://www.ngmn.org/} 3_{https://www.etsi.org/} 4_{https://www.opnfv.org/} 5_{https://ietf.org/} 6_{https://www.itu.int/} 7_{https://www.tmforum.org/} 8_{https://www.gsma.com/}

(29)

as LTE, LTE-A, etc. 3GPPP uses a system of parallel Releases which provide developers with a stable platform for the implementation of features at a given point and then allow for the addition of new functionality in subsequent Releases.

Regarding 5G standardization, 3GPPP and has already released a technical specifica-tion (Release 15) entitled ”Procedures for the 5G System” [14] in which it defines and describes the procedures and Network Function Services for the future 5G Systems and is planning on releasing Release 16 until 2020. 3GPPP’s main concerns are on Radio Access technology and Core Network technology (Network Slicing and Fixed-Mobile Integration) for submission to IMT2020.

While 3GPPP is mainly concerned with mobile and cellular telecommunications, ETSI is concerned with other aspects of the standardization of telecommunication networks operators activities namely Network Function Virtualization NFV, Muti-acess Edge Com-puting (MEC), Millimetre Wave Trasmission (mWT), Next Generation Protocols (NGP), each with a corresponding working group.

The 5G Infrastructure Pulbic Private Partnership (5GPPP) consortium, like 3GPPP, is an European joint initiative between the European Commission and European ICT industry (ICT manufacturers, telecommunications operators, service providers, SMEs and researcher Institutions) and has created several projects to further develop 5G enabling technologies such as METIS, 5GNOW or Mobile Cloud Networking. 5GPPP has already released a white paper [12], providing enhancements over several aspects of 3GPPP Release 15, addressing several specific requirements from vertical industries.

(30)

2.2 NFV & ETSI’s NFV

Service provisioning within the telecommunication industry has traditionally been based on network operators deploying physical proprietary devices and equipment for each func-tion that is part of a given service [16]. NFV is a promising new paradigm which aims to change the traditional physical network appliances driven business models by decoupling network functions from the dedicated hardware and moving them to the software plane. Such decoupling introduces many benefits which include reduction of Capital Expenditure and Operation Expense, improved flexibility of service provision, etc. [17].

NFV has been an hot topic for quite some time withing the research community and several studies have been carried out to evaluate how NFV could potentially enhance other already existing technologies and system architectures.

In [18] it is analysed how emerging microservices architectures may benefit from several aspects of NFV such as allowing independent development of small pieces of the ecosys-tem, improved utilization of underlying infrastructure resources, and increased infrastruc-ture agility for supporting the next generation of applications. In [19] it is presented a MANO framework capable of orchestrating services on both virtualized physical hardware or containerized environments agnostically. This framework offers a non-standarderized architecture based on SONATA framework who offers a set of specialized tools to deploy and manage VNF’s lifecycle and an service construction kit which emulates telecommuni-cations infrastructure on a local machine, enabling the developer to execute a portion of the tests locally without needing to interface with actual operator equipment. In [20] et al.,(2019) the authors propose an architecture for on-the-fly provisioning of Content Dis-tribution Networks components using NFV and microservice architecture principles where the services’ components are designed in form of microservices and implemented, packaged and deployed using NFV technology. Lastly, in [21] the authors propose an architecture which aims to combine NFV and Machine Learning (ML) to improve the process of op-timizing and reconfiguring network resources by creating ML-assisted enhancements into an open-source network planner tool (Net2Plan) to facilitate network planning and SDN interfacing.

2.2.1 ETSI MANO

As previously mention in 2.2, ETSI is the standards group responsible for the stan-dardization of NFV in Europe and while it would be interesting to go more in depth in the works of each of ETSI’s working groups, this work will be focusing on ETSI’s NFV as the works of the other working groups fall out of the scope of this document.

The NFV Industry Specification Group (ISG NFV) was founded in November 2012 and it has produced over 100 publications describing ETSI’s reference NFV architecture as well as both internal and external reference points and all the architecture’s logical blocks. Figure 2.2 visually describes the the said reference architecture and provides some information on the documents containing the specification for features, interfaces, and building blocks.

(31)

Figure 2.2: NFV architecture specifications map 9

The NFV Management and Orchestration block encompasses several functions regard-ing VNF and can be subdivided into 3 sub-blocks:

• The Network Functions Virtualization Orchestrator (NFVO) • The Virtualized Network Functions Manager (VNFM) • The Virtualized Infrastructure Manager (VIM)

The NFVO is responsible for orchestration and management of NFV infrastructure, software resources, and the deployment of network services on the infrastructure. It has two major functionalities: Resource Orchestration, which coordinates, authorizes, releases and engages NFV Infrastructure resources from the different VIMs and Service Orchestra-tion that is used to manage and monitor the lifecycle of NSs and VNFs. It also includes databases that are used to store the information and data models which define both de-ployment and lifecycle properties of functions,services and resources [22]. The VNFM is

(32)

used to control, manage and monitor the VNFs lifecycle. These tasks include the instanti-ation, configurinstanti-ation, start and stop, scaling and termination of VNFs. Lastly, the VIM is tasked with managing the physical infrastructure namely compute, storage, and network resources.

Regarding ETSI’s MANO one of the most relevant documents is IFA 010 V3.2.1 [11] (as of this document’s publication date) in which are described the functional requirements for each of the building blocks of the architecture.

2.2.2 ETSI NFVO

As previously said, the NFVO encompasses all aspects of NS’s lifecycle management and infrastructure management. Moreover, as it can be seen from figure 2.2, it is the functional block with the largest number of reference points and therefore the most addresses by technical specifications. These reference points are thoroughly described from IFA 005 to IFA 015 and a brief summary of them is presented below: 10

• Management of virtualized resources (IFA 005, IFA 006, and IFA 010)

• Virtualized resources information management (IFA 005, IFA 006, and IFA 010); • Fault and performance management of virtualised resources (IFA 005, IFA 006, and

IFA 010);

• Lifecycle management of VNFs (IFA 007, IFA 008, IFA 010, SOL 002, and SOL 003); • Fault, configuration and performance management of VNFs (IFA 007, IFA 008, IFA

010, SOL 002, and SOL 003);

• Lifecycle management of Network Services (IFA 010, and IFA 013);

• Fault and performance management of Network Services (IFA 010, and IFA 013); • Package and software image management (IFA 005, IFA 006, IFA 007, IFA 010, IFA

011, IFA 013, and SOL 003);

• VNF Descriptor – VNF information modelling (IFA 011, and SOL 001);

• Network Service descriptors – NS information modelling (IFA 014, and SOL 001); • Virtualised resources capacity management (IFA 005, and IFA 010);

• Hardware-independent acceleration (IFA 002, IFA 003, IFA 004, and IFA 010); • A UML Information Model (IFA 015) that consolidates the Information Elements

developed in all other reference points specs (IFA004 to 008 and IFA011 to 14); • Information modelling guidelines (IFA 016, IFA 017)

(33)

2.3 Network Virtualization & Software Defined

Net-working

In section 2.2 it has been analused how NFV effectively solves issues related to manage-ment, flexibility and scalability of physical resources but fails to offer improvements on the same issues regarding network resources. A partial solution for this was found through the virtualization of the networking stack with the objective of being able to spawn logically separated networks each with potentially different addressing and forwarding mechanisms within the same physical infrastructure.

The first layer of access to the medium is the MAC (Medium Access Controller) and thus one of the first steps towards providing a fully virtualized network was virtualizing the Network Interface Cards (NIC), an improvement which enabled the the divison of one physical NIC into many virtual ones. The second step towards a fully virtualized networking layer is the the virtualization of the Layer 2 devices: the switches.

Out this endeavor of virtualizing L2 elements were born softwares like OpenvSwitch11_,

an hypervisor networking stack focused on the need for automated and dynamic network control in large-scale virtualization environments which enabled interesting use-case sce-narios like the one seen in Firgure 2.3. Through coupling virtual L2 virtual networks with L3 physical infrastructure serving as a transport network it is possible to achieve seamless L2 communications even though there may be a physical separation of both sides of the network.

Figure 2.3: Example of L2 network extension across L3 connectivity [1]

While virtualization provided some degree of control and scalability regarding network hardware devices, it did not solve one of the biggest disadvantages of traditional networking: the stacking of data and control planes onto a singles device, traditionally a router. This

(34)

approach severely limits knowledge about the overall network topology making it much more difficult to effectively enforce QoS policies as well as making debugging of large networks extremely complex which can ultimately lead to unexpected down times[23].

The SDN concept aims to correct some of the disadvantages of traditional networks mainly by separating the data and control planes and fully virtualizing them, allowing a software-driven customization of both control and forwarding planes through an entity called the Software Defined Networking Controller (SDN-C).

Figure 2.4: Comparison of SDN and traditional networking [2]

Kreutz2014 et al.,(2014) states that SDN is an network architecture is built upon 4 pillars:

• Data and control planes are decoupled and network devices become simple (packet) forwarding elements.

• Forwarding decisions become flow based instead of decision base which, in the SDN context, are defined as a sequence of packets between a source and a destination. Also policies stop being applied to specific rules and become flow-bound, allowing per flow QoS customization and thus every packet of a flow is subjected to the same QoS [24] allowing for unprecedented flexibility limited only by the capabilities of the implemented flow tables [25].

• Control logic is moved to an external entity, the aforementioned SDN controller. • The network becomes programmable through software application on top of the SDN

controller interacting to the data plane accordingly.

Following this terminology a new layered model of the network can be derived which is shown in Figure 2.5:

(35)

Figure 2.5: SDN architecture and its fundamental abstractions [3]

Connecting the SDN Control plane and the Data plane is the forwarding abstraction layer which abstracts away the hardware through Southbound protocols such as Openflow, Netconf or BGP [26]. The second layer of abstraction abstracts away all the topology and network distribution making available to the upper layers a global view of the network providing information about its status, connection point information and statistical data while the last layer abstracts away all the details of the network functions’ implementation providing APIs through which it is possible manage, monitor and configure the network’s policies and forwarding devices.

The SDN controller can be seen as a networking operating system and thus its charac-teristics or lack of them define boundaries and degrees of liberty of the overall system and what technologies and applications can be deployed on top and below them. It is the entity managing the networks’ control plane and it has the task of making available information regarding the network topology and status to applications while providing mechanisms which allow transparent configuration of the network devices according to the applica-tions’ policies. Table 2.2 establishes a comparison between four of the most up-to-date open source controller in terms of the controller’s architecture, its programming language, their API (Northbound, Southbound and Eastbound) and the diversity of protocols each of the APIs support.

(36)

Name Architecture Programming Language Northbound API Southbound API Eastbound API

Floodlight12 _Centralized _Java _{REST, Java RPC, Quantum} _{Openflow 1.0 ,1.3}

-ONOS13 _{Distributed Flat} _C++ _{REST ,Neutron} _{Openflow 1.0 ,1.3, Netconf} _RAFT

RYU14 _Centralized _Python _REST _{Openflow 1.0, 1.2, 1.3, Netconf, OF-config} -Opendaylight15 _{Distributed Flat} _Java _{REST, Netconf, Restconf} _{Openflow 1.0, 1.3, 1.4, OVSDB, Netconf, BGP, LISP} _{RAFT, Akka}

Table 2.2: SDN Controller Comparison

Regarding the controller’s architecture there are still only two types of architecture: Distributed Flat (ONOS and Opendaylight) and Centralized (Floodlight and RYU). While distributed architectures tend to be preferable when dealing with highly scalable scenarios, they also increase the control plane’s complexity and its performance may depend on the controller’s degree of maturity and how their Easbound APIs are implemented. Centralized architectures are generally better for smaller scenarios and are less complex to manage. In terms of Northbound APIs, REST is widely and supported by all controllers followed by the Netconf. Southbound APIs are, perhaps, the most most important aspects when choosing a controller as they limit the types of protocols that can be used to communicate with dataplane devices. Opendaylight is the SDN-C with the hightest number of Southbound interfaces and thus the most suitable candidate in scenarios with great diversity of southbound protocols. Lastly, regarding Easbound APIs (a categhory which is only applicable to Distributed architecture) Opendaylight supports both RAFT and Akka while ONOS only supports RAFT.

As it can be seen there is a great controller diversity, each with its advantages and disadvantages and a more in depth research should be done in order to ensure that the chosen controller meets the applications use cases.

Like NFV, SDN aims to enhance other already existing technologies and system architectures under specific circum-stances rather than acting as a replacement for the IP protocol. Several research initiatives have taken place over the past few years in order to assess how SDN can improve other research fields.

In [27] the authors analyse how SDN and NFV can be used together to detect information leaks in traffic through the deployment of Personally Identifiable Information detectors as VNFs and using SDN to allow load-balancers to dynamically redistribute traffic among the VNF instances. The authors state that the proposed architecture was found to outperform a non-virtualized implementation in terms of latency, packet loss and throughput. In [28] the authors use SDN as key-enabler technique for their architecture, allowing dynamic resource allocation in the cloud for recently attached devices and automatically mapping User Equipments to their correspondent virtual User Equipments.

12_{http://www.projectfloodlight.org/floodlight/} 13_{https://onosproject.org/}

(37)

2.3.1 Openflow

The OF protocol has been the de facto protocol chosen as the communication protocol between the SDN-C and the physical devices, the SDN switches, and is the most widespread design of SDN data plane devices. While OF begun at Stanford University in 2008 and its first version (1.0) was released in December 2009, currently OF is being managed by the Open Networking Foundation (ONF).

As with any protocol, OF has had several releases with each newer release adding new functionalities and improving already existing ones. Figure 2.7 illustrates the main differences between several OF releases. From this point onward, any specific reference towards OF will refer to OF v1.3 unless otherwise specified since this is the version this work is based upon.

Figure 2.6: Packet flow through the processing pipeline. [4]

Through this protocol, an SDN-C is able manage the flow table of each SDN switch both preemptively (through configurations) or reactively (responding incoming packets in the switch) as well creating complex pipelines which may perform various action upon arriving packets based on several match fields.

Traditional routing vs Openflow

In Traditional IP routing, IP datagrams are processed and forwarded by routers based on a next-hop method in which the next-hop is found by performing a lookup on the routing table. The routing table maintains records of the routes to various network destinations and is typically filled by a routing protocol although it also allows manual configuration. The decision method about which of the paths is best differs according to the algorithm in use but generally the best path is the one which minimizes the routing algorithm’s metric. Metrics vary from routing algorithm to routing algorithm and thus different algorithms may end up choosing different paths for the same network topology.

(38)

Unlike in IP, the OF protocol doesn’t process incoming packets on a next-hop basis but in and pre-defined orgin-to-destination path forwarding mechanism called a ”flow”. Figure 2.7 provides an overview of the functional blocks of a typical OF switch.

Figure 2.7: Main components of an OF switch [4]

On the control plane, each OF-enabled switch connects to one or more SDN-Cs through dedicated OF channels established between the switch and the controllers. On the data plane, each OF switch processes traffic through a set of OF input and output ports, sev-eral flow tables, group tables and meter tables which enable complex packet processing pipelines.

OF ports, both input and output, are ethernet ports which may be either physical or logical ports. A logical port is some sort of port that is not physical, such as a VLAN port, or an Ethernet tunneled port. Flow tables are constituted by flow entries each with 6 entries : match field, priority, counter, instructions, timeouts and cookies.

Group tables consists of a group entries. The ability for a flow entry to point to a group enables OF to represent additional methods of forwarding (e.g. select and all).

A meter table consists of meter entries, defining per-flow meters. Per-flow meters enable OF to implement various simple QoS operations, such as rate-limiting, and can be combined with per-port queues to implement complex QoS frameworks, such as DiffServ [4].

Each packet that enters the switch is sent through a processing pipeline whose behavior is depicted in picture2.8. This pipeline can be broadly separated into two stages: ingress and egress. Due to the fact that OF separates ingress from egress allowing for complex pipelines which can with multiple traffic forwarding patter for similar flows. Although the pipeline is divided into ingress and egress both stages are roughly similar. The pipeline process starts by finding highest-priority matching flow entry in flow tables while keeping

(39)

track of the actions it must later execute until there are no is no table next-hop. Then, it performs the specified operations (update action set, packet headers, etc) and finally it outputs the packet.

Figure 2.8: OF packet processing pipeline

Matching

When a packet is received,the switch starts by performing a table lookup in the first flow table and, depending on the processing pipeline, may perform other lookups in other flow tables. Packet header fields may be used as matching conditions for table lookups depending on the packet type. Typical match conditions include various protocol header

(40)

fields, such as Ethernet source address or IPv4 destination address. In addition to packet headers, matches can also be performed against the ingress port and metadata fields. A packet is only matched to a flow entry if all the match fields of the flow entry are match all the corresponding header and pipeline fields from the packet [4]. Also , metadata may be used to pass information between tables in a switch. The matching field supported varies with the OF Version supported by the Controller and the devices.

Actions & Action Sets

An action set is associated with each packet and it is empty by default. Flow entries can modify the action set and action sets are carried between flow tables. When the instruction set of a flow entry does not contain a Go-to-Table instruction, pipeline processing stops and the actions in the action set are applied.

Meters

A meter table consists of meter entries that enable the implementation of various per-flow, simple QoS operations such as rate-limiting. By combining OF meters with per-port queues it is possible implement complex QoS frameworks such as DiffServ.

A meter measures the rate of packets assigned to it and enables control over the rate of those packets. Meters are attached directly to flow entries (as opposed to queues which are attached to ports). Meters measure and control the rate of the aggregate of all flow entries to which it is attached.

Each meter may have one or more meter bands. Each band specifies the rate at which the band applies and the way packets should be processed. Packets are processed by a single meter band based on the current measured meter rate. The meter applies the meter band with the highest configured rate that is lower than the current measured rate. If the current rate is lower than any specified meter band rate, no meter band is applied.

(41)

2.4 Service Function Chaining

Over the last sections it has been analyzed how recent advances in NFV and SDN tech-nologies have changed the Network Service Orchestration (NSO) paradigm: SDN enables the virtualization of network resources while NFV virtualizes both physical resources and VNF while taking car of several aspects of their lifecycle management. However, traditional e2e NS provided by TSPs must also guarantee that the service traffic is steered through the services’ VNFs according to a predefined route i.e., the traffic of a secure network service must first be steered through a firewall before reaching an authentication server.

SFC is a relatively new technology that refers to the steering of NS’ traffic through a set of VNFs. IETF has had a major role towards the standarderization of SFC by releasing several Request for Comments (RFC) and Internet-Drafts tackling several aspects such as SFC architecture [29], Network Service Header (NSH) [30] or more recently SFC MANO [31].

According to [29], the SFC architecture can be divided in three layers: data layer, service layer and control layer. The first layer is relates to the network forwarding devices. The second layer refers to the classifiers, Service Funtion Forwarder (SFF) and the Service Function (SF). Lastly, the control layer manages the two other layers and manages policies.

(a) IETF architecture (b) ETSI architecture

Figure 2.9: SFC architecture without SDN (left) and with SDN (right) [5]

Figure 2.9a describes the traditional way TSPs setup a service by manually creating virtual channels for each service and configuring the required forwarding rules onto each forwarding devices participating in the service chain thus creating the notion of service.

Figure 2.9b show the new SFC paradigm achieved by combining traditional SFC, SDN and NFV. This new paradigm enables the creation of SFC chains which describe the service’s traffic steering specifications. They are then passed to the SDN-C which interprets them and makes the necessary configurations on the forwarding devices to establish the Service Function Path (SFP). The architecture also contemplates retrocompatability with legacy equipment through the deployment of SFC proxies that translate communications between SFC and non-SFC entities.

While ETSI’s SFC defines a reference architecture it does not define how it should be implemented. In [32] the authors divide the traffic steering types into three categories: header-base methods, tag-based methods, programmable switch-based methods.

(42)

2.4.1 Header-based methods

As the name suggests, in Header-based methods the SFP information is shared through a specific packet header that carries services’ information meaningful only to SFC entities. The most widely accepted header-based method is the NSH [30] which refers to a network header that is inserted on the service’s traffic to be read by SFFs.

This network header reflects the information present in the Virtual Network Function Forwarding Graph (VNFFG) namely the services’ Classifiers, composing Service Functions and the order in which they must be traversed.

At the entry point of the network a classifier tags the incoming traffic adding the NSH header and the SFFs forward the traffic across the network according to the information present in the header.

The SFP then becomes the mechanism used by the Orchestration platforms to express the result of applying a more granular policy and operational constraints to the abstracted requirements of a VNFFG,

This implementation has the disadvantage that it requires a network which is NSH-aware which should ensure that the packet follows the correct path throughout the network possibly adding, modifying or even removing the NSH if needed. NSH proxies are also an option to be considered as they may be used as intermediaries between NSH-unaware nodes and the rest of the network.

2.4.2 Tag-based methods

Already existing packet field information can also be used as a SFC mechanism. Packet fields such as Mac Address, Vlan, VXlan or MPLS can be used for SFC purposes and avoid the creation of specific header such as those presented in the previous subsection. This approach does come with the disadvantage that these packet fields may already be associated with other policies present in the network such as a QoS policy, which may lead unexpected network behavior.

2.4.3 Programmable switch-based methods

The last type of traffic steering methods are programmable switches which posses mul-tiple flow tables. Programmable switches play the role of classifiers and SFF. Traffic forwarding is usually based on re-classifying traffic based on programmed flow rules, such as access lists. This method has the disadvantage of being costly since it reclassifies flows traversing every switch and requires the installation of a sizable number of flow rules in different switches.

(43)

2.5 Network Service Orchestration Initiatives and

Technologies

2.5.1 Research Initiatives

T-NOVA

The focus of the FP7 T-NOVA project 16 _{was to create an integrated management}

architecture for the automated provision, configuration, monitoring and optimization of network connectivity and Network Functions as a Service (NFaaS). T-NOVA leverages cloud management architectures for the provision of resources and is based on ETSI’S NFV reference architecture, expanding it through the creation of a marketplace were both the community and vendors can share their NSs.

SONATA

The Service Programming and Orchestration for Virtualized Software Networks (SONATA) project focus on creating flexible programmability and optimizing the deployment of soft-ware networks for complex services/applications. SONATA provides an integrated develop-ment and operational process for supporting network function chaining and orchestration based on its own development kit and service platform. The SONATA Software Devel-opment Kit (SDK) offers functionalities and tools for the develDevel-opment and validation of VNFs and NS and the SONATA Service Platform offers functionalities which enable the orchestration and management of network services’ lifecycle.

5G-Transformer

The 5G-Transformer Project aims to transform the current mobile transport network into a Mobile Transport and Computing Platform (MTP) based on SDN, NFV, orches-tration, and analytics, which brings the Network Slicing paradigm into mobile transport networks. The 5G-Transformer architecture is based on three main components: a verti-cal slicer capable of creating network slices, as Service Orchestrator for end-to-end service orchestration and a Mobile Transport and Computing Platform which integrates fronthaul and backhaul networks.

2.5.2 Technological Solutions

Out of the research initiatives mentioned in the previous chapter were born several projects, each focusing on one or more different functional blocks of the NFV reference architecture. Figure 2.10 summarizes some of the most important projects and the their areas of research.

(44)

Figure 2.10: Comparison of the main MANO platforms [6]

From the TSP point of view, the projects that better tackle relevant use-cases are those which encompass NFV, NFVO, VNFM and VIM and preferably SDN and multi-site capabilities. While the need for NFV, NFVO, VNFM are self-explanatory due to the NS-driven business model of TSPs, VIM capabilities are also a requirement due to the fact that TSPs also usually own the infrastructure they operate upon and, therefore, they need to be able to configure it instead of delegating this function to another entity (i.e. cloud service provider). SDN and multi-site capabilities are also strongly recommended features since many of 5G future networks’ use-cases already see them requirements. From inspecting Figure 2.10 we see that roughly four of these projects fall under the scope we defined earlier: OSM, OpenBaton, ONAP.

ONAP

ONAP, the Open Network Automation Platform, is an orchestration platform that de-ploys a unified architecture and implementation enabling the design, creation and lifecycle management of both VNFs and NSs. ONAP was born out of the union of AT&T’s ECOMP (Enhanced Control, Orchestration, Management and Policy), an engine that powers its software-centric network, with the goal of enabling high utilisation of network resources by combining dynamic, policy-enforced functions for component and workload shaping, placement, execution and administration, and OPEN-O (OPEN-Orchestrator Project) an open source software framework and orchestrator to enable agile SDN and NFV operations. Some of its most notable features the ability to design closed-loop models for automating behaviors when recovery of faults reported by traps or alarms or capacity management as performance thresholds are crossed. ONAP is currently support by some of the most prominent telecommunications service providers such as AT&T, Huawei, Cisco, Ericsson and China Mobile.

(45)

Open Baton

Open Baton was created by the the Fraunhofer Fokus Institute and the Technical Uni-versity of Berlin and is an open source implementation if ETSI’s NFV MANO specification in conjunction with the TOSCA Standard. Open Baton (release 4 at the time of this publication) allows NFVO through TOSCA-based APIs, generic VNFM and Juju VNFM and an autoscaling and fault management system. Open Baton also has an integrated marketplace from which both vendors and the community can share content.

OSM

Open Source MANO, OSM, is ETSI initiative with the objective of creating an Open Source NFV-MANO framework aligned with ETSI’s reference architecture and standards. This framework is perfectly aligned with European TSP interests and follows all ETSI’s standards regarding data models and logical block interfacing. OSM separates service orchestration from virtual resources orchestatrion resulting in a modular framework which can be extended in several directions without affecting its overall operation. OSM also allows VNFM through Juju and has support for multiple VIMs for both computing and network resources. Its service orchestrator is based on open information/data models such as YANG. One of its major drawbacks when comparing with other MANO platforms is that its architecture is only able to orchestrate NSs in a single administrative domain.

2.5.3 Research Avenues

In the previous subsection it was analyzed the currents state-of-the-art reagarding NFV and NFV MANO. While many of the already existing solutions offer a certain degree of maturity, there is still a long road ahead before achieving carrier-grade orchestration systems that fully harness NFV’s performance while retaining legacy’s systems reliability and robustness. In this section it will be analyzed the research challenges that the state-of-the-art technologies still needs to overcome.

Network Service Lifecycle Management

The main challenges regarding NS Lifecycle Management are tightly coupled with the diversity of the requirements of the multitude of the NS ecosystem and the need solutions capable of performing management operations on an automatic fashion while the being able to accommodate this said diversity. These challenges may range from configuring interdomain mechanisms which may vary from domain to domain or being able to manage run-time features of NSs on an automated fashion. One of the possible solutions for this is the creation of algorithms based on heuristics and machine learning principles which automate these tasks such as ONAP’s CLAMP 17_.

(46)

Scalability

One of the main 5G future networks’ use-cases is the promise of Smart Cities and the massification of the Internet of Things. This massification can be translated into an increase in the number of connected devices in such a magnitude that it will start to pose a challenge to modern MANO frameworks since most solutions available centralize both design-time and run-time operations, an implementational aspect may pose scalability issues in the future. In order to avoid bottlenecks some works have been started towards distributed orchestration instead of using a single orchestrator [33].

Security

Virtualized environments are software-based by nature and thus they suffer from both all possible underlying hardware flaws and the hypervisor’s software flaw . Moreover, with the centralization of control of computing and network resources, a possible security breach in one node may potentially affect the whole domain that node is part of, leading to potential information leak or wider attack surfaces across the whole domain. Thus, an essential requirement for a multi-domain orchestration platform is the capability to hide specific details of each domain, ensuring the domain segmentation and preserving capabilities and resources to an external component.

Interoperability

The end-game for TSP is, naturally, interoperability across several domains which may spread across different geographical location and may be managed by different entities and based on different business models and orchestration frameworks. While ETSI MANO architecture standerderizes the orchestration frameworks’ functions blocks, it does not standerderizes its implementation nor does it define the data models they operate upon. This heterogeneity makes it difficult to chain NS leveraging different frameworks and thus, an essential step towards achieving full interoperability is standerderizing data and in-formation models as well as the general APIs which will enable seamless communication between orchestrators regardless of their implementations.

2.6 Chapter Summary

This chapter started by providing a quick overview about the new mobile telecommu-nications generation, 5G, followed by an analysis over the new generation’s enabling tech-nologies and the organizations involved in their standardization process. Since the scope of this work is focused on the Network Architecture & Core Network fields of research, it is then presented how NFV, SDN and SFC are changing TSP’s traditional business mod-els towards newer ones based on NFV architectures where both computing and network resources and network appliances’ functionalities are pushed towards the software plane,

(47)

leading to the creation of VNFs-based NSs. It is then seen how ETSI has been the organi-zation leading NFV’s standardiorgani-zation efforts and how ETSI NFVO and ETSI MANO led to the creation of several MANO frameworks such as OSM and ONAP that manage the lifecycle of both VNFs and NSs. This chapter wraps up by comparing these frameworks and identifying possible research avenues.

(48)

(49)

Chapter 3 Problem Statement

Throughout chapter 2 it has been seen that the virtualization of many aspects of NSO brought change to the traditional NS’s ecosystem and, although there have been many efforts by different organizations to both standardize and develop this new-found NSO model there are still many improvements that must be made before carrier-grade stable production environments are achieved.

In this chapter it is conducted a study focused on understanding what are new mobile generations’ new application domains and use-cases that leverage the technologies and architectures analysed in chapter 2, their challenges and their maturity state. The insights yielded by this study are then analysed and from them are be derived this work’s final scope and objectives.

3.1 IP-based Evolved Packet Core and its problems

The Evolver Packet Core (EPC) [34] is a framework for providing converged voice and data for fixed and 4G Long Term Evolution (LTE) networks while also providing interoperability features for 2G and 3G services.

The EPC has several functions such as aggregating traffic coming from fixed and mobile access networks to and from the Internet gateway, handling mobility management between base stations connected to the EPC and managing bandwidth and congestion in order to provide better than best effort QoS for applications.

The EPC is build on top of IP tunnel overlays to handle mobility management which poses a problem since tunnel management is centralized but the routing control upon which the tunnel routing depends on is distributed [7]. This approach not only severely constrains the link’s effective bandwidth due to tunneling headers but also poses a big problem for handovers since after a change in routing, different elements in the IP network can complete reconfiguration at different times, which may lead to routing instabilities, unwanted handover delays and misrouted packets.

SDN can improve this situation by simplifying management through the removal of the distributed IP routing control plane, replacing it with a centralized control plane and

(50)

allowing the mobile control plane for the EPC to be run entirely in a data center. Moreover, SDN guarantees that the control plane converges at a particular time which reduces packet loss and greatly improves handover flexibility.

Another important task of the EPC, as previously mentioned, is traffic classification and QoS levels assurance for different types of traffic. In the EPC context, QoS is assured either through DiffServ Code Points (DSCP) at the network layer or through IEEE 802.1q VLANs with 802.1p traffic class priorities [35].

3.1.1 Use-cases of SDN-enabled EPCs

In-Line Network Services

As previously seen, a NS can be defined as a final product sold to an end-user composed by a set network functions deployed between two locations which perform various actions upon the traffic that enters the service. SDN can be used as a means to simplify the traffic steering along the service chain without complex having to perform complex IP routing.

Figures 3.1a and 3.1b show contrast an use-case with and without SDN.

(a) Use case without SDN [7] (b) Use case with SDN [7]

In the traditional routing case, the service’s traffic need to be routed from the VM handling the ad insertion service back out to the IP router and then from there to the transcoding service, while in the OpenFlow case, the packet flows can be routed directly between the services.

Multihomed Terminals

In IP networks, providing support for multihoming terminals can cause lots of additional complexity, especially in mobile networks [36]. However, through the usage of the SDN protocol OF one can greatly simplify traffic forwarding due to the fact that OF’s traffic forwarding does not rely on IP network prefix matching but on programmed flows which do not forward traffic based on a destination IP. As a consequence, the EPC can advertise

(51)

different sets of network prefixes externally and have different kinds of traffic directed to different gateways.

Figure 3.2 illustrates an example where OF can be beneficial in a multihoming scenario. Generally, video traffic should be sourced over an upstream Internet Service Provider (ISP) with low bulk bandwidth rates and good caching while generic traffic can be sourced over lower tier ISPs. The OpenFlow controller and PCRF (Policy Control and Charging Rules Function) can set up different tunnels to the different gateways in the upstream path based on traffic type and on the downstream path the IP addresses are rewritten and pointed to the same terminal.

Figure 3.2: Upstream Multihoming with OpenFlow [7]

Moreover, upstream multihoming sometimes does not provide adequate provision for operator accounting and charging an aspect in which OF excel at since it has the option to natively collect per flow statistics which can be used for accounting and charging purposes.

Security & Isolation

Another set of import use-cases are tied to security issues and network isolation. Figure 3.3 illustrates one possible use-case.

TSPs usually provide security services in the form of Intrusion Detection Service (IDS) and Intrusion Protection Service (IPS) services which deployed to detect and prevent secu-rity problems. Since in these virtualized environments these secusecu-rity appliances share the same ”virtual space” with the EPC services and SDN-C on the application level, when a IDS or IPS system identifies a terminal as compromised it could issue modifications directly to the SDN-C which immediate isolates the compromised terminal’s traffic and redirects it to a different location for separate processing.

Inter point-of-presence network service orchestration

Jorge Vicente

de Oliveira

Orquestra¸

c˜

ao de Servi¸

cos de Rede com m´

ultiplos

Pontos de Presen¸

ca

Inter Point-of-Presence Network Service

Orchestration

Jorge Vicente

de Oliveira

Orquestra¸

c˜

ao de Servi¸

cos de Rede com m´

ultiplos

Pontos de Presen¸

ca

Inter Point-of-Presence Network Service

Orchestration

Contents

List of Figures

List of Tables

Acronyms

Chapter 1

Introduction

1.1

Motivation

1.2

Objectives

1.3

Dissertation Outline

1.4

Contributions

Chapter 2

State of the Art

2.1

5G - From Use-Cases to Standardization

2.2

NFV & ETSI’s NFV

2.2.1

ETSI MANO

2.2.2

ETSI NFVO

2.3

Network Virtualization & Software Defined

Net-working

2.3.1

Openflow

2.4

Service Function Chaining

2.4.1

Header-based methods

2.4.2

Tag-based methods

2.4.3

Programmable switch-based methods

2.5

Network Service Orchestration Initiatives and

Technologies

2.5.1

Research Initiatives

2.5.2

Technological Solutions

2.5.3

Research Avenues

2.6

Chapter Summary

Chapter 3

Problem Statement

3.1

IP-based Evolved Packet Core and its problems

3.1.1

Use-cases of SDN-enabled EPCs