5G testbed platform

(1)

Universidade de Aveiro Departamento de Eletrónica,Telecomunicações e Informática 2019

Eduardo Santos

Sousa

Plataforma de testes 5G

5G testbed platform

(2)

(3)

“Resilience is not a commodity you are born with, waiting silently on tap. It is self-manufactured painstakingly over time by working through your problems and never giving up, even in the face of difficulty or failure.”

— Lorii Myers

Universidade de Aveiro Departamento de Eletrónica,Telecomunicações e Informática

2019

Eduardo Santos

Sousa

Plataforma de testes 5G

5G testbed platform

(4)

(5)

Universidade de Aveiro Departamento de Eletrónica,Telecomunicações e Informática 2019

Eduardo Santos

Sousa

Plataforma de testes 5G

5G testbed platform

Dissertação apresentada à Universidade de Aveiro para cumprimento dos requisitos necessários à obtenção do grau de Mestre em Engenharia de Com-putadores e Telemática, realizada sob a orientação científica do Doutor Diogo Nuno Pereira Gomes, Professor Auxiliar do Departamento de Eletrónica, Tele-comunicações e Informática da Universidade de Aveiro, e do Doutor Rui Luís Andrade Aguiar, Professor Catedrático c/ Agregação do Departamento de Eletrónica, Telecomunicações e Informática da Universidade de Aveiro.

(6)

(7)

Dedico este trabalho à minha familia, que sempre me apoiou e em particular à minha mulher que me deu todo o apoio e suporte possível durante todos estes anos.

(8)

(9)

o júri / the jury

presidente / president Prof. Doutor Arnaldo Silva Rodrigues de Oliveira

Professor Auxiliar do Departamento de Eletrónica, Telecomunicações e Informática da Universi-dade de Aveiro

vogais / examiners committee Prof. Doutor Paulo Alexandre Ferreira Simões

Professor Auxiliar da Faculdade de Ciências e Tecnologia da Universidade de Coimbra

Prof. Doutor Diogo Nuno Pereira Gomes

Professor Auxiliar do Departamento de Eletrónica, Telecomunicações e Informática da Universi-dade de Aveiro

(10)

(11)

agradecimentos /

acknowledgements Desde já queria agradeçar aos meus orientadores a oportunidade que mefoi dada. Queria também a agradecer a todo o apoio, de familia e amigos, que me ajudou a levar esta fase até à sua conclusão. Não posso deixar de agradeçar também a todos os colegas do Instituto de Telecomunicações, do projeto 5GinFIRE e da comunidade OSM, que me ajudaram a compreender os mais variados aspetos relativos aos tópicos abordados nesta dissertação. Agradecimento ao IT UID/EEA/50008/2019.

(12)

(13)

Palavras Chave 5G, SDN, NFV, MANO, OSM, virtualização, plataforma de testes, 5GinFIRE

Resumo As operadoras de telecomunicações estão a evoluir as suas redes para

su-portar os verticais do 5G, em todos os aspectos que os definem e afetam a parte central da rede. As operadoras estão a deixar de utilizar dispositivos especificos que têm um único propósito para passar a virtualizar as mesmas e utilizar servidores genéricos, o que permite a redução de custos enquanto suporta uma multitude de cenários sem ser necessário fazer nenhuma alte-ração à rede. Este trabalho serve para determinar se a arquitetura proposta pelo projeto 5GinFIRE e recorrendo a projetos de software open source é pos-sível criar uma plataforma de testes para 5G, usando as lições aprendidas por esses projetos e melhorá-los para suportar para suportar mais cenários. Os projetos utilizados são o Openstack e o Open Source MANO e foram usados para construir a plataforma de testes 5G, podendo esta ser utilizada tanto lo-calmente como de uma forma distribuída. Esta plataforma de testes deverá suportar virtualização de funções de rede, sendo que estas tanto podem ser criadas localmente por um orquestrador local ou integrando um cenário onde existem várias plataformas que são controladas por um orquestrador central, que aloca as diferentes funções pelas plataformas disponíveis.

Tendo feito uma revisão da literatura disponível sobre os tópicos de redes de 5G, NFV, SDN, MANO e uma análise da arquitetura proposta pelo projeto 5GinFIRE, esta dissertação demonstra como é possível construir uma plata-forma de testes 5G usando projetos de software open source bem como a testar e validar que funciona como é suposto. Este trabalho também apre-senta como a plataforma foi interligada com outras para formar um cenário distribuído. A validação é também um aspeto importante deste obra, onde fo-ram realizados vários testes funcionais de forma a verificar que a plataforma poderia ser utilizada localmente como sendo parte de uma rede de platafor-mas. Foi também criado um conjunto de funções de rede virtualizados, que constituem um serviço de rede complexo, que serviu para determinar que era possível instanciar o mesmo na plataforma bem como determinar quais eram os pontos fracos do orquestrador. Os resultados indicam que é possível criar uma plataforma de testes 5G usando software open source, usando diferen-tes cenários e que um orquestrador genérico pode ser utilizado para instanciar serviços de rede localmente como de um ponto central que usa diferentes pla-taformas. Foram realizadas varias contribuições para o projeto Open Source MANO e a criação de um serviço de rede complexo que pudesse ter diferentes funções em diferentes plataformas é possível. Será necessário mais trabalho para continuar a melhorar o Open Source MANO e outras tecnologias, para se tornarem mais robustas e que tenham uma maior adoção.

(14)

(15)

Keywords 5G, SDN, NFV, MANO, OSM, virtualization, testbed platform, 5GinFIRE

Abstract Telecommunication networks are evolving their networks to support 5G

ver-ticals and all that it entails for the core aspects of the infrastructure. These networks are moving from the use of single purpose appliances to virtualiza-tion and the usage of commercial off the shelf hardware, allowing for cost reduction while supporting a myriad of use cases without a major change to the infrastructure. This work aims to determine if the proposed architecture by the 5GinFIRE project can be used for a 5G testbed and using open source software to make all possible, reusing lessons learned from those projects and enabling them to support more use cases. It builds on all the work made on those open source projects such as Openstack and Open Source MANO and apply them to create a 5G testbed that can be used both locally and a distributed way. This testbed should be able to support network function vir-tualization workloads either deployed by a local orchestrator or be part of a multi-site setup, where a central orchestrator controls multiple sites and parti-tions that workload into as many sites as necessary.

Based on a review of the literature available for 5G networks, NFV, SDN, MANO and an analysis of the proposed architecture of the 5GinFIRE project, this thesis demonstrates how to build a 5G testbed using open source projects and how to test it, to validate that it is working properly. It also presents how the testbed was interconnected with other testbeds to create a multi-site envi-ronment. The validation part has a big emphasis in this work, where multiple functional tests were ran to see if it worked both locally and in a multi-site setup. It was also created a complex workload to determine that a multi-site scenario was achievable and to determine what were the pitfalls of the orches-trator. The results indicate that it is possible to create the 5G testbed, that can be used in various configurations and that a generic orchestrator can be used to deploy workloads both locally and from a central location to many points of presence. There were several additions contributed to the Open Source MANO project and the creation of a complex workload was beneficial for the whole community, to demonstrate that it is possible to create workloads that span multiple sites and can be deployed from a central location. Further work is needed to keep improving Open Source MANO and other technologies, to mature them and drive them for a wider adoption.

(16)

(17)

Charm generator . . . 61 General fixes . . . 61 Appendix B 63 Local Testing . . . 63 VNF 01 . . . 64 VNF 02 . . . 65 VNF 03 . . . 66 NS 01 . . . 68 NS 02 . . . 68 NS 03 . . . 69 NS 04 . . . 69 NS 05 . . . 70 NS 06 . . . 71 Heimdall VNFs . . . 71 Cloud Gateway VNF . . . 72 Cloud Classifier VNF . . . 74 Cloud nDPI VNF . . . 76

(19)

Cloud MITM VNF . . . 77 Cloud DNS VNF . . . 79 Cloud E2G VNF . . . 80 Cloud HAVP VNF . . . 82 Cloud Squid VNF . . . 84 Cloud Ziproxy VNF . . . 85 Cloud Router VNF . . . 87

Cloud uC (New Component) VNF . . . 88

Edge Classifier VNF . . . 90 Edge Gateway VNF . . . 92 Edge MITM VNF . . . 94 Edge Router VNF . . . 95 Edge Squid VNF . . . 97 Heimdall NS . . . 99 Cloud Classifier NS . . . 99 Cloud nDPI NS . . . 100 Cloud MITM NS . . . 101 Cloud Hybrid NS . . . 102 Edge Classifier NS . . . 104 Edge Gateway NS . . . 106 Edge MITM NS . . . 106

(20)

(21)

List of Figures

2.1 Mobile data traffic by application category per month in percentage . . . 5

2.2 Mobile subscriptions by technology (billion) . . . 6

2.3 Cellular IoT connections by segment and technology (billion) . . . 6

2.4 SDN Planes . . . 7

2.5 Dedicated network appliances and NFV comparison . . . 9

2.6 Network Function Virtualization (NFV) Management and Orchestration (MANO) high level components . . . 11

2.7 NFV MANO functional blocks . . . 14

3.1 Testbed organization . . . 26

3.2 Internet connection . . . 27

3.3 Map of the European testbeds . . . 28

3.4 Authentication plugin system . . . 30

3.5 Hierarchical Permission Tree . . . 34

4.1 Placement of nodes . . . 39

4.2 Internal topology of Virtual Network Function (VNF) 01 . . . 41

4.3 Internal topology of VNF 02 . . . 42

4.4 Internal topology of VNF 03 . . . 42

4.5 Internal topology of Network Service (NS) 01 . . . 43

4.6 Internal topology of NS 02 . . . 43

4.11 Heimdall Web Architecture . . . 49

4.12 Heimdall Hybrid Web Edge Topology . . . 50

4.13 Heimdall Hybrid Web Cloud Topology . . . 51

(22)

(23)

List of Tables

4.1 Result of tests performed locally . . . 47 4.2 Result of tests performed in 5GinFIRE project . . . 47

1 Unit test contributions . . . 59 2 Authentication backend contributions . . . 60 3 Service Function Chaining contributions . . . 61 4 Charm generator contributions . . . 61 5 General fixes contributions . . . 61

(24)

(25)

Glossary

4G Fourth Generation

5G Fifth Generation

ACL Access Control List

AD Active Directory

API Application Programming Interface

BSS Business Support System

CAPEX Capital Expenditure

CI Continuous Integration

CNF Containerized Network Function

COTS Commercial Off-The-Shelf

CPU Central Processing Unit

CRUD Create, Retrieve, Update and Delete

DNS Domain Name System

DPDK Data Plane Development Kit

EM Element Management

ETSI European Telecommunications Standards Institute

FCAPS Fault Management, Configuration Management, Accounting Management, Performance Management and Security Management

FIRE Future Internet Research Experiments

HNF Hybrid Network Function

HTTP HyperText Transfer Protocol

IETF Internet Expert Task Force

IM Information Model

IoT Internet of Things

IP Internet Protocol

IT Information Technologies

I/O Input/Output

LDAP Lightweight Directory Access Protocol

LFN Linux Foundation Networking

LTE Long-Term Evolution

LTS Long Term Support

MANO Management and Orchestration

MDL Module Development Lead

MITM Man-in-the-middle

mmWave Milimiter Wave

MTU Maximum Transmission Unit

NBI Northbound Interface

NC Network Controller

NFV Network Function Virtualization

NFVI Network Function Virtualisation Infrastructure

NFVO NFV Orchestrator

NIC Network Interface Controller

NS Network Service

NSD NS Descriptor

NSH Network Service Header

NUMA Non-Uniform Memory Access

ODL Open Day Light

ONAP Open Network Automation Platform

ONOS Open Networking Operating System

OPEX Operational Expenditure

OPNFV Open Platform for NFV

OSM Open Source MANO

OSS Operational Support System

PCI Peripheral Component Interconnect

PNF Physical Network Function

PoP Point of Presence

RAT Radio Access Technology

RBAC Role-based Access Control

REST Representational State Transfer

RO Resource Orchestrator

SDN Software Defined Network

SFC Service Function Chaining

SLA Service Level Agreement

SR-IOV Single Root Input/Output Virtualisation

SSH Secure Shell

TACACS Terminal Access Controller Access Control System

URL Uniform Resource Locator

(26)

VL Virtual Link

VLD VL Descriptor

VM Virtual Machine

VNF Virtual Network Function

VNFD VNF Descriptor

VNFFG VNF Forwarding Graph

VNFFGD VNFFG Descriptor

VNFM VNF Manager

VPN Virtual Private Network

(27)

CHAPTER

1

Introduction

1.1 Motivations

In an ever evolving world, where virtualization is now replacing dedicated hardware it is now more pertinent than ever to start tackling the problems with NFV and its orchestration. These are relevant technologies which will bring the required agility and control to the new Fifth Generation (5G) networks being developed and deployed. For this, projects like Openstack1_,

Open Platform for NFV (OPNFV)2_{, OSM}3_{and Open Network Automation Platform (ONAP)}4

are relevant and will shape the face of the new virtualization methodologies being applied to these new 5G networks. Virtualization also allows to reduce Capital Expenditure (CAPEX) and Operational Expenditure (OPEX) when using the right tools such as the above mentioned projects and building the right skill set in the companies that will implement and manage the 5G environments.

It is also important to take into account that this is work is developed in the context of the 5GinFIRE5_{, which introduces an architecture for interconnected 5G testbeds being}

centrally managed by a single MANO stack, in this case OSM. Therefore, it is important to apply these technologies to 5GinFIRE to see how a 5G networks with multiple NFV PoP can be implement and managed, to also see what kinds of NS can be deployed there.

One of the constraints introduced by the 5GinFIRE project introduced in this work is the usage of OSM as the MANO stack to be used, for which there is a need to evaluate its stability and enhance it to make a more powerful MANO, so it can cover as many use cases as possible. Also the contributions made should also ease the work of the end user, for the development and onboarding of VNFs.

The deployment of a 5G testbed is also essential to support the development of new technologies and products associated with 5G. The testbed should be built using open source

1 https://www.openstack.org 2_{https://www.opnfv.org} 3 https://osm.etsi.org 4 https://www.onap.org 5 https://5ginfire.eu

(28)

technologies and support different industry verticals, so these industries can innovate and create new products that benefit from all the advantages of this new connectivity. Some of the verticals that can benefit from this testbed are automotive and massive Internet of Things (IoT). This testbed should be orchestrated using a MANO platform, in order to provide the required single pane of glass to instantiate, manage and monitor the different applications to be deployed.

1.2 Objectives

The main objectives of this work are:

• Analysis the state of the art of 5G, MANO and virtualization technologies • Study of the 5GinFIRE project architecture

• Design and implementation of the testbed • Contributions to the OSM project

• Development of complex NS

As a starting point in this work, there needs to be an analysis of the work done and the state of the art in the various areas where it is inserted. This allows to verify that the work being performed is aligned with the rest of the academic community while being innovative. The 5GinFIRE architecture needs also to be studied and reviewed, after that needs to be adapted to resources allocated and different constraints. With the study of the 5GinFIRE project architecture completed, the work should follow suit on design and implementation of the testbed, which will generate contributions to the OSM project because of problems or bugs found that were fixed. The development of complex NS is a byproduct that will be generated from the necessity of creating a complex use case for testing in the platform while using all the functionalities of the testbed.

1.3 Thesis organization

The thesis is divided into five chapters and one appendix. The five chapters are: • Chapter 1: Introduction

• Chapter 2: 5G Networks

• Chapter 3: 5G Testbed implementation • Chapter 4: Results

• Chapter 5: Conclusions

The chapter 1 presents an introduction to the work, the motivations and objectives. Chpater 2 is an overview of the state of the art in 5G networks, NFV MANO and the various solutions for NFV orchestration. It also shows the constrains defined by the 5GinFIRE project on this thesis. Chapter 3 presents the implementation of the 5G Testbed and the various contributions to OSM, from multi-tenancy support and RBAC to multiple Linux distributions. Chapter 4 introduces the tests performed in more detail and the results of those tests, it

(29)

also shows the major contributions made to the OSM project. Finally, it also presents the developed NS to test complex multi site deployments. Last but not least, is chapter 5 which condenses the conclusions of the performed work, while presenting the future work that can be performed and key points of the thesis work.

Appendix A contains all the changes performed to the OSM project which were accepted into the project codebase. Appendix B contains all the VNF and NS descriptors created during the development work.

(30)

(31)

CHAPTER

2

Fifth Generation Networks

This chapter briefly explains what 5G Networks are, how they differ from previous specifications and what technologies enable their creation, control and management. It explains 5G networks concepts in broad strokes and afterwards drills down on to more relevant themes for the presented work, like NFV, SDN and MANO.

2.1 5G Networks

Nowadays, users demand more from their networked devices (ex. computers, mobile phones) and their requisites are not only faster processors or more memory. They demand higher bandwidth and lower latency of the networks since they use a multitude of online services. Those services may require higher bandwidth (ex. video streaming), lower latency (ex. online gaming) or both (ex. video conferencing).

(32)

Mobile data traffic is also on the rise [1], as shown in Figure 2.1, and networks must support it. In 2018, the total traffic traversing the internet amounted to around 28 exabytes per month. Ericsson1 _{projects that the trend is to continue to rise since it is estimated to}

reach 131 exabytes per month by 2024.

Figure 2.2: Mobile subscriptions by technology (billion) [1]

Figure 2.3: Cellular IoT connections by segment and technology (billion) [1]

Devices connected to the Internet are also rising, as shown in Figure 2.2 and Figure 2.3 due to the fact of the increase in mobile subscriptions and the ascension of the IoT devices. These are low power devices that connect to the network to send their data to a central server. Just in 2017, there are an estimated 0,6 billion devices connected, and by 2024 there are going to be 4,1 billion.

1

(33)

5G networks are a new approach for communication networks [2][3]. It is a new paradigm in contrast with Fourth Generation (4G) networks. 5G networks bring new Radio Access Technology (RAT), like for example Milimiter Wave (mmWave), integrating them with old technologies such as Long-Term Evolution (LTE) and IEEE 802.11 (WiFi). The roots of this work are not to focus on RATs but the virtualization of the network core components. The network core components can be virtualized to bring down CAPEX and OPEX and gain further agility. To support the previously mentioned requisites, 5G networks must leverage concepts like SDN and NFV.

2.2 Software Defined Network

Software Defined Network is a technology where the network intelligence is separated from the switching and routing hardware, making possible for a network controller, that knows the whole network and its status, to create the best path to forward traffic. To make this possible, as shown in Figure 2.4, the network is divided into three planes: management plane, control plane and data plane.

Figure 2.4: SDN planes

The management plane is where networking applications run. These applications are responsible for demanding from the Software Defined Network (SDN) controllers that their requests be satisfied by the network. These requests can be to forward traffic by a specific path, allocate a minimum amount of bandwidth or block traffic from traversing the network. The more widely used SDN controllers used are Open Day Light (ODL)2 _{and Open Networking}

Operating System (ONOS)3_.

The control plane are the connections used by the SDN controller to communicate with the SDN switches. In this plane, the SDN controller can configure the forwarding tables of

2_{https://www.opendaylight.org} 3

(34)

the SDN switch as well receive information about network flows and switch loads to make decisions.

The data plane is composed of SDN switches the forward the traffic according to the forward tables provided by the SDN controller.

SDN is an essential technology because it allows controlling network forwarding policies dynamically using a central repository of knowledge about the network [4]. This programma-bility allows other solutions to work on top of the SDN controller to organize the network in a way that suits their needs.

2.3 Network Function Virtualization

Network Function Virtualization is an approach where the network function software is decoupled from a specific piece of hardware, leveraging the power of virtualization, which leads to less dedicated network appliances [5]. Network functions are separated into VNFs, where a VNF is an abstract concept that encapsulates all the Virtual Machines (VMs) required to run the software, internal networks that those VMs use to communicate. These VNFs also possess connection points that are used to interconnect them to other VNFs or networks provided. The connection points can be mapped to an interface in one of the VMs that compose the VNF or one of the internal networks. A VNF can also be implemented using containers, in this case it would be called a Containerized Network Function (CNF), or it could be implemented using both technologies, which would make it a Hybrid Network Function (HNF). VNFs can run on Commercial Off-The-Shelf (COTS) hardware or on a virtualized environment, enabling more flexibility on deployment. Running VNFs on virtualized hardware facilitates scaling their capabilities, allowing for scaling out in peak times and scaling in when the network traffic is low, releasing resources for other VNFs [6].

(35)

Figure 2.5: Dedicated network appliances and NFV comparison

Using NFV reduces CAPEX because there is no need to buy expensive hardware to perform a determined network function, this cost is now part of the cost of the high capacity servers running as part of the Network Function Virtualisation Infrastructure (NFVI) in the datacenters, as shown in Figure 2.5.

It also reduces OPEX because network operators will not need to waste money on space for the proprietary hardware nor will they waste money on replacing that hardware in case of failure since it can reboot quickly the VM or container where the VNF is running. There is also no need for highly specific certifications just to manage a certain network appliance, which reduces this cost from the OPEX. The reduction of the OPEX might not come from the reduction of the energy requirements, as this topic requires further study and testing [7]. NFV success is dependent on other technologies like hypervisors and hardware virtual-ization extensions. A VNF should be portable between different hypervisors and to be high performance the hypervisors where VNFs are running should support hardware virtualization extensions or technologies for multiple components like the processor or networking compo-nents. These hardware virtualization extensions or technologies are relevant to accelerate

(36)

packet processing like Data Plane Development Kit (DPDK)4 _{or to provide dedicated}

net-work hardware like Peripheral Component Interconnect (PCI) Passthrough and Single Root Input/Output Virtualisation (SR-IOV).

VNFs can be large shared VMs for the case of routers and firewalls, but they can also be implemented using single-purpose VMs, VMs running unikernels or in containers, for single applications that are smaller like logging or monitoring. These are called micro-VNFs [8] [9]. So there is the possibility to run a smaller VNFs for single-purpose applications for different projects/tenants, while using fewer resources and maintain a similar performance.

In summary, NFV allows reducing costs while gaining agility when managing networks, it also helps reduce the time to market of a new network service, because networking services are now software.

2.4 ETSI NFV MANO Architectural Framework

NFV as a concept and methodology is limited to virtualization of network functions. To extend the capabilities of NFV, the European Telecommunications Standards Institute (ETSI)5

introduced the NFV MANO architectural framework to provide management and orchestration capabilities. This architectural framework formalizes concepts and objectives to be solved [10] [11]. It presents relevant aspects of Management and Orchestration of NFVI, VNFs and NSs. It also addresses other relevant themes like: fault and performance management, policy management and testing aspects of NSs. It makes also some observations on the interconnection with other systems like Operational Support System (OSS) and Business Support System (BSS) to integrate business rules to make decisions and bridge the gap between the technology side and business side.

4

https://www.dpdk.org

5

(37)

2.4.1 Concepts and Objectives

Figure 2.6: NFV MANO high level components

The NFV MANO architectural framework revolves around three principal components called NFV Orchestrator (NFVO), VNF Manager (VNFM) and Virtualized Infrastructure Manager (VIM), as shown in Figure 2.6. These components are responsible for managing and orchestrate: NSs, VNFs and NFVI. These components map different concepts and responsibilities of the framework.

Management and Orchestration of NFVI takes into account all the provisioning that is required, such as VMs, Virtual Links (VLs) or VNF Forwarding Graphs (VNFFGs). It also considers the different relations that exists between the previously cited resources that are created on demand and resources that are already present in the network, which are the case of the Physical Network Functions (PNFs). The component that will handle these concepts will be the VIM. It will be responsible for managing the compute, storage and network resources and interconnect with the present PNFs if needed. So the MANO component present in the architectural framework should take all this into consideration as well, as it should understand that a NFVI may be composed of one or multiple NFV-Point of Presence (PoP).

VNF Management and Orchestration aspects include the following topics: instantiation, scaling, update/upgrade, termination. Instantiation is the act of reserving the resources and instantiating the necessary VMs for the VNF as it is specified in the template. Scaling is the act of increasing or decreasing the amount of resources allocated to that VNF. Scaling has four actions that should be taken into account: scale in, scale out, scale up and scale down. Scale in is to reduce the capacity allocated to the VNF, by terminatingVMs that are no longer required for the current capacity level of the VNF. Scale out is the opposite of

(38)

scale in, therefore when there is a need to increase the capacity of the VNF more VMs can be instantiated to deal with that extra effort being required from the VNF. Scaling up is increasing the capacity of the current VMs that belong to the VNF, that increase in capacity can be mapped to different resources made available to the VNF, such as the number of Central Processing Units (CPUs) required or the amount of memory used. Scaling down is the opposite of scaling up, where you decrease the amount of resources in the VMs used by the VNF. Scaling might require reconfiguration of the VNF. Aside from scaling, there is also update and/or upgrade of the VNF which represents the act of changing the software that the VNF is running and/or the change of the configurations on which the VNF is operating. Termination represents the last aspect that should be under consideration in managing and orchestrating VNFs, which handles the release of the resources that were associated with the VNF and its composing VMs.

A Network Service is the combination of VNFs and PNFs to perform a specific task, therefore for the Management and Orchestration of NSs there is a need to interconnect with the Management and Orchestration of VNFs. The aspects under consideration are: on-boarding, instantiation, scaling, update, Create, Retrieve, Update and Delete (CRUD) aspects of VNFFG and termination. NSs template should be stored in a catalogue, making them available for multiple instantiations. The act of storing a template for a NS in the catalogue is called on-boarding. This operation might be quite complex since it can undergo many phases, like validation and verification. Once the NS is on-boarded it is available to be instantiated. Instantiation is the next phase and represents the act of instantiating the constituent VNFs and the reservation of the required networking resources that are specified in the VNFFG. The VNFFG models the traffic paths that should be established for the NS to work properly. Once a NS is instantiated and processing network traffic there might be a need to scale it, to be able to support more network traffic. Scaling a NS is a complicated process, since there might be distinct groups of VNFs that might need to be scaled in different forms, so this process requires a detailed specification to be well managed and orchestrated. A VNFFG is a model, where it is specified how the traffic is supposed to be steered in the composing VLs, from VNF to VNF. This model might be already created in the NS template or might be added at a later time when is needed. Since networks are a dynamic environment and different events may occur, which may require the VNFFG to be updated. There is a myriad of cases where this might be necessary, like scaling or fail-over. A VNFFG might also not make any sense to be active, so there is also the option to delete it. In this case all the other VNFFGs that are still active will be the ones steering the traffic. Once a NS is no longer needed, it should be terminated and its resources freed back to the NFVI pool.

There are also other aspects to take into account such as Fault Management, Configuration Management, Accounting Management, Performance Management and Security Management (FCAPS), policy management and testing. FCAPS can be very broad set of topics, therefore NFV MANO prefers to focus on certain topics, such as: Service Level Agreement (SLA) fulfilment, performance measuring, fault correlation, fault resolution and result calculation and aggregation. SLA fulfilment is the most important topic because these agreements are

(39)

how companies establish the quality of service that should be expected from the service provider. These agreements establish the metrics and values that those metrics should have. If those agreements aren’t kept it allows the company that is purchasing the service to ask for compensation or stop using that service. Performance measuring is also mentioned, to the extent that it should be present to allow the operator of the MANO platform to check if NSs and VNFs are working within the expected performance ranges for the resources allocated to them. Performance measurements should also allow VNF developers to debug their VNFs when in the development phase. Fault correlation is corner stone for root-cause analysis and fault resolution. Without fault correlation it will be more difficult to identify what is the source of the problem, since without this it is impossible to correlate the different events and faults that might appear to identify which lead to that specific fault, which blocks the determination of the root-cause. To able to do fault resolution, the previous two topics, fault correlation and root-cause analysis, must be present in the platform, so the fault can be corrected and the damage fixed. Policy management is another aspect to have into consideration, since it refers to the management of the rules the specify the behaviour of the NFV MANO functions. These policies specify how a determined function in the MANO should behave, in what conditions it can be executed and to which VNF, NS or NFVI it is bound to. Policies can be defined for example on how to do scaling and under what conditions. The policies can be automatically triggered or manually by another system such as an OSS or a BSS. Finally, there is the testing aspects, since whenever a VNF or an NS is being developed there is need to do a barrier of tests to check for performance issues, operational problems, if it provides the desired functions while matching the specifications among other tests that might be needed.

For a MANO platform to be useful there is the need to be interconnected with other systems, so it can be used to provide functions in the network that the operators need. There needs to be a connection/relation between the MANO and the other systems from the operators, systems like the OSS and BSS. These systems need this connection in order to orchestrate the MANO platform to create the functions that they need, in a personalized manner, according to their business rules. For this connection to be useful the interface must be an open standard which is consistent and well defined. There should be common data models, so that the information exchanged by all the interacting systems is interpreted in the correct sense. The data should be modelled into multiple data models, allowing for decoupled data models that can be related to each other using mappings like identifier fields. These decoupled information models allow for a greater agility since a data model can be changed without having to change all the other data models. Making these data models available and well defined and having an open, consistent and well defined interface allows these systems to easily integrate their calls to the MANO. Having standardized data models and interfaces should increase the speed of on-boarding new VNFs and NSs, or new features, into the network and allows these systems also to orchestrate one or multiple MANOs platforms. This interface should also contemplate methods to collect real-time data analytics. A standard interface is interesting but is only possible if every vendor/implementer converges to implement it into his system and if in the end it is useful and broadly used by the systems that should use them.

(40)

There are two different administrative domains, the infrastructure domain and tenant domain. These two domains are used to separate responsibilities in the NFV system because there might be cases where an organization will not control the whole system. The infras-tructure domain is application agnostic and is responsible by maintaining the NFVI. An infrastructure domain may provide resources from the NFVI pool to one or many tenant domains. A tenant domain is where the responsibilities for NSs and VNFs instances resides at. A tenant domain may require resources from one or more infrastructure domains. So when a tenant domain requires resources to create new instances, it must require those resources from the infrastructure domains to which it has access. When it no longer need those resources, it should also inform that specified infrastructure domains that those resources are no longer being used. Domains are necessary to establish boundaries to define who manages what.

2.4.2 Functional Blocks

Figure 2.7: NFV MANO functional blocks

In a high-level block overview of the framework, it is composed of the following blocks: VNFs, NFVI and NFV MANO. Each block may be composed of multiple solutions working together or may be made of a single solution that fulfils all the functions assigned to that block.

The VNFs block is the software that runs on the VMs supplied by the NFVI to perform a specific network function. This function can be a firewall, a router among others.

The NFVI block is responsible for managing the infrastructure and virtualization software. It must also provide VMs to run the VNFs supplied by the VNFs block. The software component that this block represents is a VIM. The VIM is responsible for managing the infrastructure where the VNFs will run. There is a multitude of VIMs, while the most used are OpenStack and VMware vSphere6_.

6

(41)

The MANO block is responsible for the management and orchestration of VNFs. To do this, it orchestrates the VIMs to provide VMs to run VNFs and is responsible for the management of those VNFs, their configuration and their life cycles. The MANO block should also interconnect with the OSS and the BSS since those components are responsible for controlling the business rules in communication networks. There are several contenders in this space to fulfil the role of the MANO block, like for example OSM, OpenBaton7_{, ONAP,}

Openstack Tacker8_. NFV Orchestrator

The NFVO functional block fulfils two different functions: Resource Orchestration and Network Service Orchestration. These functions can be kept in the same component or separated. They map to different administrative domains, so there are different responsibilities at stake. It makes sense to keep them together, since it will increase the integrity of the common information base and certain functions directly depend from functions that belong to a different administrative domain. These complex functions that depend from functions on both administrative domains benefit from being implemented in the same component, since it reduces the integration of a complex set of calls between administrative domains [12] [13].

Here it is presented a non-exhaustive set of capabilities that should be provided by the NFVO in the context of the Resource Orchestration functions.

• Validation and authorization of NFVI resource requests from VNFMs. • NFVI resource management across operator’s Infrastructure Domains.

• Supporting the management of the relationship between the VNF instances and the NFVI resources.

• Policy management and enforcement for the NS and VNF instances.

• Collect usage information of NFVI resources by VNF instances or groups of VNF instances.

Resource requests from the VNFM need to be validated to guarantee that the VNFM is not requesting resources that do not exist, that are currently are being used or that overload the NFVI. Resource requests from the VNFM should also be verified for authorization purposes to make sure that VNFMs are allowed to issue that resource request for that VIM and what resources they are allowed to request. NFVI resource management across operator’s Infrastructure Domains is also important, since it allows to have multi-PoP deployments, which may reduce costs, fulfil SLAs or reach more clients. To have the ability of efficiently managing the NFVI resources, the NFVO must maintain and update information about the relationships between the VNF instances and the NFVI resources. This in turn will allow for better resource management, specifically when destroying VNF instances, since resource release will be easier and cleaner. Policy management and enforcement for the NS and VNF instances is required because it there may be certain policies that will affect reservation and/or allocation of NFVI resources. These policies may also be put in place for placement

7_{https://openbaton.github.io} 8

(42)

optimization such as affinity/anti-affinity, PoP geolocation, regulatory rules, resource usage optimization, among others. Usage information of the NFVI resources by VNF instances or groups of VNF instances is important to collect because these metrics will help determine the total amount of resources used across all the VIMs present in the NFVI, it will have metrics that can be useful in pinpointing where there unused resources that are allocated to VNF instances. This information can all be aggregated to determine when a PoP needs to be upgraded or more capacity needs to be added to the PoP or by creating a new PoP with new resources. All these items relate with the Infrastructure Domain because of the interaction from the VIMs in the NFVI and the NFVO that must orchestrate all the resources.

Here it is presented a non-exhaustive set of capabilities that should be provided by the NFVO in the context of the Network Service Orchestration functions.

• Management of NS deployment templates and VNF packages. • NS instantiation and NS instance life cycle management. • Management of the instantiation of VNFMs.

• Management of the instantiation of VNFs.

• Validation and authorization of NFVI resource requests from VNFMs. • Management of the integrity and visibility of the NS instances.

• Management of the NS instances topology. • Management of the NS instances automation.

• Policy management and evaluation for the NS instances and VNF instances.

Part of the Network Service Orchestration functions is to manage NS deployment templates and VNF packages, in order to be able to instantiate them and manage their life cycle. This includes also the management of the instantiation of VNFs associated with those NSs. Management of the instantiation of VNFMs is a function that could be important, but right now the functionality of the VNFM is being integrated directly into the MANO platform software. Certain VNFs might require their own VNFM or Element Management (EM), meaning that in practice there will be two VNFMs, one generic which is part of the MANO platform software and one that is specific to the VNF. So validation and authorization for the usage of NFVI resources from the VNFMs might not happen, since they’re internally managed in the MANO platform software and the dedicated VNFM will not interact with the NFVO since it is not responsible by the management of the VNFs.

NFVO will manage the integrity of NS instances, meaning that it needs to guarantee that the NS instances maintain all their VNFs and VNFFGs, in all the states that they might go through, such as scale in or out and changes to the VNFFG. This also reflects into the management of the NS instances topology, since integrity of the NS instances and its topology are intimately connected. It also needs to control, which users/systems have visibility into the information of those NS instances.

As the standards advance there is an higher desire to increase the automation of the VNFs and NSs instances. The NFVO will need to encompass the functionality of managing the instances automation, in order to achieve an higher independence from the operators. This automation will be triggered via the policies defined. Policy management will be another

(43)

factor to have into account as well as the evaluation of the instances states through the use of metrics provided by the NFVI or the VNFs themselves.

VNF Manager

Here it is presented a non-exhaustive set of capabilities that should be provided by the VNFM [14].

• VNF instantiation.

• VNF instantiation feasibility checking. • VNF instance software update/upgrade. • VNF instance modification.

• VNF instance scaling.

• Collection of metrics related to VNF instances. • VNF instance healing.

• VNF instance termination.

• VNF life cycle management change notification. • VNF integrity management.

• Coordinator and information broker between the VIM and the EM.

The NFVO and the VNFM might overlap in certain functionalities and the disambiguation of where these functionalities will be implemented will vary due to the interpretation of the specification and the team which implement these functional blocks in software. From the above mentioned list, there are some functionalities which fit this description such as: VNF instantiation, VNF instantiation feasibility checking and VNF integrity management.

Apart from those functionalities, the others will be part of the VNFM. VNF instance software update/upgrade is an important functionality since it enables both infrastructure operators and VNF vendors to update/upgrade VNFs while they are running without the need to take the entire NS instance to which the VNF belongs to just for an update/upgrade. The update/upgrade will be done in place like a kernel live patch, minimizing the down time for said NS instance. Scaling is also another important functionality, where a VNF instance can be scaled in/out and up/down. Scaling out is when the number of VMs is increased in order to support a bigger load and/or reduce latency. Scaling in is the opposite of scaling out, which means the reduction of the number of VMs that compose the VNF instance. Scaling in normally occurs when the load goes down and the number of VMs is no longer cost effective. Scaling up is when the VMs that compose a VNF need to increase their processing capacity by using more CPU cores, memory, disk or networking capabilities. In order for this operation to be possible, the underlying hypervisor must support the dynamic allocation of resources when the VM is already instantiated and running. Scaling down is the opposite of the previous operation. Scaling down normally is performed when the VMs that compose the VNF instance no longer require that amount of resources and they can be released back to the NFVI.

To be able to perform scaling decisions, there needs to be data. Despite the fact that these decisions can be made manually, that is not the true goal of said functionality. Scaling operations should be made relying on data that shows that a VNF instance needs to be

(44)

scaled either because it needs more capacity or because the resources allocated to it are being wasted and can be released. This data comes from metrics either collected directly from the VNF instance or from the VIM. The VNF instance might expose directly or through the EM metrics that can be collected by the VNFM. Other metrics might be gathered directly from the VIM, since this one also contains metrics like CPU usage, memory reserved or bandwidth consumed.

Metrics are also important to determine if a VNF instance is working flawlessly or if it is an erroneous state. If it is working flawlessly, there is nothing for the VNFM to do, except keep monitoring it. When it in an erroneous state, it needs to be fixed. The functionality which the VNFM should implement in order to be able to revert the VNF instance into a good state is healing. Healing is a complex operation, since the VNFM is required to determine how to get the VNF instance back into a good state, while maintaining the integrity of NS instance. There will also be cases where that responsibility will fall onto the EM, keeping the VNFM more simple since it would only require it to trigger a healing action on the responsible EM.

VNF instance termination is a simple functionality, where the VNFM will send an action to VIM to destroy all the VMs that compose the VNF instance and if there is an EM responsible for the VNF instance, warn it that said VNF instance no longer exists.

Other components within the MANO platform need to know about the VNF instances’ life cycle events that occur. For this purpose, the VNFM is responsible for notifying the other components of life cycle events that occur with the VNF instances, so other components might act on those notifications if needed. These notifications could trigger a change in topology or an alert being sent to the OSS and/or BSS.

There needs to be a bidirectional flow of information between the EMs and the VIMs. These information flows are required in order for the EM to configure its VNF instances. The EMs should not access the VIMs directly because they are third party software and would have direct control over the NFVI, that would be a security risk. In order to minimize and control the information available to the EMs, the VNFM will act as a broker between those entities, managing the access to the information and alerts.

Virtualized Infrastructure Manager

VIMs are the functional blocks responsible by managing the available infrastructure and abstracting them for the other functional blocks. They implement an assortment of func-tionalities and might not always respect the specifications the industry sets. The next list mentions some of the provided functionalities of the VIMs as functional block in the NFV MANO platform space.

• Orchestrating the allocation/upgrade/release/reclamation of NFVI resources.

• Management of the association of the virtualized resources to the physical compute, storage and networking resources.

• Supporting the management of the VNFFGs.

(45)

• Management of the virtualized resource capacity. • Management of the software images.

• Collecting performance and fault information from the NFVI.

• Management of catalogues of virtualized resources that can be consumed from NFVI. The most important functionality that the VIM will support is the orchestration of the NFVI resources. The VIM manages three kinds of resources present in the NFVI: compute, storage and networking. It is the most important functionality because it will allocate the resources necessary for the creation of the VMs necessary to implement VNF instances that are part of the NS being instantiated. All the other operations associated with this functionality will be relevant for the maintenance of the NFVI resources and capabilities, either for the creation of new NS instances or for the scaling of existing VNF instances.

The VIM, as mentioned previously, will manage networking resources, this translates in the management of virtual links, virtual networks, subnets as well as the application of the VNFFGs present in the NS instance. This ensures that the routing and security policies inside the NS instance are fulfilled as specified in the NS Descriptor (NSD) [15]. The VNFFG is not always implemented through the VIM, there might be other entities in the NFVI that need information from the VIM in order to be able to implement the aforementioned VNFFG. The most relevant entity would be a SDN controller, since it would allow to steer the traffic in the physical networking equipment present in the NFVI.

Since the VIM manages the hardware resources in the NFVI, it will need to maintain an information repository about the hardware present in the NFVI. This information repository is needed to know if specific resource allocations can be made such has a VM in a host server in a specific Non-Uniform Memory Access (NUMA) topology or a specific CPU architecture. It should also retain the amount of used hardware resources, in order to foresee if future reservations are possible within the available resource pool.

When the VIM receives a request to allocate an amount of resources, it will also require a VM image in order to bootstrap them. The storage and management of these images should be fulfilled by the VIM and it plays an important role during the VM allocation, since it should image the virtual disk with the VM disk image supplied and boot the VM with the supplied information.

During the operation of the NFVI, it is required to collect performance and fault metrics. These metrics will allow to determinate the state of health of the physical machines in the NFVI and the VMs hosted there. It will , for example, allow to determine if any of the physical hosts is malfunctioning or the NFVI needs to be scaled up and/or out. It also provides the other functional blocks with information about the VMs’ state, this will aid in discovering if a VNF instance requires to be scaled or healed.

Last but not least, the VIM should maintain catalogues of the virtual resources that the NFVI can provide. These resources might be integrated directly into the VNF instances or the consumed in an ad-hoc way.

(46)

NS Catalogue

In the MANO platform there must a functional block that is responsible for storing and managing the NSs. This falls under the responsibility of the NS Catalogue functional block. This block is responsible for being the repository where NS are stored and must support the creation and management of the NSD, VL Descriptor (VLD) and VNFFG Descriptor (VNFFGD). The NS Catalogue must be accessible to the NFVO.

VNF Catalogue

In the MANO platform there also must be a functional block responsible for storing and managing the VNF Packages. A VNF Package is composed by a VNF Descriptor (VNFD) and other optional files, like software images, logos, scripts [16]. The functional block that has the responsibility of being the repository for on-boarded VNF Packages is the VNF Catalogue and it must be accessible to the NFVO and the VNFM because it will support the operations that they need to perform, such as validation or instantiation feasibility checking.

NFV Instances repository

All the VNF and NS instances information must be kept. To fulfil this requirement the MANO platform must have a functional block responsible with this task. The functional block responsible with this task is NFV Instances repository. It must maintain those records and update them when changes occur during the life cycle of the VNF or NS instances. This functional block supports the operations that the NFVO and VNFM must do, by providing them visibility about the NS, their constituent VNF instances and the relationship that is established between them.

NFVI Resources repository

The MANO platform should retain information about the resources avail-able/reserved/allocated in the NFVI and this information should be processed so it is not VIM specific, abstracting it from the underlying VIMs platform provided by the Infrastructure Domains.

Element Management

For a per VNF FCAPS management to be present in the MANO platform there is a need of functional block that will bridge the VNFM and the VNF instances. These functionalities shall be fulfilled by the EM. These functionalities include:

• Configuration of the VNF and the provided network function. • Fault management of the VNF and the provided network function. • VNF functions usage accounting.

• VNF functions performance measurement results collection. • VNF functions security management.

Not all the ˇvnf instances will require to have an EM managing them, it is good practice to have one, since it will ease the process of managing the VNF instances. It translates generic

(47)

calls into specific actions on the VMs that compose the VNF instances. The management functionalities provided by this functional blocks span a wide variety of topics such as configuration, fault management, accounting, performance and security.

VNFM can execute configuration actions over VNF instances, these configuration actions could either be directly or applied through the EM. In order for the VNFM to execute such an action over a VNF instance, it would require the VNFM to possess a mechanism to execute. In the case in which the EM implements the logic to configure the VNF instance, the VNFM is only required to call an HTTP REST endpoint in order to activate the configuration primitive. This simplifies the VNFM implementation since it only needs to have the ability to call an HTTP REST endpoint instead of implementing a remote configuration system to configure each VNF instance.

EM should have monitoring capabilities, in order to, capture metrics for performance and fault management purposes. Since the EM can implement this monitoring capability, it will have a better understanding of how the VNF instance is performing, if it needs to be scaled or not. Also, it can provide insight to when a VNF instance requires healing either because it is an erroneous state or it was compromised.

Usage accounting is a specific activity, instead of that business logic being implemented as part of the VNFM, it is implemented into the EM. This maintains the VNFM implementation cleaner, while the EM can fetch, aggregate and process all the information for usage accounting.

Operational Support System/Business Support System

In the NFV MANO Architectural Framework proposed by ETSI there is not explicitly an OSS or BSS component, but a connection that allows the MANO platform to be orchestrated by these software should be possible to make and it must allow for information to be exchanged between them. This connection ensures that orchestration is possible and may provide visibility into legacy systems managed or accessible by those systems.

Network Function Virtualisation Infrastructure

The NFVI is a functional block that aggregates all the hardware resources and software components provided by the VIMs, where VNFs might be deployed. The NFVI also as responsibilities over partially virtualized network functions, where part of the functionality is provided by software and the other part is provided by specialized hardware, either due to being a digital to analogue physical interface or vendor choice. Management of PNFs falls under the responsibilities of the Network Controllers (NCs).

2.4.3 Available solutions

Currently there are myriad of MANO solutions. In this subsection there is a small overview of the most relevant contenders in the NFV MANO space.

Open Source MANO

Open Source MANO or OSM as it is called by the community is an open-source MANO platform hosted by ETSI, which is being actively developed by more than a 100 companies

(48)

from Information Technologies (IT) and the telecommunications world. It was project started by Telefónica9_{, Canonical}10_{, VMware}11 _{and Rift.io}12 _{and it bases the development effort in}

being the de facto implementation that complies with the different ETSI standards. Currently, it can deploy over different VIMs, such as: Openstack, VMWare, OpenVIM13_{, OpenNebula}14

and Amazon AWS15_.

Open Network Automation Platform

Open Network Automation Platform or ONAP as it is called by the community is an open-source MANO platform hosted by the Linux Foundation Networking (LFN)16_{. ONAP is}

MANO platform that does not comply with ETSI NFV MANO specification. The project is a merger of OpenECOMP, which was an AT&T17 _{project, and the Open-Orchestrator project}

from Linux Foundation18 _{with China Mobile}19_{, Huawei}20 _{and ZTE}21 _{as lead contributors.} Openstack Tacker

Openstack Tacker is an official Openstack project. It is based in the ETSI NFV MANO specification and implements an NFVO and a VNFM. It can deploy and manage NSs and VNFs but it only works with Openstack as a VIM.

OpenBaton

OpenBaton is an open-source implementation of a MANO platform that is compliant with ETSI NFV MANO specification. It is a project developed by Fraunhofer FOKUS22 _{and TU}

Berlin23_{, while being publicly funded by European projects, such as: NUBOMEDIA}24_{, Mobile}

Cloud Networking25 _{and SoftFIRE}26_{. It is also one of the main components of the 5G Berlin}

initiative27_.

2.5 5GinFIRE

With the transformation of key industrial sectors, that are now using newer digital and communication technologies like automotive and the appearance of new sectors, such as

9 https://www.telefonica.com 10_{https://www.canonical.com} 11 https://www.vmware.com 12 https://www.riftio.com 13 https://osm.etsi.org/gitweb/?p=osm/openvim.git;a=summary 14 https://www.opennebula.org 15_{https://aws.amazon.com} 16 https://www.linuxfoundation.org/projects/networking/ 17_{https://www.att.com} 18 https://www.linuxfoundation.org 19_{https://www.chinamobileltd.com} 20 https://www.huawei.com 21 https://www.zte.com.cn 22 https://www.fokus.fraunhofer.de 23 https://www.tu-berlin.de 24_{https://www.nubomedia.eu} 25 https://www.mobile-cloud-networking.eu 26_{https://www.softfire.eu} 27 https://www.5g-berlin.org

(49)

Smart Cities, that will require and inspire a new wave of applications and services, it appears a new concept called verticals. These verticals that are built on top of common physical infrastructure and resources are becoming more open ecosystems. These vertical industries should converge technologically, so they can learn with each other experience and drive their need to develop innovative new products, services and applications. These verticals will drive the development of 5G network infrastructure and all the technologies required by them. They will be one of the stakeholders in the developments of 5G, since they will be the businesses that will need to use the resources provided to be able to supply their products, applications and services.

5GinFIRE - Evolving FIRE into a 5G-Oriented Experimental Playground for Vertical industries is a project supported by the European Commission Horizon 2020 Programme. 5GinFIRE project aspires to answer two interlinked questions:

• Q1: How such a holistic and unified environment should look like?

• Q2: How can 5GinFIRE host and integrate verticals and concurrently deal with recon-ciling their competing and opposing requirements?

To answer these two questions, 5GinFIRE proposes to build and operate an Open, and Extensible 5G NFV-based Reference (Open5G-NFV) ecosystem of Experimental Facilities that integrate existing Future Internet Research Experiments (FIRE) facilities with new ones that are vertical specific and enable experimentation of vertical industries.

To achieve these, 5GinFIRE aligns with the on-going standardization efforts, with revolving open source activities and projects, therefore allowing for an architectural and technological convergence with them. The Open5G-NFV FIRE ecosystem may serve as a playground for testing innovations. These innovations may in the future be ported to emerging "mainstream" 5G networks.

The work here presented is partly developed in the scope of the 5GinFIRE project. 2.6 Synthesis

This chapter describes what are the future problems in the world of telecommunications. It provides an overview of the predictions of how data usage will evolve and what proportions will assume.

It shows what technologies are being used to tackle the problems that the increasing amount of data will bring. It shows the technologies that enable 5G networks and how they can be used to fulfil the promises of 5G. How technologies like NFV and SDN bring agility to the deployment of new NS while bringing the costs down.

It also presents the topic of MANO, in which it enables this agility to be controlled from a single place while spanning a multitude of domains.

(50)

(51)

CHAPTER

3

5G Testbed implementation

Due to the nature of today’s world, every change needs to occur faster and in each iteration new specifications are added. This agility applies mostly to software development, where a fast pace is needed to maintain with requisites from stakeholders.

The networking ecosystem wasn’t capable to have this agility because it was stuck in the old ways, where you planned for a certain capacity and executed that plan, until there was a need to add more capacity or new features and the cycle was repeated. Since this methodology was followed, in conjunction with the vendor lock-in to their proprietary platforms, it was natural for CAPEX and OPEX to raise. There was also a need for engineers to be certified by some solution providers, which in turn raised costs even higher. The networking ecosystem needed change, to fulfil this lack of agility and to lower costs, so different approaches were introduced: SDN and NFV.

3.1 Testbed

This work is done in the context of an European project called 5GinFIRE. During this project it was planned to create a 5G testbed in each site, in order to test 5G solutions using NFV. Each testbed would then be interconnected with the others, allowing them to have different capabilities and to test distributed scenarios.

The testbed is made by two software components: OpenStack and Open Source MANO. Openstack provides the VIM capabilities and manages the NFVI, while OSM is responsible for the management and orchestration of VNFs.

The NFVI is composed by four servers, one server act as the controller and the other three provide compute resources. The controller is also responsible for storage and networking, meaning that it is the volume manager for the instantiated VMs and acting as the bridge between internal network traffic from VMs and the external networks. The compute servers where fully dedicated to hosting VMs. All the servers are regular COTS servers. Figure 3.1 presents the organization of the servers as functional nodes in Openstack. The control

(52)

node is represented by CNTRL01, while the compute nodes are represented by CMPT01,

CMPT02, CMPT03.

Figure 3.1: Testbed organization.

While the organization presented in figure 3.1 is normal for an Openstack deployment there are aspects that make this deployment unique for NFV use cases. The three different coloured lines represent different networks. The internal network is used for the internal components of Openstack to communicate with each other, it is regular to create a separate network for this kind of communication. Having this network isolated from the others protects it from getting accessed from third parties with nefarious intents. This network is also used by system administrators to access the physical nodes for maintenance purposes.

There are three networking planes relevant for this deployment: management, control and data. The control plane is important for the MANO platform to orchestrate the VIM. Openstack exposes its Representational State Transfer (REST) Application Programming Interface (API) endpoints in the control plane, so that the MANO platform, which in this case is OSM, can manage and orchestrate all the resources in the NFVI. The management plane is used by the MANO platform to configure and communicate with the VNF instances. The VNF instances must expose interfaces in this plane that are understood by the MANO platform in order to establish a connection for configuration or information retrieval. The most used communications channels are Secure Shell (SSH) and REST API endpoints. In this case the management plane and the control plane were merged into a single network, the control network, represented in figure 3.1 with a red colour.

The other networking plane is the data plane. The data plane is where the VNF instances communicate with each other and receive traffic from the external world for processing. The data plane is composed of a public network called data network, represented in figure 3.1

(53)

with a blue colour. There might also be private virtual networks in this plane. The private networks will depend what is specified in the NSDs. Private networks are mainly used for the internal communication of the VNFs or the communications between the VNFs that belong to a particular NS.

Figure 3.2: Internet connection.

In order for the VNFs to have access to the internet a router was installed (as shown in figure 3.2). This router also has two other functionalities: access control and Virtual Private Network (VPN) connection. The access control is implemented using routing policies and Access Control List (ACL) rules to control what traffic can access the networks and from where that traffic is being generated. The other functionality was a VPN connection which interconnected the testbed with other testbeds in 5GinFIRE. The VPN allows for a central OSM instance to orchestrate the instantiated VNF instances while their traffic can reach any other VNF instance, which can be located in any other testbed belonging to 5GinFIRE.