CPN simulation-based test case generation from natural language requirements

BRUNO CESAR FERREIRA SILVA

CPN SIMULATION-BASED TEST CASE GENERATION FROM

NATURAL LANGUAGE REQUIREMENTS

Universidade Federal de Pernambuco [email protected] www.cin.ufpe.br/~posgraduacao

RECIFE 2016

Bruno Cesar Ferreira Silva

CPN SIMULATION-BASED TEST CASE GENERATION FROM

NATURAL LANGUAGE REQUIREMENTS

M.Sc. Dissertation presented to the Centro de Informática of Universidade Federal de Pernambuco in partial fulfilment of the requirements for the degree of Master of Science in Computer Science.

Supervisor: Augusto Sampaio (UFPE, Brazil) Co-Supervisor: Gustavo Carvalho (UPE, Brazil)

RECIFE 2016

Catalogação na fonte

Bibliotecária Monick Raquel Silvestre da S. Portes, CRB4-1217

S586c Silva, Bruno Cesar Ferreira

CPN simulation-based test case generation from natural language requirements / Bruno Cesar Ferreira Silva. – 2016.

62 f.: il., fig., tab.

Orientador: Augusto Sampaio.

Dissertação (Mestrado) – Universidade Federal de Pernambuco. CIn, Ciência da Computação, Recife, 2016.

Inclui referências e apêndices.

1. Ciência da computação. 2. Redes de Petri. 3. Simulação de modelos. I. Silva, Bruno Cesar Ferreira (orientador). II. Título.

004 CDD (23. ed.) UFPE- MEI 2017-27

Bruno Cesar Ferreira Silva

CPN SIMULATION-BASED TEST CASE GENERATION FROM

NATURAL LANGUAGE REQUIREMENTS

Dissertação de Mestrado apresentada ao Programa de Pós-Graduação em Ciência da Computação da Universidade Federal de Pernambuco, como requisito parcial para a obtenção do título de Mestre em Ciência da Computação

Aprovado em: 05/09/2016.

BANCA EXAMINADORA

__________________________________________ Prof. Dr. Eduardo Antônio Guimarães Tavares

Centro de Informática / UFPE

__________________________________________ Prof. Dr. Jorge Cesar Abrantes de Figueiredo Departamento de Sistemas e Computação / UFCG

__________________________________________ Prof. Dr. Augusto Cezar Alves Sampaio

Centro de Informática / UFPE

I dedicate this master thesis to my mother, my wife, my children and professors who gave me the necessary support to get here.

Acknowledgements

I would like to thank some people who have contributed to the achievement of the results presented in this paper. In addition to my supervisors, other teachers of the Centro de Informática at Universidade Federal de Pernambuco provided me the necessary background in the topics covered; among them, i would like to mention Professor Eduardo Tavares, Professor Fernando Castor, and Professor Alexandre Mota. Regarding the company where I work, I would also dedicate a special thanks to Adelia Mangelli the trust and support, recognizing the importance of the work that would be developed.

The journey was undertaken would not be possible without the participation of the other two authors of this paper. I have the most sincere gratitude for my latest idols, Professor Augusto Sampaio and Professor Gustavo Carvalho. They were tireless, highly competent, gentle and human in the development of this work. I received them the necessary motivation and encouragement in the most difficult moments.

No one was so important as my family and my best friends forever. Swami Nogueira and Pompeu Jácome Lima (my dear friends) for the moments of entertainment and reflections, conveying his wise words. João Carlos and Maria Cristina (my dear father and mother-in-law) for transmitting its tranquillity.

Finally, but not least, Samanta Della Bella (my wife, buddy and support) for her under-standing, competence and sensitivity. Bruna, Stella, Gustavo and Riva (my darling children and motivation) who encouraged me to never give up. And, above all, God.

Abstract

Software Engineering faces challenges such as difficulty in understanding the user needs, am-biguous specifications, poorly defined requirements and therefore problems in interpreting the system requirements. Model-based testing (MBT) is presented as an alternative for solving these problems by using (semi-)formal methods, in the specification, modelling or analysis of requirements, as well as by automatically generating test cases.

This work proposes and implements a test generation strategy from Natural Language (NL) requirements via translation into Coloured Petri Nets (CPN), an extension of Petri Nets that supports model structuring. This strategy extends previous work on the NAT2TEST framework, which involves syntactic and semantic analyses of NL requirements and the generation of Data-Flow Reactive System (DFRS) as an intermediate representation, from which target formal models can be obtained for the purpose of test case generation. Our contributions include a systematic translation of DFRSs into CPN models, besides a strategy for test generation. We illustrate our overall approach with a running example.

Therefore, this work presents a variant for the NATural Language Requirements to TEST Cases (NAT2TEST) strategy, in which the Coloured Petri Nets (CPN) is used as an intermediate model. The NAT2TEST strategy, which is applicable to discrete or continuous systems, consists of five phases: syntactic and semantic analyses, DFRS generation, CPN generation, and generation of test cases. The approach proposed here, which is based on Petri Nets simulation, has as benefit the maturity of the theory and tools related to CPN. Moreover, there are available resources for analysing structural and behavioural properties of the model. The process is automated by the NAT2TEST tool until the DFRS generation. The model translation from the DFRS to the CPN is automated by Spoofax framework. Finally, the test cases generation occurs automatically via simulations held in the CPN Tools.

Our strategy was evaluated considering examples from the literature (Vending Machine and Nuclear Power Plant) and the aerospace industry (Priority Control). We analysed performance and the ability to detect defects generated via mutation. In general, our strategy outperformed the considered baseline: random testing. We also compared our strategy with the CSP version.

Keywords: Model-based testing. Controlled natural language. Data-flow reactive system. Automatic test generation. Coloured Petri nets. Simulation of models.

Resumo

A Engenharia de Software possui desafios clássicos como dificuldade no entendimento das neces-sidades dos usuários, especificações ambíguas, requisitos mal definidos e, portanto, problemas na interpretação destes. A utilização de testes baseados em modelos (MBT) apresenta-se como alternativa para solução destes problemas, através do uso de métodos (semi-)formais, seja na especificação, modelagem ou análises de requisitos, bem como na geração automática de casos de testes.

Este trabalho propõe e implementa uma estratégia de geração de testes a partir de requisitos escritos em linguagem natural (NL) através da tradução para modelos em Redes de Petri Colorida (CPN), uma extensão de Redes de Petri que incorpora estruturação de modelos. Esta estratégia estende um trabalho anterior (NAT2TEST framework), que envolve análises sintática e semântica de requisitos em linguagem natural (NL) e geração do modelo de sistemas reativos baseados em fluxos de dados (DFRS) como uma representação intermediária, a partir do qual outros modelos formais podem ser obtidos com o propósito de geração de casos de testes. Nossa contribuição inclui uma tradução sistemática de DFRSs para modelos CPN, assim como uma estratégia para geração de testes. Ilustramos nossa abordagem através de um exemplo prático.

Assim sendo, este trabalho apresenta uma variante da estratégia NAT2TEST, na qual formalismo intermediário é Redes de Petri Colorida (CPN), sendo aplicável a sistemas discretos e contínuos, e que consiste de cinco etapas: análises sintática e semântica, gerações dos modelos DFRS e CPN e de casos de testes. A abordagem empregada, através da simulação de Redes de Petri, tem como benefícios a maturidade da teoria e das ferramentas associadas a CPN, além de permitir a análise de propriedades estruturais e comportamentais do modelo. A ferramenta NAT2TEST já automatiza a tradução de requisitos em linguagem natural na notação do DFRS. A tradução do modelo DFRS para o formalismo CPN é uma primeira contribuição do presente trabalho e foi automatizada através do ambiente Spoofax. A geração dos casos de testes foi desenvolvida, de forma automatizada, através de simulações realizadas na ferramenta CPN Tools.

A estratégia aqui proposta foi avaliada considerando exemplos da literatura (Vending Machine (VM) e Nuclear Power Plant (NPP)) e da indústria aeroespacial (Priority Control (PC)). Foram analisados o desempenho e a capacidade de detectar defeitos gerados através de operadores de mutação. Em geral, a nossa estratégia apresentou melhores resultados do que a referência adotada: testes aleatórios. A estratégia também foi comparada com a versão que utiliza Communicating Sequential Processes (CSP) como modelo formal intermediário e apresentou melhor desempenho nos três estudos realizados. Em um deles, encontrou a mesma quantidade de defeitos, sendo superior nos demais.

reativos baseados em fluxos de dados. Geração automática de testes. Redes de Petri coloridas. Simulação de modelos.

List of Figures

2.1 The VM abstract representation . . . 20

2.2 Example of VM signals . . . 20

2.3 NAT2TEST strategy . . . 21

2.4 An example of a Coloured Petri Net (CPN) . . . 26

3.1 Page created to represent the requirement REQ001 . . . 31

3.2 Part of the eXtensible Markup Language (XML) representation of the DFRS for the VM example . . . 34

3.3 DFRS grammar . . . 35

3.4 The rule create-cpn-inputs . . . 36

3.5 The rule page-reqx . . . 37

4.1 Page created to generate the inputs randomly . . . 39

4.2 Page created to deal with no system reaction . . . 40

4.3 Page created to save the test cases in a Comma-separated values (CSV) file . . 41

4.4 Page created to represent the requirement REQ002 . . . 42

4.5 Generating random inputs . . . 43

List of Tables

2.1 Example of a requirement frame for REQ002 (Vending Machine) . . . 22

2.2 Example of test case for REQ002 (Vending Machine) . . . 23

2.3 The variables and types of DFRS for the VM example . . . 23

2.4 The functions of DFRS for the VM example . . . 24

4.1 Example of a test case for the adapted Vending Machine(VM) . . . 45

5.1 Perfomance Analysis . . . 47

5.2 Mutant Analysis . . . 49 6.1 Related work –– modelling timed-systems from NL requirements using CPNs . 53

List of Acronyms

ACT Action . . . 22

AGT Agent . . . 22

AST Abstract Syntax Tree . . . 35

CAC Condition Action . . . 22

CFV Condition From Value . . . 22

CMD Condition Modifier . . . 22

CPN Coloured Petri Nets . . . 17

CPT Condition Patient . . . 22

CSP Communicating Sequential Processes . . . 16

csptio CSP timed input-output conformance relation . . . 53

CSV Comma-separated values . . . 37

CTV Condition To Value . . . 22

DFRS Data-Flow Reactive System . . . 16

DSL Domain-Specific Language . . . 28

DTD Document Type Definition . . . 35

IMR Intermediate Model Representation . . . 16

MBT Model-based testing . . . 15

NAT2TEST NATural Language Requirements to TEST Cases . . . 16

NL Natural Language . . . 6

NLP Natural Language Processing . . . 17

NPP Nuclear Power Plant . . . 49

OCS Output Colour Set . . . 32

PAT Patient . . . 22

PC Priority Control . . . 7

SUT System Under Test . . . 15

SCR Software Cost Reduction . . . 16

SysReq-CNL System Requirements Controlled Natural Language . . . 21

VM Vending Machine . . . 21 XML eXtensible Markup Language . . . 30

Contents

1 Introduction 15

1.1 Research question . . . 17

1.2 Main contributions . . . 17

1.3 Organization of the dissertation . . . 18

2 Background 19 2.1 NAT2TEST strategy . . . 20

2.2 Data-flow reactive systems . . . 23

2.3 Coloured Petri nets . . . 24

2.4 Spoofax framework . . . 27

3 Translating from DFRS to CPN 30 3.1 CPN models . . . 30

3.1.1 Representing input variables . . . 30

3.1.2 Representing output variables . . . 31

3.1.3 Representing functions . . . 32

3.2 Systematic translation . . . 33

3.2.1 Defining a DFRS grammar . . . 33

3.2.2 Creating transformation rules . . . 35

4 Test vectors based on CPN simulation 38 4.1 Auxiliary components . . . 38

4.1.1 Generating entries . . . 38

4.1.2 Dealing with no system reaction . . . 39

4.1.3 Creating a test case file . . . 40

4.1.4 Repeated places . . . 41

4.2 Test vectors generation . . . 41

4.2.1 Random inputs generation . . . 42

4.2.2 System reaction . . . 43

4.2.3 Test vectors . . . 43

5 Empirical analysis 46 5.1 Performance analysis . . . 46

6 Conclusion 51 6.1 Related work . . . 51 6.2 Future work . . . 53 References 55 Appendix 57 A List of requirements 58

A.1 Vending machine . . . 58 A.2 Nuclear power plant . . . 58 A.3 Priority command . . . 59

15 15 15

1

Introduction

Since the so-called software crisis (in the sixties), when the term software engineering was conceived, difficulty on understanding the user needs, ambiguous specifications, poorly specified and interpreted requirements are still common issues. In the field of requirements engineering, several studies have been conducted focusing on the use of (semi-)formal methods to specify, model and analyse requirements, besides automatically generating test cases, resulting in greater maturity and in the construction of underlying theories and supporting tools.

Some models can be suitable to prevent ambiguous, missing, contradictory, incorrect, obscured, incomplete requirements. Thus, some techniques, by using models, are applied to help improve the quality of systems. Model-based testing (MBT) is an approach based on creating test cases derived from a behaviour model of the System Under Test (SUT). This model describes the expected behaviour of the test object. These test cases are then, where possible, automatically generated from the test object. The challenge with this approach lies creating a formal behaviour model in which the system is represented. According to the Dutch test association TestNet1, MBT is:

 "Software testing in which a test model is created that describes some of the expected behaviour (usually functional) of the test object. The purpose of creating this test model is to review the requirements thoroughly and/or to derive test cases in whole or in part from the test model. The test models are derived from the requirements or designs."

Despite the benefits of formal MBT, those who are not familiar with the models syntax and semantics may be reluctant to adopt theses formalisms. Moreover, most of these models are not available in the very beginning of the project, when usually natural-language requirements are available. The model notations may be not easy to interpret by aerospace and automotive development engineers. Hence, a specialist (usually mathematicians, logicians, computer engi-neers and scientists) is required when such languages, and their corresponding techniques, are used in business contexts.

16 One possible alternative to overcome these limitations is to employ Natural Language Processing (NLP) techniques to derive the required models from natural-language specifica-tions automatically. Using NLP techniques to derive formal models from natural-language requirements, besides applying MBT techniques, one can reason formally about properties of specifications that can be difficult to analyse by means of manual inspection, such as inconsis-tency and incompleteness. A complete NLP system usually counts on five processing levels, depending on its aim: morphological analysis, syntactic analysis, semantic mapping, discourse analysis and pragmatic analysis.

Concerning MBT, the works ((BODDU et al.,2004), (SNEED,2007), (SANTIAGO

JU-NIOR; VIJAYKUMAR,2012)) address the joint use of MBT and NLP based on the second and

third NLP levels cited before. There is a trade-off concerning the application of NLP in MBT. Some studies are able to analyse a broad range of sentences, whereas others rely on controlled versions of natural language. The works that adopt the former approach usually depend on a higher level of user intervention to derive models and to generate test cases. Differently, the restrictions imposed by a Controlled Natural Language (CNL) might allow a more automatic approach when generating models and test cases. Ideally, a compromise between these two possi-bilities should be sought to provide a useful degree of automation along with a natural-language specification feasible to be used in practice.

As formal methods can be used to find errors in requirements, such as inconsistency and incompleteness, they play an important role in the earlier discovery of errors and, thus, reducing costs and correction efforts (MYERS; SANDLER; BADGETT,2004). In this light, the strategy NATural Language Requirements to TEST Cases (NAT2TEST) (CARVALHO et al.,2015) was devised to generate test cases from natural language requirements. In this strategy, the system behaviour is formally represented as a Data-Flow Reactive System (DFRS). Then, this model is translated into a target formalism for generating test cases. The purpose of the NAT2TEST strategy is to be easily extensible to various target formalisms. The use of Communicating Sequential Processes (CSP) (CARVALHO; SAMPAIO; MOTA,2013), Software Cost Reduction (SCR) (CARVALHO et al.,2014) and Intermediate Model Representation (IMR) (CARVALHO

et al.,2014) have been explored, considering their particular advantages, but also tailored to

their limitations. For instance, test generation from SCR and IMR is based on commercial tools, which is a practical advantage, but, on the other hand, is not formal in the sense that no explicit conformance relations are adopted. On the other hand, the test strategy for CSP is formal and can be proved sound, with an explicit conformance notion, and the FDR tool2is used to generate test cases as counterexamples of refinement verifications. However, test cases are obtained via refinement checking, demanding the complete expansion of the underlying label transition system, which can easily lead to state explosion problems. Concerning the target languages, from which test cases are generated, a comparison is out of the scope of the current work. The purpose here is only to emphasise that models in several different notations can be generated from a

1.1. RESEARCH QUESTION 17 DFRS. It would be possible to define a testing theory for the DFRS formalism, but this is not the purpose of the NAT2TEST framework. The idea is use DFRSs just as an intermediate notation, and allow the translation into several target formalisms with their specific testing theories and tool support.

Here we explore the use of Coloured Petri Nets (CPN) as an alternative extension of the NAT2TEST strategy where simulation of CPNs is used for generating test cases. To simulate a CPN model it is not necessary to explore its complete state space first and, thus, we minimise state space explosion problems. Also, we benefit from the diversity and maturity of CPN tools3. A testing theory based on CPN can also be entirely formal; although our focus here is on developing a first testing strategy that still needs to be proved sound taking into account a suitable conformance relation.

1.1

Research question

In the light of the discussion presented so far, the main research question of this work is: how to efficiently simulate, and generate test cases for timed reactive systems from natural-language requirements? In particular, regarding systems whose behaviour can be described via data operations and that is time-dependent.

Here, we propose a variant for the formal MBT strategy (NAT2TEST) for generat-ing test cases from natural-language requirements usgenerat-ing the formalism (CPN) via simulation. NAT2TEST strategy dispense the need to know the syntax and semantics of the underlying notations, besides allowing early use of MBT, by means of Natural Language Processing (NLP). Nevertheless, the state explosion problem easily occurs in CSP version, therefore, our approach using CPN simulation, where it is not necessary to explore its complete state space, minimises this problem.

1.2

Main contributions

The main contributions of this paper are the following:

 A systematic translation of DFRSs into CPN models, creating an alternative version of the MBT strategy (NAT2TEST) for generating test cases using the formalism (CPN) via simulation;

 An improvement of the performance and detection of defects with respect to the strategy via CSP;

We illustrate our overall approach with a running example.

1.3. ORGANIZATION OF THE DISSERTATION 18

1.3

Organization of the dissertation

 Chapter 2 introduces the NAT2TEST strategy, Data-Flow Reactive Systems, Coloured Petri Nets, CPN Tools, and the Spoofax framework.

 Chapter 3 explains how to generate CPN models from DFRSs.

 Chapter 4 describes how to generate test vectors from CPN models via simulation.  Chapter 5 presents an empirical analysis.

19 19 19

2

Background

Here, we briefly introduce the concepts and tools used in this work. Section 2.1 presents more details about the NAT2TEST strategy and its constituent phases, in particular, its underlying model DFRS (Section 2.2). Section 2.3 explains CPNs, the target formalism of this work, and briefly describes the support tool for simulating CPN models by going through some of its constituent components. Finally, Section 2.4 describes the Spoofax framework, which is used here to develop a systematic translation from DFRSs to CPN models. As a running example, we consider a Vending Machine (VM), which is an adaptation of the coffee machine presented

inLARSEN K.; MIKUCIONIS(2004).

Runnig example From the initial state (idle), the VM goes to the choice state when it receives a coin (REQ001). If the coffee request button is pressed, the state changes to the weak, provided the selection occurs within 30 seconds after inserting the coin (REQ002). Other-wise, the system changes to the strong state (REQ003). The time to produce a weak coffee (REQ004) is different from the time required to produce a strong one (REQ005). After producing the coffee, the VM goes to the idle state.

As shown in Figure 2.1, in the VM example we have two input signals related to the coin sensor (sensor) and the coffee request button (request). A true value means that a coin was inserted and the coffee request button was pressed. There are two output signals related to the system mode (mode) and the vending machine output (output). The values communicated by these signals reflect the system possible states (idle, choice, weak, strong, and reset) and the possible outputs (undefined, weak, and strong).

The VM has just one timer: the request timer, which is used to register the moments when a coin is inserted, when the coffee request button is pressed, and when the coffee is produced. Figure 2.2 illustrates a scenario where there is continuous observation of the input and output signals. If we had chosen to observe the system discretely, we would have a similar scenario, but with a discrete number of samples over time.

In Figure 2.2, a coin is inserted 2 seconds after starting the vending machine (the signal sensorchanges to 1 —- true). Immediately, the system state changes from idle to choice. Here,

2.1. NAT2TEST STRATEGY 20

Figure 2.1: The VM abstract representation

the system states are encoded as follows: idle 7→ 1, choice 7→ 0, weak 7→ 3, strong 7→ 2, and reset7→ 4. Therefore, this change is represented by changing the value of the signal mode from 1 to 0. In this example, the signal sensor remains true for 3 seconds.

When 10 seconds have elapsed since the coin was inserted, which happens 2 seconds after starting the vending machine, the user requests a coffee (the signal request becomes true when the system global clock is equal to 12). At this moment, the system state changes to weakcoffee (the signal mode becomes 3). In this example, the signal request remains true for 4 seconds.

As the coffee request occurs within 30 seconds of the coin being inserted, the system produces a weak coffee, which is represented as the value 2 of the signal output, 20 seconds after receiving the coffee request. We recall that a weak coffee is produced within 10 and 30 seconds after the coffee request. Then, as stated by the requirements, the system goes to the reset state (the value of signal mode becomes 4), and 3 seconds later it goes back to the idle state, besides resetting the output to undefined (the signal output becomes 1).

Figure 2.2: Example of VM signals

2.1

NAT2TEST strategy

In this section we briefly describe the NATural Language Requirements to TEST Cases (NAT2TEST) strategy for generating test cases from requirements written in natural language (CARVALHO et al., 2015). It is tailored to generate tests for Data-Flow Reactive Systems

2.1. NAT2TEST STRATEGY 21 (DFRS): a class of embedded systems whose inputs and outputs are always available as digital signals. In particular, regarding systems whose behaviour can be described by data operations and that is time-dependent. The input signals can be seen as data provided by sensors, whereas the output data are provided to system actuators.

It receives, as input, requirements (Example 2.1) written using the System Requirements Controlled Natural Language (SysReq-CNL), specially tailored for writing unambiguous re-quirements of data-flow reactive systems (CARVALHO,2016). As output, it produces test cases. This test-generation strategy comprises a number of phases (see Figure 2.3). The three initial phases are fixed: (1) syntactic analysis, (2) semantic analysis, and (3) DFRS generation; the remaining phases depend on the target formalism from which test cases are generated. Some variations have been explored (SCR (CARVALHO et al.,2014), IMR (CARVALHO et al.,2014) and CSP (CARVALHO; SAMPAIO; MOTA,2013)). As already mentioned, we explore the use of CPN.

Example 2.1 This requirement (Vending Machine (VM)) adheres to the SysReq-CNL grammar.  REQ002 – When the system mode is choice , and the coin sensor is false, and the

coin sensor was false, and the coffee request button changes to pressed, and the request timer is lower than or equal to 30.0, the coffee machine system shall: reset the request timer, assign preparing weak coffee to the system mode.

Figure 2.3: NAT2TEST strategy

The syntactic analysis phase receives as input the system requirements, and performs two tasks: it verifies whether these requirements are in accordance with the SysReq-CNL grammar,

2.1. NAT2TEST STRATEGY 22 Table 2.1: Example of a requirement frame for REQ002 (Vending Machine)

Condition 1 - Main Verb (CAC): is

CPT: the system mode CFV:

-CMD: - CTV: choice

Condition 2 - Main Verb (CAC): is

CPT: the coin sensor CFV:

-CMD: - CTV: false

Condition 3 - Main Verb (CAC): was

CPT: the coin sensor CFV:

-CMD: - CTV: false

Condition 4 - Main Verb (CAC): changes

CPT: the coffee request button CFV:

-CMD: - CTV: pressed

Condition 5 - Main Verb (CAC): is

CPT: the request timer CFV:

-CMD: lower than or equal to CTV: 30.0 Action 1 - Main Verb (ACT): reset

AGT: the coffee machine system TOV: -PAT: the request timer

Action 2 - Main Verb (ACT): assign

AGT: the coffee machine system TOV: preparing weak coffee PAT: the system mode

besides generating syntactic trees for each correctly edited requirement. The second phase maps these syntax trees into an informal NL semantic representation, where thematic roles are identified. As one can see in Table 2.1, the requirement frame (collection of thematic roles) of the example 2.1 contains five conditions and two actions. The roles that appear in actions are the following: Action (ACT) — the action performed if the conditions are satisfied; Agent (AGT) — entity who performs the action; Patient (PAT) — entity who is affected by the action; and To Value (TOV) — the patient value after action completion. In conditions, the roles are: Condition Action (CAC) — the action that concerns each condition; Condition Patient (CPT) — the entity related to each condition; Condition From Value (CFV) — the CPT previous value; Condition Modifier (CMD) — a modifier related to the condition; and Condition To Value (CTV) — the value satisfying the condition.

The third phase (DFRS generation) derives an intermediate formal characterization of the system behaviour (s-DFRS) for the requirement frames. Other formal notations can be derived form the DFRS. The possibility of exploring different formal notations allows analyses from several perspectives, using different languages and tools, besides making the strategy extensible; as discussed previously, the target formalism in the present work is CPN, which is obtained via an systematic translation from DFRS. These first three phases are the same as the MBT strategy NAT2TESTCARVALHO et al.(2015), and the NAT2TESTCPN can be considered as an

2.2. DATA-FLOW REACTIVE SYSTEMS 23 specialisation of the NAT2TEST framework.

In the next phase (CPN generation), the s-DFRS is translated to the CPN model using the Spoofax framework. This systematic translation involves defining an SDF3 grammar for DFRS models and transformation rules from DFRS to CPN. Finally, the test cases are generated by simulating the CPN model via the CPN Tools. Table 2.2 shows part of a concrete test case obtained from the previous requirement (REQ002).

Table 2.2: Example of test case for REQ002 (Vending Machine)

TIME request coin system mode output Trace

0 false false idle strong

18 false true choice strong REQ001

56 false false choice strong

72 true false preparing_weak_coffee strong REQ002

2.2

Data-flow reactive systems

A DFRS model (CARVALHO et al., 2014) provides a formal representation of the requirements semantics. It has a symbolic and an expanded representation. Briefly, the symbolic version is a 6-tuple: (I, O, T, gcvar, sO, F). Inputs (I) and outputs (O) are system variables,

whereas timers (T) are used to model temporal behaviour. These three sets (I, O and T) are disjoint. There is a global clock denoted as gcvar. The element s0is the initial state. The last

element (F) represents a set of functions, each one describing the behaviour of one system component. The expanded DFRS comprises a (possibly infinite) set of states, and a transition relation between states. This expanded representation is built by applying the elements of F to the initial state to define function transitions, but also letting the time evolve to define delay transitions.

Table 2.3: The variables and types of DFRS for the VM example

Kind Name Type Expected Values Initial Value

INPUT the_coffee_request_button BOOLEAN {false, true} false

INPUT the_coin_sensor BOOLEAN {false, true} false

{choice, idle,

OUTPUT the_system_mode INTEGER preparing strong coffee, choice

preparing_weak_coffee}

OUTPUT the_coffee_machine_output INTEGER {strong, weak} strong

TIMER the_request_timer INTEGER - 0

GLOBAL CLOCK gc INTEGER - 0

Tables 2.3 and 2.4 illustrate a DFRS for an adapted version of the Vending Machine (VM) (LARSEN K.; MIKUCIONIS,2004). From the initial state (idle), it goes to the choice state when it receives a coin. If the coffee request button is pressed, the state changes to the weak

2.3. COLOURED PETRI NETS 24 Table 2.4: The functions of DFRS for the VM example

Static Guard Timed Guard Statements Trace Functions: the_coffee_machine_system

¬(prev(the_coin_sensor) = true) the_request_timer := gc, REQ001 AND the_coin_sensor = true the_system_mode := 0

AND the_system_mode = 1

¬(prev(the_coffee_request_button)=true)

AND the_coffee_request_button = true the_request_timer := gc, REQ002 AND prev(the_coin_sensor) = false gc - the_request_timer<=30 the_system_mode := 3

AND the_coin_sensor = false AND the_system_mode = 0

¬(prev(the_coffee_request_button)=true)

AND the_coffee_request_button = true the_request_timer := gc, REQ003 AND prev(the_coin_sensor) = false gc - the_request_timer>30 the_system_mode := 2

AND the_coin_sensor = false AND the_system_mode = 0

state, provided the selection occurs within 30 seconds after inserting the coin. Otherwise, the system changes to the strong state. After producing the coffee, it goes to the idle state. In Table 2.4, one can note that the corresponding DFRS has two input (the_coin_sensor and the_coffee_request_button) and two output (the_system_mode and the_coffee_machine_output) variables. The system behaviour is formalised by the function the_coffee_machine_system. It has five entries. Each entry comprises a 3-tuple: a static guard, a timed guard, and the expected system reaction (a list of assignments).

2.3

Coloured Petri nets

Coloured Petri Nets (CPN) is a language for the modelling and validation of systems in which concurrency, communication, and synchronisation play a major role. It is a discrete-event modelling language combining high-level Petri nets with an extension of the functional programming language Standard ML (CPN ML).

CPN ML provides the primitives for the definition of data types, describing data manip-ulation, and for creating compact and parametrised models. A CPN model of a system is an executable model representing the states of the system and the events (transitions) that can cause the system to change state. The CPN language makes it possible to organise a model as a set of modules, and it includes a time concept for representing the time taken to execute events in the modelled system.

As described in (JENSEN,1996), the formal definition of a CPN is given by a 9-tuple (Σ, P, T, A, N, C, G, E, I), satisfying the following properties:

 Σ is a finite set of non-empty colour sets (types).  Pis the finite set of all places.

2.3. COLOURED PETRI NETS 25

 Ais the finite set of all arcs such that P ∩ T = P ∩ A = T ∩ A = /0  N is the node function such that N : A → P × T ∪ T × P

 Cis the colour function such that C : P → Σ

 Gis the guard expression function such that G : T → Expr, and ∀t ∈ T : [Type(G(t)) = Bool ∧ Type(Var(G(t))) ⊆ Σ]

 Eis the arc expression function such that E : A → Expr, and ∀a ∈ A : [Type(E(a)) = C(p(a))MS∧ Type(Var(E(a))) ⊆ Σ],

where p(a) is the place of N(a);

 I is the initialization function such that I : P → Expr, and ∀p ∈ P : [Type(I(p)) = C(p)MS]

We denote by Expr the set of expressions provided by the inscription language (CPN ML), and by Type(e) the type of an expression e ∈ Expr, i.e., the type of Var(e) (the values obtained when evaluating e). The subscript MS denotes the multiset of the associated place.

Informally, a CPN model describes both the states and the events of a system. Events are represented by transitions (drawn as rectangles) and the states are represented by places (drawn as ellipses or circles). These two kinds of nodes are connected by arcs that indicate the direction in which tokens (data values) are moved. Arcs are used to connect places with transitions, and vice-versa, but never two places or two transitions. Places hold collections of tokens, and thus represent local states (markings). The global state (or global marking) of a model is represented by the distribution of tokens throughout its places. The places also have an initial marking representing the initial local state. Figure 2.4 shows part of the CPN obtained for the VM example (explained in the beginning of this chapter and illustrated in tables 2.3 and 2.4). In this illustration, we have part of the CPN that models the requirement REQ005—returning to the idle state after producing coffee

As described in (TJELL,2006), a transition (see the rectangle REQ005 in Figure 2.4) typically represents an event that occurs in the model. When this event occurs, the transition is enabled to fire. When a transition is enabled it means that it is ready to fire. When a transition fires, it is able to remove tokens from its input places and to produce new tokens on its output places, in a atomic action. The inscriptions in the arcs leading to a transition define the quantity of tokens needed for enabling the transition. Moreover, qualitative constraints can be specified for the tokens from the input places. This can be done by the use of pattern matching or guards in the transition.

A guard (see the text below REQx in Figure 2.4) is a predicate that must evaluate to true for enabling the transition. For a given transition, if there is an available set of tokens from the input places (in Figure 2.4 all places are input to the transition REQ005) while satisfying these constraints, this transition is enabled. If another transition also depends on some of the same

2.3. COLOURED PETRI NETS 26

Figure 2.4: An example of a Coloured Petri Net (CPN)

tokens for being enabled, the transitions are said to be in conflict. This property is part of what makes (Coloured) Petri Nets a very strong tool for modelling distributed and concurrent systems with shared resources, parallel processes and other characteristics that come with this type of systems.

In Figure 2.4, the first restriction to enable the transition REQ005 is that there are tokens in the places Input, F_the_request_timer, F_the_system_mode, and F_the_coffee_machine_output. While there is some input to be processed, there will be tokens in Input. The remaining places will always have tokens because the tokens will ever be maintained or replaced by another when the transition REQ005 fires. The tokens in the place Input are only consumed when the transition noReqfires in the page NoAttendedReq. So, one can note that there is a bidirection arc from the place Input to the transition REQ005, unlike other places in the page REQ005. The other restriction is related to the guard condition that must be satisfied.

The graphical definition of CPNs is supplemented by declarations of functions, opera-tions, constants, variables, colour sets and priorities; all of them in the functional programming language CPN ML (JENSEN; KRISTENSEN, 2009): an extension of the more commonly known Standard ML (MILNER; HARPER; TOFTE,1990). Therefore, CPN models differ from low-level Petri Nets(such as Place/Transition Nets) by the fact that the tokens have types (colour sets) and values, and by the combination of programming languages.

The CPN modelling language also supports the specification of hierarchically structured models as a collection of connected modules (or pages). This makes it possible to work with different levels of abstraction, reuse of sub-modules in different parts of the model and

2.4. SPOOFAX FRAMEWORK 27 the construction of compact and parametrised models (JENSEN; KRISTENSEN,2009). A module is itself a CPN model (possibly hierarchical too). Structuring is performed through two mechanisms: fusion places or substitution transitions. Here, we use the former. A fusion place is a set containing multiple places that may be found in different modules. Fusion sets are created for places that are repeated in more than one module (CPN page). This allows interaction to traverse boundaries of the modules in the model. For instance, in Figure 2.4, the place Trace is contained in all pages of the VM model, except in Generator, and TestCase, so the fusion set Traces is associated to all the places Trace (see a thin rectangle at the bottom of this place in Figure 2.4). Other fusion sets that exist in this figure are Inputs, F_the_request_timer, F_the_system_mode, and F_the_coffee_machine_output.

CPN also allows the specification of time-based behaviour. This is accomplished by adding time stamps to tokens. In this way, delays are expressed as timing annotations in arcs and transitions. The firing of transitions is still an atomic event, but the calculation of enabling conditions for a given transition depends on the time stamps in the tokens it needs to consume through possible input arcs. Intuitively, the time stamp in a token can be seen as a specification of the model time at which the token is available for consumption from a place.

CPN Tools1is an industrial-strength computer tool suite for editing, simulation, state space analysis, and performance analysis of untimed and timed, hierarchical Coloured Petri nets (CPN or CP-nets). Basic functions of CPN Tools consist in creation (editing) of model, analysis of models behaviour via its simulation, and creation and analysis of model’s state space (full and partial). Using CPN Tools, it is possible to investigate the behaviour of the modelled system using simulation, to verify properties by means of state space methods and model checking, and to conduct simulation-based performance analysis. User interaction with CPN Tools is based on direct manipulation of the graphical representation of the CPN model using interaction techniques, such as tool palettes and marking menus.

The tool features incremental syntax checking and code generation, which take place while a net is being constructed. A fast simulator efficiently handles untimed and timed nets. The state space resource produces report containing information, such as boundedness and liveness properties. The tool allows manual simulations or several replications.

2.4

Spoofax framework

Spoofax2(KATS; VISSER,2010) is a platform for the development of textual domain-specific languages. It is an Eclipse-based language workbench that uses the modular syntax definition formalism SDF3, which supports language composition. Spoofax provides a compre-hensive environment that integrates syntax definition, program transformation, code generation, and declarative specification of IDE components. Actually, there are meta-languages for defining

1_{http://cpntools.org/}

2.4. SPOOFAX FRAMEWORK 28 languages in Spoofax: a syntax definition formalism (SDF3), a name binding language (NaBL), a type specification language (TS), and a transformation language (Stratego), as also the Spoofax Testing Language (SPT). In what follows, we present an overview of the resources used in this work, based on (VISSER,2015), where a more comprehensive explanation is provided.

The SDF language describes a grammar, an algebraic signature, and a mapping from parse trees to abstract syntax trees at once. Indeed, from just an SDF3 syntax definition, one can generate the abstract syntax signature, error recovery rules, a parser, a mapping from parse trees to abstract syntax trees, a pretty-printer (mapping from abstract syntax trees to text), syntax highlighting rules, and folding rules. Therefore, an SDF3 definition is a declarative, integrated, and modular description of all aspects of the syntax of a language, including its lexical syntax.

Using the high-level SDF grammar formalism, one can write the language grammar. Based on this grammar, basic editor services such as syntax highlighting and code folding are automatically provided. Using high-level descriptor languages, these services can be customised. More sophisticated services such as error marking and content completion can be specified using rewrite rules in the Stratego language. In Example 2.2, it is defined a SDF3 grammar for a Domain-Specific Language (DSL), whose programs in that language have a name, then define a list of zero or more classes, and finally an expression to be executed in the context of these class definitions. The module imports the syntax of Classes and Expressions and defines a context-free syntax productionto define the syntax of programs.

Example 2.2 A short example of an SDF3 definition (VISSER,2015). module ProgramsPlain

imports Classes Expressions sorts Program

context-free syntax

Program.Program = [program [ID] [Class*] run [Expr]]

An SDF3 definition defines both the concrete syntax and abstract syntax views of a language. The following is a minimal program in the concrete syntax of the language that executes the expression 1 in the context of no classes:

program program01 run 1

Here is the same program in abstract syntax representation: Program("program01", [], Num("1"))

Thus, the Program production is equivalent to the context-free grammar production: Program = "program" Class* "run" Expr

defining the concrete syntax notation of programs. By the way, the * in Class* is the Kleene-star and denotes a sequence of zero or more strings matching the Class non-terminal sort.

2.4. SPOOFAX FRAMEWORK 29 The Stratego language provides rewriting rules for expressing basic transformations, programmable rewriting strategies for controlling the application of rules, concrete syntax for expressing the patterns of rules in the syntax of the object language, and dynamic rewrite rules for expressing context-sensitive transformations, thus supporting the development of transformation components at a high level of abstraction. In Example 2.3, the rule to-xml translates a program from the language defined above to XML format by exploring the abstract tree. The abstract term Program is decomposed via string interpolation into the XML tags. The stratego library defines the dbg(|msg) strategy, which prints the current term prefixed with msg, and the strategy map(s), which applies s to all the elements of a list (cs). The strategy combinator <+ is a ordered choice between two strategies. If the application of the primer is successful, its result is returned, otherwise the latter is applied to the original term.

Example 2.3 Translating programs to XML format using string interpolation (VISSER,2015). module to-xml imports src-gen/signatures/Programs-sig imports to-xml/classes-to-xml imports to-xml/expressions-to-xml rules to-xml : Program(name, cs, e) -> $[<program name="[name]"> <classes> [<map(to-xml <+ dbg(|"class: "))> cs] </classes> <run>

[<to-xml <+ dbg(|"exp: ")>e] </run>

</program>]

The translation from the DFRS syntax to the CPN format in Spoofax involves defining an SDF3 grammar for DFRS and transformation rules from DFRS to CPN. Therefore, via the SDF3 language, it is created a domain specific language in Spoofax. The main advantage of using this kind of platform is the systematisation of the translation process from DFRS, allowing other intermediate formalisms, besides Petri nets, being considered with minimal effort.

In summary, for generating testes cases, we use the tool NAT2TEST to write system requirements in SysReq-CNL and generate a DFRS model; from this model in XML format, we use the Spoofax framework to translate to the CPN model. By CPN model simulations using CPN Tools, we generate the test cases.

30 30 30

3

Translating from DFRS to CPN

In this chapter we explain how DFRSs, encoded as eXtensible Markup Language (XML) files, are systematically translated to CPN models, in accordance to the CPN format accepted by CPN tools. In Section 3.1 we explain how DFRSs are represented as CPN models. Afterwards, in Section 3.2 we describe how to perform this translation in a systematic way: first, proposing a grammar for DFRSs (Section 3.2.1), then defining translation rules in Spoofax (Section 3.2.2).

3.1

CPN models

Intuitively, the behaviour of a DFRS is encoded as transitions that modify the correspond-ing output variables, which are modelled as CPN places (see Section 2.3). Therefore, the target of these transitions are the output places, and their source are auxiliary places that provide to the transitions the system inputs, along with the output and timer places. The generation of CPN models from DFRSs is accomplished in three steps, as explained below.

So, for representing a DFRS as a CPN model, firstly DFRS input variables, and their types, are translated to CPN variables and colour sets (Section 3.1.1), respectively. In the following step, DFRS output variables are mapped to places along with the corresponding colour set (Section 3.1.2). Finally, the DFRS functions are represented in the CPN model as transitions (Section 3.1.3). In what follows, we detail these steps. To illustrate the relevant concepts, in the next sections, we consider the adapted VM example briefly described in Section 2.2.

3.1.1

Representing input variables

By definition, the DFRS functions (requirements) do not change the DFRS input states; therefore, it is not necessary to represent each one of them through CPN places, which would increase the complexity of the model. Hence, just one place (Input) is created to store all the values generated for the input variables. For each input variable of the DFRS, two CPN variables are defined along with the corresponding colour set (type). These two variables store the previous and current values (used by the transition guards when the respective requirement contains the verb becomes or changes) for each entry. For a DFRS variable named the_coin_sensor, we generate the variables pthe_coin_sensor and the_coin_sensor, to stand for its previous and

3.1. CPN MODELS 31

Figure 3.1: Page created to represent the requirement REQ001

the current value, respectively. These mentioned CPN variables are used to transmit the input values (tokens) to the transitions during the simulations. The place Input is created in the page Generator(see Figure 4.1), in the page NoAttendedReq (see Figure 4.2) and in all the pages that model the DFRS functions (see Figure 3.1). These places are linked by a fusion set (see the label Inputsin the just mentioned figures). The type (INPUTLIM) in the place Input comprises an integer value (to limite the number of input values being randomly generated) and the types of the input variables (repeat twice for each input variable).

3.1.2

Representing output variables

In this case of output variables, as DFRS functions (requirements) can change DFRS output states, they are represented by places (to store states), variables (to transmit the previous and current values), and a colour set equivalent to their type. In contrast with the input variables, the output variables do not store a DFRS state, they just transmit or change the state store in the place marking (see the variables the_system_mode and the_coffee_machine_output in Figure 3.1). When a transition that modifies an output is fired, it leads the tokens to the respective place changing its marking and, thus, the value of the corresponding CPN variables accordingly. Each one of these places has as initial marking the initial value of the corresponding DFRS output variable.

Considering the VM example, we have two DFRS output variables that represent the system mode and the coffee machine output. The system mode has four possible modes: choice, idle, preparing strong coffee and preparing weak coffee. The coffee machine output has two

3.1. CPN MODELS 32 possible values: strong and weak. As previously said, for each DFRS output variable, we create a place (the system mode and the coffee machine output), a variable (the_system_mode and the_coffee_machine_output), and a colour set (OCSTHE_SYSTEM_MODE1– an enumeration comprising the four aforementioned values, and OCSTHE_COFFEE_MACHINE_OUTPUT – other enumeration comprising the two values, previously listed). When the transition REQ001 is fired (see Figure 3.1), the marking of the place the system mode is changed from idle to choice. As explained in Section 2.3, CPN transitions are created to represent the system requirements. The arc leading to the place the system mode has as inscription choice, since the requirement REQ001 describes the situation when the system mode becomes choice. Note that this requirement does not cause any change in the variable the_coffee_machine_output, so both arcs have the same inscription. Also, note that the page NoAttendedReq produces no change in the places that represent the DFRS outputs (see Figure 4.2).

3.1.3

Representing functions

For each function of the DFRS (a system requirement), a page is created to represent the function behaviour, thus simplifying individual behaviour analysis of system requirements. Therefore, one CPN page is created to model each function entry. For easing traceability of CPN pages to the requirements, the page is named according to the requirement related to the considered entry.

Each requirement page comprises some places and one transition. The transition (named after the requirement identifier) represents a possible system reaction – models how the output variables must be updated. Therefore, this transition has outgoing arcs to one or more places (the ones that represent DFRS output variables) whose marks must be modified when the transition is fired. Besides the places that represent DFRS output variables, there are two other places (Input and Trace) that are auxiliary elements (later explained in Section 4.1).

Regarding DFRS models, the functions represent the system actions arising from the requirements. Therefore, as explained in Section 2.3, they are translated into transitions within a CPN model. These transitions have guards (conditions to be enabled), which denote a conjunction of the static and timed guards of the DFRS. The ingoing arcs inscriptions of these transitions transmit the values of the input and output DFRS variables, which are used by the guards and by the output arcs of these transitions. Considering the requirement REQ001 of the VM example, and the corresponding function entry (REQ001 line in Table 2.4), we create the arcs the_system_modeand the_coffee_machine_output (see Figure 4.1). As one can see, these arcs are guarded by the conjunction of the corresponding static and timed guards. (see Figure 3.1).

To give a concrete example, consider the adapted VM and the function entry related to REQ001 (see Table 2.4): (¬(prev(the_coin_sensor) = true) ∧ the_coin_sensor = true ∧ the_system_mode = 1). It means that when the system mode is 1 (idle), and the coin sensor changes to true, the coffee machine system shall change the system mode to 0 (choice) and reset

3.2. SYSTEMATIC TRANSLATION 33 the request timer(assigns the global clock value to the timer).

For modeling this function entry, the page REQ001 is created, along with the transition REQ001. To enable this transition, the associated guard needs to evaluate to true. This guard ([not(p_the_coin_sensor = true) andalso the_coin_sensor = true andalso the_system_mode = idle, not(the_system_mode = choice)]) is a translation of the guards of the corresponding function entry. The variable p_the_coin_sensor stands for the previous value of the coin sensor. The last restriction of this guard (not(the_system_mode = choice)) is required to ensure this transition just becomes enabled when a state change will occur as a consequence of firing this transition (later explained in Section 4.2.2).

Besides satisfying its guard, the transition also needs to know the current and previous values of the system inputs, along with the value of the system outputs. Note that the afore-mentioned guard also depends upon the value of a system output – the system mode. These values (tokens) are sent to the transition REQ001 by the arcs emanating from the places Input and the system mode. When this guard evaluates to true, and the required tokens are available, the transition fires and assigns choice to the place the system mode. As previously said, for every entry of each DFRS function, a page similar to this one is created.

3.2

Systematic translation

In this section we describe the approach used to translate from DFRS to CPN model in a systematic way: first, proposing a grammar for DFRSs (Section 3.2.1), then defining transformation rules in Spoofax (Section 3.2.2).

3.2.1

Defining a DFRS grammar

As explained in Section 2.2, a DFRS model, an intermediate formalism describing the system behaviour, is composed by three disjoint sets of input and output variables, as well as internal timers. It also has a global clock. The system behaviour is represented by functions, each on with a pair of guards, along with its corresponding reaction, represented by assignments. The system needs to have at least one input and one output variable; however, there may be no timers, when the system has no temporal aspects. For each variable it is known its name, its type (boolean, integer or float), and its initial value. The global clock is always named gc. When translating to CPNs, to enable simulation (see Section 4.1.1), we also need to have a list of values to be considered during simulation. This information is required only for input variables whose type is integer. For variables of enumerated types we can consider all the possible values, as these are necessarily finite.

A function, which describes the system reaction in a given context, is defined by a non-empty set of tuples. Each tuple is composed by a name, and a pair of static and timed guards (boolean expressions), and a set of assignments that shall be performed when both guards evaluate to true. The static guards shall refer to input and output variables, whereas the timed

3.2. SYSTEMATIC TRANSLATION 34

Figure 3.2: Part of the XML representation of the DFRS for the VM example

guards are restricted to expressions over timers. One of the guards can be empty, but not both. Besides the aforementioned information, the NAT2TEST tool also keeps traceability information (the identifier of the requirement that is the source of the corresponding tuple). See a part of the XML representation of the DFRS for the Vending Machine (VM) example in Figure 3.2.

To define a systematic translation from DFRSs to CPNs, we first need to define a grammar, which, concretely, we adopt an XML representation of DFRS models; so far, these models were only held in memory. In the XML grammar for DFRS models (Figure 3.3), one can note that the input (INVar) and output (OUTVar) variables have the same definition, except for their XML tags. The Timer (TimerDef ) and Global Clock (GClock) definitions are also similar; however, the global clock name is the constant gc. A function can have one or more tuples (FuncTuple). Each tuple comprises a pair of static (SGuard) and time (TGuard)) guards, the corresponding assignments (Stm+), besides the requirement ID for traceability purposes (Req). Each assignment is composed by a variable identifier (VarName) and a value to be assigned (Expression).

3.2. SYSTEMATIC TRANSLATION 35

Figure 3.3: DFRS grammar

3.2.2

Creating transformation rules

The translation from DFRSs to CPNs occurs via transformation rules2, which are divided into modules for easy understanding and maintenance. In this work, they were structured in modules according to the XML format for .cpn files, that is, the order they should be presented in a CPN file. (See the Document Type Definition (DTD) for .cpn files in Appendix B)

The first section of the CPN file is the globbox, which is divided into two blocks: Standard priorities(for the transitions) and Standard declarations (wherein are declared the constants, variables, coloursets, functions and operations of the CPN). Therefore, the module cpnDeclarationsis created for defining the transformation rules for these blocks. For instance, the rule create-cpn-inputs (Figure 3.4) creates the declaration of input variables, where the current and previous values for each given input of the DFRS are stored. Note that the rule input is the abstract term INVar, obtained from the generated Abstract Syntax Tree (AST) according to the DSL proposed here, and it generates as output the variable declarations contained in this term (INVar). The rule create-colorset-input provides the type (colorset) of these input variables.

Afterwards, the modules for creating CPN pages are specified. The module pgGenerator contains the rules that create the page Generator, which generates the system input values and

3.2. SYSTEMATIC TRANSLATION 36

Figure 3.4: The rule create-cpn-inputs

comprises a place to store all of them, a place to limit the quantity of input vectors and the transition to generate the random system entries, along with their respective arcs. Most of this page contains fixed components that do not depend on the modelled system. The variable part consists only by the vector of the initial input values, the vector of the generated values for the entries, the vector of the input variable names and a setting for working with continuous or discrete time, as appropriate. The transformation rule round-tistamp (see Code 3.1) performs this configuration. From the term Timer (line 2), the aforementioned rule defines whether the model will work with continuous or discrete time. When the timer type (vt) is Integer (line 5), the model timestamps are rounded by the operator round (line 6).

Code 3.1: Stratego – The transformation rule round-tistamp 1 r o u n d −t i s t a m p : 2 T i m e r ( T i m e r D e f ( VarName ( vn ) , v t , I n i t i a l V a l u e ( i v ) ) ) −> c s 3 w h e r e 4 s w i t c h ! v t 5 c a s e ! v t => I n t e g e r ( ) : 6 c s : = $ [ r o u n d ] 7 o t h e r w i s e : 8 c s : = $ [ + ] 9 end

The pgNoAttReq module specifies the rules that create the page NoAttendedReq (men-tioned in Section 3.1.2), which deals with the situation when no different system reaction is observed. This page contains a unique transition, with low priority, which is enabled only in a situation where no system requirement (DFRS function) is satisfied. In this case, the input is consumed, being transmitted from the Input place to the Test Oracles place. The Input place is a clone of the place with the same name in the Generator page, which stores the entries

3.2. SYSTEMATIC TRANSLATION 37

Figure 3.5: The rule page-reqx

generated for the simulation model. The place Test Oracles stores the test vectors to be written to a Comma-separated values (CSV) file. This page also has fixed and variable portions.

The pgReqX module contains the rules that generate the requirement pages, which model the DFRS functions (see example in Figure 3.1). These pages are similar to the pgNoAttReq page with respect to its components; however, it does not have the Next TO and Test Oracles places. As in the previous page, rules are created for generating places for each of the output variables and timers. Note in the Figure 3.5 that the page-reqx rule organizes the terms of the syntax tree so that a page is created for each system requirement. Then the page-reqx-body rule calls the rules: place-input-reqx (to create the Input place), reqx-trans (to create the transition associated to the corresponding requirement), reqx-vo-places (to create places that model DFRS outputs) and reqx-ti-places (to create places that model the timers).

The pgTestCase module contains the rules that create the TestCase page, responsible for processing the generated test vectors, that are stored in a CSV file. In this module, rules are created for deriving the names of the input and output variables, as well as their initial values.

The cpn-options module creates a special section of the cpn file containing: fusions (for each place that is repeated on different pages), instances (one for each CPN page required) and some settings (options) of the Coloured Petri Nets (CPN). The only option we need to handle is the realtimestamp, to specify whether the CPN model will handle continuous or discrete time, which depends on the system being modelled.

It is worth mentioning that a lot of information, such as size, shape and position of the figure representing the transitions or places, as well as the colours were defined from trials and are not relevant for generating test cases, but only for visual inspection of the models.

38 38 38

4

Test vectors based on CPN simulation

In order to generate test cases via CPN simulation, it is necessary to complement the model generated in Section 3.1 with some auxiliary elements (Section 4.1). Section 4.2 presents how test vectors are generated via simulation of a CPN.

4.1

Auxiliary components

After constructing a CPN model to represent a DFRS (Section 3.1), via the translation mentioned in Chapter 3, we now create auxiliary components, which are used to stimulate the model and to generate test cases. These components (places, transitions, and arcs) are always the same, but parametrised by the number of inputs and outputs. Considering the constant InputQty, which is defined beforehand to limit the number of generated inputs, the following auxiliary elements are created: Generator, NoAttendedReq, TestCase.

4.1.1

Generating entries

A page (Generator) is created to generate input values randomly (Figure 4.1). When fired, the transition Generate Inputs produces the system inputs, and transmits them, along with an index i, to the place Input. This index is used to ensure the order the generated inputs are consumed by the transitions that represent the system behaviour. This index is initially 0 (see the first element of the place Input’s initial marking). We note that the generated inputs are also an input to the transition Generate Inputs. It is necessary because when generating the next inputs, the token sent to the place Inputs might comprise the newly generated values (TYPE.ran()), but also the previous values of the inputs. When the number of generated inputs reaches InputQty, the transition Generate Inputs becomes disabled. The place Next Input aids this control. At this moment, no more inputs (tokens) are generated by this transition. The place Input is initialised by the initial values of the DFRS input variables. The aux text (CPN’Replications.nreplications InputQty) is used to perform a predefined quantity of simulation.

4.1. AUXILIARY COMPONENTS 39

Figure 4.1: Page created to generate the inputs randomly

4.1.2

Dealing with no system reaction

A page (NoAttendedReq) similar to those that represent the requirements (DFRS func-tions) is created. Although this page has a transition (noReq) that consumes the inputs (tokens in the place Input), this transition, unlike those in the pages that model requirement functions, does not change the value of the places that represent output variables (see in Figure 4.2 the bidirectional arcs pointing to the places the_system_mode and the_coffee_machine_output). This transition is enabled when no other transition is enabled, and it represents the idea that when no system reaction is expected for a given input, the system does not change its outputs (their values remain the same). To ensure that noReq is only enabled when no other transition is enabled, it has a lower priority (P LOW).

The place Next TO is used to order the obtained test vectors. When the transition noReq fires, it produces a token that represents a test vector: the received inputs, along with the expected output values. As one can see in Figure 4.2, this token begins with an index z, and the expression time(). This index, which is controlled by Next TO, establishes an order for the tokens stored in Test Oracles. The expression time() is used to get the time from the global clock in which the test vector is generated. Another element of this tuple is the label trace to track which requirements produced the output. This label allows us to relate generated test vectors with requirements and, thus, extract coverage information with respect to the system requirements. The place Trace stores the label transmitted by the transitions that represent the requirements.

4.1. AUXILIARY COMPONENTS 40

Figure 4.2: Page created to deal with no system reaction

4.1.3

Creating a test case file

As one can see in Figure 4.3, the page TestCase stores the steps (test vectors) of the generated test cases in a CSV file. The transition Write File that manipulates the CSV files is enabled when all inputs are generated, that is, when the index i (received from Next Input) is equal to the defined number of the test case inputs (InputQty). The created test vectors, which are stored in the place Test Oracles and ordered by the index z, are written in the CSV file when the transition fires.

This transition executes the associated code segment (see Code 4.1) to create a CSV file in the output simulation path. First, when the transition receives the first tuple whose variable rt is equal to 0, it initialises a file (TC.csv), and adds to it a header – the name of the columns, which are named after the system input and output variables (lines 1 – 3). Afterwards, it appends to this file a test vector every time the transition Write File is fired (lines 6 – 8). When all generated outputs are processed from the place Test Oracles, and it has only the tuple whose label req is STOP, the transition fires closing the file previously created (line 9).

Code 4.1: Action of the transition Write File code segment 1 i f r t =0 t h e n ( o u t f i l e : = T e x t I O . o p e n O u t ( OS . P a t h . c o n c a t 2 ( O u t p u t . g e t S i m O u t p u t D i r ( ) , "TC . c s v " ) ) ; 3 T e x t I O . o u t p u t ( ! o u t f i l e , Head ) ) e l s e ( ) 4 i f r t >=0 t h e n 5 i f r e q <> "STOP" t h e n 6 T e x t I O . o u t p u t ( ! o u t f i l e , v e c t o r ( r t , t h e _ c o f f e e _ r e q u e s t _ b u t t o n , 7 t h e _ c o i n _ s e n s o r , t h e _ s y s t e m _ m o d e ,

4.2. TEST VECTORS GENERATION 41

8 t h e _ c o f f e e _ m a c h i n e _ o u t p u t , r e q ) )

9 e l s e ( T e x t I O . c l o s e O u t ( ! o u t f i l e ) ) e l s e ( ) ;

Figure 4.3: Page created to save the test cases in a CSV file

4.1.4

Repeated places

Fusion sets are created to aggregate places replicated throughout the model pages, that is, all of them represent the same place. For instance, the place Input is an element of all pages, except for TestCase; so, the fusion set Inputs is associated to all the places Input (see a thin rectangle at the bottom of these places in Figures 3.1, 4.1 and 4.2). The same occurs to the places Test Oracles, to which the fusion set Tests is created, and comprises the places replicated in the pages NoAttendedReq and TestCase (Figures 4.2 and 4.3, respectively). Other fusion sets are NextInput (in the pages Generator and TestCase), and Traces (in the pages NoAttendedReq and those that represent the system behaviour). Finally, fusion sets are created to aggregate all places that represent the DFRS output variables or the DFRS timers (see examples of these fusion sets in Figure 4.2).

4.2

Test vectors generation

Once the CPN model is generated, as previously explained in Chapter 3 and Section 4.1, it is possible to generate test cases automatically via CPN simulation: it randomly produces inputs, along with the expected outputs. The tool (CPN Tools) has a number of resources to run simulations; for instance, it can be performed with user intervention or automatically, it can also be repeated n times (simulation replications). Besides generating test cases, simulations