Understanding software product lines inspection in an industrial setting

Texto

(1)“Understanding Software Product Lines Inspection in an Industrial Setting” By. Iuri Santos Souza M.Sc. Dissertation. Federal University of Pernambuco posgraduacao@cin.ufpe.br www.cin.ufpe.br/~posgraduacao. RECIFE, SEPTEMBER/2011.

(2) Federal University of Pernambuco Center for Informatics Graduate in Computer Science. Iuri Santos Souza. “Understanding Software Product Lines Inspection in an Industrial Setting”. A M.Sc. Dissertation presented to the Center for Informatics of Federal University of Pernambuco in partial fulfillment of the requirements for the degree of Master of Science in Computer Science.. Advisor: Silvio Romero de Lemos Meira Co-Advisor: Eduardo Santana de Almeida. RECIFE, SEPTEMBER/2011.

(3) Catalogação na fonte Bibliotecária Jane Souto Maior, CRB4-571 Souza, Iuri Santos. Understanding software product lines inspection in an industrial setting / Iuri Santos Souza - Recife: O Autor, 2011. xxvi, 154 folhas. : il., fig., tab. Orientador: Silvio Romero de Lemos Meira. Dissertação (mestrado) - Universidade Pernambuco. CIn, Ciência da Computação, 2011.. Federal. de. Inclui bibliografia e apêndice. 1. Ciência da computação. 2. Engenharia de software. 3. Reuso de software. I. Meira, Silvio Romero de Lemos (orientador). II. Título. 004. CDD (22. ed.). MEI2011 – 143.

(4)

(5) I dedicate this dissertation to myself and all my family, friends and professors who gave me all necessary support to get here..

(6) Acknowledgements. Initially, I would like to thank God, who protected and guided me during this hard journey and for providing me this opportunity. I would like to thank my great family and friends. Especially, my parents, Antonio and Valentina, my brothers and sisters José Ubirajara, João Emiliano, Ana Nery, Maria Goreti, Francisco Kennedy, Edson Fábio, Lúcia Daniela, and Uoston, that always stood by me with everything I needed during my life. I love them and appreciate the efforts they have put into giving me the life I have now. I would like to express my heartiest thanks to my girlfriend, Renata for her understanding, patience and encouragement when it was most required. My cordial thanks to her family members for their kindness and affection, especially to Ayres, Jamile and Tatiana. I would like to gratefully acknowledge the supervision of Dr. Eduardo Almeida during this work. He provided me with many helpful discussions, suggestions and encouragement during the course of this work as well as the challenging research that lies behind it. I also wish to express my appreciation to Dr. Silvio Meira, for accepting me as M.Sc. Student. Special Gratitude goes to the rest of the teaching staff of the CIn/UFPE, for their valuable classes. My sincere thanks are extended to Dr. Gecynalda Soares, from IM/UFBA, for her help during the data analysis in this study. I would like to thank the Brazilian National Research Council (CNPq) and Foundation for Science and Technology of Pernambuco (FACEPE). Without their grant, this M.Sc. would not have been possible. I cordially thank my friends and colleagues that I have met during my journey in Recife. I thank my housemates Bruno, Ivan, Jonatas, and Leandro for their good friendship and for being the surrogate family during the years I stayed there. I would like to thank all my friends from RiSE (Crescêncio, Fernando Almeida, Flavio, Heberth, Hernan, Ivonei, Leandro Marques, Luanna, Marcela, Padraig, Raphael, Ricardo, Simone, Thiago Fernandes, Williams, Vanilson, Vinicius e Yguaratã), friends from LES (Renato, Bruno, Thiago).. iv.

(7) Agradecimentos. Inicialmente, agradeço a Deus quem me protegeu e guiou nessa dura jornada e por me oferecer essa oportunidade. Agradeço a minha família e irmãos. Especialmente meu pais, Antonio and Valentina, meus irmãos José Ubirajara, João Emiliano, Ana Nery, Maria Goreti, Francisco Kennedy, Edson Fábio, Lúcia Daniela e Uoston, que sempre estiveram ao meu lado em todos os momentos da minha vida. Aprecio e agradeço todos os esforços prestados a mim. Gostaria de externar os meus sinceros agradecimentos a minha namorada, Renata por sua compreenção, paciência e encorajamento quando mais precisei. Meus cordiais agradecimentos aos membros da sua família por suas gentilezas e afetos, especialmente a Ayres, Jamile e Tatiana. Agradeço a orientaç ao do professor Dr. Eduardo Almeida durante esse trabalho. Ele me proveu valiosas discussões, sugestões e encorajamentos durante o mestrado que mostrou-se bastante desafiador. Expressar também o meu agradecimento ao Professor Silvio Meira, por ter me aceito como Estudante de Mestrado. Agradeço ainda ao demais docentes do CIn/UFPE com quem tive a oportunidade de trabalhar, por suas valiosas aulas e discuss oes. Meus sinceros agradecimentos também a professora Dr. Gecynalda Soares, do Instituto de Matemática da Universidade Federal da Bahia, por sua ajuda durante a análise dos dados desse estudo. Meus sinceros agradecimentos ao Conselho Nacional de Pesquisa (CNPq) e a Fundação de Apoio a Ciência e Tecnologia de Pernambuco (FACEPE), sem o apoio financeiro concedido por esta agência, não seria possível realizar este trabalho. Cordialmente agradeço aos amigos e colegas que fiz durante minha jornada em Recife. Obrigado meus colegas de moradia Bruno, Ivan, Jonatas e Leandro por substituir tão bem minha família durante a minha temporada em Recife. Agradeço aos meus amigos do RiSE (Crescêncio, Fernando Almeida, Flavio, Heberth, Hernan, Ivonei, Leandro Marques, Luanna, Marcela, Padraig, Raphael, Ricardo, Simone, Thiago Fernandes, Williams, Vanilson, Vinicius e Yguaratã) e do Labaratório de Engenharia de Software (LES) (Renato, Bruno, Thiago).. v.

(8) Resumo. Reuso de software é um aspecto fundamental para as organizações de software interessadas em melhorar os custos, a qualidade e reduzir o tempo de entrega dos produtos. Engenharia de Linhas de Produtos de software (SPL) é um conceito crescente em engenharia de software e foi desenvolvido objetivando uma maneira sistemática e prescrita para alcançar reuso. SPL é uma estratégia de reuso pró-ativa, que explora semelhanças e gerencia variabilidades dentro de um conjunto de produtos. O inerente reuso de artefatos de software em combinação com o desenvolvimento usualmente iterativo, traz um conjunto de melhorias para o desenvolvimento de software no contexto de SPL. Além disso, o conjunto de benefícios de SPL não acontecem sem esforços, tem alguns desafios e requer abordagens de garantia de qualidade, como testes e inspeção. Se esses pontos não são considerados, o sucesso da linha pode não ser alcançado, contrariando os enormes benefícios econômicos. Embora alguns autores discutam a importância de inspeções de software em todo o desenvolvimento de engenharia de software, na prática, poucos estudos discutem a relação entre inspeção e artefatos das fases iniciais de SPL. O cenário de Qualidade em SPL tem uma carência substancial da literatura em discutir as técnicas de garantia de qualidade. Neste cenário, esta dissertação apresenta um conjunto de evidências empíricas sobre Inspeção em Linhas de Produtos de Software fornecidas por um estudo empírico embutido, realizado em um ambiente industrial com objetivo de compreender e caracterizar como a atividade de inspeção deve ser tratada nas fases iniciais de SPL (escopo e engenharia de requisitos), especialmente para os artefatos de especificação de features, requisitos funcionais e casos de uso. Além disso, com base nos resultados coletados no estudo empírico alguns modelos de predição foram construídos a fim de estimar o número de não-conformidades para os artefatos investigados neste trabalho. Palavras-chave: Linhas de Produtos de Software, Inspeção de Software, Reuso de Software, Qualidade de Software, Engenharia de Software Empírica.. vi.

(9) Abstract. Software reuse is a key aspect for software organizations interested in improving costs, time-to-market and quality. Software Product Lines (SPL) engineering is a growing concept in software engineering and it was developed towards a systematic and prescribed way to achieve reuse. SPL is a pro-active reuse strategy that explores commonalities and manages variabilities within a set of products. The inherent reuse of software artifacts in combination with the usual iterative development brings a set of improvements for the software development in an SPL context. However, the set of SPL benefits do not come for free, it demands some challenges and requires quality assurance approaches, such as testing and inspection. If these issues are not considered, as otherwise huge economic benefits, the line success can be missed. Although some authors discuss the importance of software inspections in every software engineering development, in practice, few studies discuss the relationship among inspections and artifacts from early SPL phases. The SPL Quality scenario has a lack of substantial literature discussing quality assurance techniques. Ê In this context, this dissertation presents a set of empirical evidences about Software Product Lines Inspection provided by an embedded empirical study conducted in an industrial environment, with the goal of understanding and characterizing how the inspection activity should be handled in the SPL early phases (Scoping and Requirements engineering), particularly on features, functional requirements and use cases specifications artifacts. Keywords: Software Product Lines, Software Inspection, SPL Inspection, Software Reuse, Software Quality, Empirical Software Engineering.. vii.

(10) Contents. List of Figures. xxi. List of Tables. xxiii. List of Acronyms 1. 2. 3. Introduction 1.1 Motivation . . . . . . . . . . . 1.2 Problem Statement . . . . . . 1.2.1 Context . . . . . . . . 1.3 Out of Scope . . . . . . . . . 1.4 Statement of the Contributions 1.5 Dissertation Structure . . . . .. xxv. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. . . . . . .. Software Product Lines: An Overview 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . 2.2 Software Reuse . . . . . . . . . . . . . . . . . . . 2.3 Software Product Lines . . . . . . . . . . . . . . . 2.3.1 SPL Essential Activities . . . . . . . . . . 2.3.2 Software Product Lines Practice Areas . . . Software Engineering Practice Areas . . . Technical Management Practice Areas . . . Organizational Management Practice Areas 2.4 SPL Variability Management . . . . . . . . . . . . 2.5 Software Product Lines Quality . . . . . . . . . . . 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . Foundations on Software Inspection 3.1 Introduction . . . . . . . . . . . . . . . . 3.2 Technical Dimension Software Inspection 3.3 Software Inspection Processes . . . . . . 3.3.1 Fagan’s Inspection Process . . . . 3.3.2 Inspection Process Variations . . Active Design Review . . . . . . Two-Person Inspection . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. . . . . . . . . . . .. . . . . . . .. . . . . . .. 1 3 4 4 6 7 7. . . . . . . . . . . .. 9 9 9 11 13 14 16 16 16 16 18 18. . . . . . . .. 19 19 20 22 22 23 24 24. viii.

(11) 3.4. 3.5. 3.6. 3.7 4. N-fold Inspection . . . . . . . . . . . . . . Phased Inspection . . . . . . . . . . . . . . Sampling-based Inspections . . . . . . . . Other Inspection Processes . . . . . . . . . Inspection Reading Techniques . . . . . . . . . . . 3.4.1 Ad-hoc Reading . . . . . . . . . . . . . . 3.4.2 Checklist-Based Reading . . . . . . . . . . 3.4.3 Reading by Stepwise Abstraction . . . . . 3.4.4 Scenario-Based Reading . . . . . . . . . . 3.4.5 Perspective-Based Reading . . . . . . . . . 3.4.6 Other Reading Techniques . . . . . . . . . Organizational Dimension of Software Inspection . 3.5.1 Software Inspection Team . . . . . . . . . 3.5.2 Project Development Structure . . . . . . . 3.5.3 Project Development Environment . . . . . Managerial Dimension of Software Inspection . . . 3.6.1 Quality . . . . . . . . . . . . . . . . . . . 3.6.2 Cost . . . . . . . . . . . . . . . . . . . . . Software Inspection Cost Analysis Reports 3.6.3 Duration . . . . . . . . . . . . . . . . . . 3.6.4 Inspection Metrics . . . . . . . . . . . . . 3.6.5 Defect Estimation . . . . . . . . . . . . . . Chapter Summary . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . .. Understanding Software Product Lines Inspection in an Industrial Setting 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Empirical Study Context . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 MedicWare Background . . . . . . . . . . . . . . . . . . . . . 4.2.2 Technology Transfer . . . . . . . . . . . . . . . . . . . . . . . 4.3 Empirical Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Definition of the Empirical Study . . . . . . . . . . . . . . . . Research Questions . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Data Collection Procedures . . . . . . . . . . . . . . . . . . . . 4.3.3 Analysis Procedure . . . . . . . . . . . . . . . . . . . . . . . . 4.3.4 Validity Procedure . . . . . . . . . . . . . . . . . . . . . . . . 4.4 The Proposed Inspection Approach . . . . . . . . . . . . . . . . . . . .. 25 26 26 27 27 29 29 29 30 31 32 33 33 34 34 35 35 35 35 38 38 39 41 43 43 44 44 45 47 47 48 49 50 50 51. ix.

(12) 4.5 5. Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 53. Results of the Empirical Study 5.1 Unit of Analysis: Inspection on Features specifications . . . . . . . . . 5.1.1 RQ1. What is the distribution of non-conformities in the features specifications? . . . . . . . . . . . . . . . . . . . . . . . . . . RQ1.1. How the features non-conformities are spread in terms of sub-domains? . . . . . . . . . . . . . . . . . . . . RQ1.2. What is the distribution of features non-conformities occurrences on types of non-conformities? . . . . . . RQ1.3. How distributed are the non-conformities regarding items of the features specification template? . . . . . . . . 5.1.2 RQ2. Is there any correlation between the feature information and the non-conformities? . . . . . . . . . . . . . . . . . . . . RQ2.1. Are new non-conformities inserted into the project when new features are added? . . . . . . . . . . . . . . . . RQ2.2. Is there any correlation between feature types and the amount of non-conformities recorded? . . . . . . . . RQ2.3. Is there any correlation between features hierarchy and the nonconformities found? . . . . . . . . . . . . . . . . . . RQ2.4. Is there any correlation between feature interactions and non-conformities? . . . . . . . . . . . . . . . . . . . 5.1.3 RQ3. Is there any correlation between features sub-domains information and features non-conformities? . . . . . . . . . . . RQ3.1. Are sub-domains with high number of features responsible for high number of non-conformities? . . . . . . . RQ3.2. Is there any correlation between the scope phase subdomains information and features non-conformities? . 5.1.4 RQ4. How much effort is spent during the inspection on features specification? How is it distributed in terms of inspection tasks? 5.2 Unit of Analysis: Inspection on Functional Requirements specifications 5.2.1 RQ1. How the non-conformities are distributed in the functional requirements specification? . . . . . . . . . . . . . . . . . . . . RQ1.1. How the functional requirements non-conformities are spread in terms of sub-domains? . . . . . . . . . . .. 55 56 56 56 58 61 63 65 65. 69 70 72 72 73 75 79 80 80. x.

(13) 5.3. RQ1.2. Which requirement non-conformities types present the major occurrences? . . . . . . . . . . . . . . . . . . 80 RQ1.3. How the non-conformities are distributed regarding items of the requirement specification template? . . . . . . 81 5.2.2 RQ2. Is there any correlation between requirements information and requirement non-conformities? . . . . . . . . . . . . . . . 83 RQ2.1. Is there any correlation between requirement types and requirement non-conformities? . . . . . . . . . . . . 84 RQ2.2. Is there any correlation between requirements priority and requirements non-conformities? . . . . . . . . . 86 RQ2.3. Is there any correlation between requirements with variation points and non-conformities numbers? . . . . . . 88 5.2.3 RQ3. Is there any correlation between scoping phase sub-domains information and requirements non-conformities? . . . . . . . . 90 5.2.4 RQ4. Is there any correlation between features and functional requirements non-conformities? . . . . . . . . . . . . . . . . . 91 5.2.5 RQ5. How much effort is spent during the inspection on functional requirements specification? How is it distributed in terms of inspection tasks? . . . . . . . . . . . . . . . . . . . . . . . . 91 Unit of Analysis: Inspection on Use Cases specifications . . . . . . . . 96 5.3.1 RQ1. What is the distribution of non-conformities in the use cases specifications? . . . . . . . . . . . . . . . . . . . . . . . 96 RQ1.1. How the use cases non-conformities are spread in terms of sub-domains? . . . . . . . . . . . . . . . . . . . . 96 RQ1.2. What is the distribution of use cases non-conformities occurrences on the types of non-conformities? . . . . 97 RQ1.3. How distributed are the use cases non-conformities regarding items of the use case specification template? . 99 5.3.2 RQ2. Is there any correlation between use cases information and use case non-conformities? . . . . . . . . . . . . . . . . . . . . 101 RQ2.1. Is there any correlation between use cases type and use cases nonconformities? . . . . . . . . . . . . . . . . . . . . . 102 RQ2.2. Is there any correlation between use cases with secondary flow and use cases non-conformities? . . . . . . . . . 103. xi.

(14) 5.3.3. 5.4 5.5 5.6 5.7 5.8 6. RQ3. Is there any correlation between the scoping phase subdomains information and use case non-conformities? . . . . . . 5.3.4 RQ4. Is there any correlation between functional requirements and use cases non-conformities? . . . . . . . . . . . . . . . . . 5.3.5 RQ5. How much effort is spent during the inspection on use cases specification? How is the effort distribution across the inspection tasks? . . . . . . . . . . . . . . . . . . . . . . . . . Main Findings of the Study . . . . . . . . . . . . . . . . . . . . . . . . Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Drawbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Threats To Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. Conclusion 6.1 Research Contributions . 6.2 Related Work . . . . . . 6.3 Future Work . . . . . . . 6.4 Academic Contributions 6.5 Concluding Remarks . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. . . . . .. 104 106. 106 111 112 113 114 115 117 118 118 119 120 120. Bibliography. 121. Appendices. 135. A Inspection on Features Specification Instruments 137 A.1 Feature Specification Quality Attribute List . . . . . . . . . . . . . . . 137 A.2 Inspection Checklist for Feature Specification . . . . . . . . . . . . . . 139 A.3 Feature Non-conformities Spreadsheet . . . . . . . . . . . . . . . . . . 140 B Inspection on Functional Requirements Specification Instruments 143 B.1 Functional Requirements Specification Quality Attribute List . . . . . . 143 B.2 Inspection Checklist for Functional Requirements Specification . . . . . 145 B.3 Functional Requirement Non-conformities Spreadsheet . . . . . . . . . 146 C Inspection on Use Cases Specification Instruments 149 C.1 Use Cases Specification Quality Attribute List . . . . . . . . . . . . . . 149 C.2 Inspection Checklist for Use Cases Specification . . . . . . . . . . . . 151. xii.

(15) C.3 Use Case Non-conformities Spreadsheet . . . . . . . . . . . . . . . . . 152. xiii.

(16) List of Figures. 1.1 1.2. RiSE Labs Influences . . . . . . . . . . . . . . . . . . . . . . . . . . . RiSE Labs Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 2.1 2.2 2.3 2.4 2.5 2.6 2.7. Road to Systematic Reuse . . . . . . . . . . . Staged adoption of reuse . . . . . . . . . . . . Economics of software product line engineering Essential product lines activities . . . . . . . . Core Asset . . . . . . . . . . . . . . . . . . . . Product Development . . . . . . . . . . . . . . Categories of Practice Areas . . . . . . . . . .. . . . . . . .. 10 11 12 13 14 15 15. 3.1 3.2. Technical Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . Characterization Reading Techniques . . . . . . . . . . . . . . . . . .. 21 28. 4.1 4.2 4.3. MedicWare Products and their relationship. . . . . . . . . . . . . . . . RiPLE Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Systematic Inspection Approach on Feature Specification. . . . . . . .. 45 46 54. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. . . . . . . .. 5 6. 5.1 5.2 5.3. Control Chart - Feature non-conformity densities per sub-domain. . . . 58 Feature non-conformity types distribution Pareto’s chart. . . . . . . . . 60 Feature non-conformities distribution: items of template by types of non-conformity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.4 Defective features distribution by items of template. . . . . . . . . . . . 63 5.5 Hierarchical Feature non-conformities distribution per type of non-conformities (%). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.6 Inspection Effort - Features specification. . . . . . . . . . . . . . . . . 76 5.7 Boxplot effort of Inspection on Features by iteration. . . . . . . . . . . 76 5.8 Effort distribution across Feature Inspection tasks - 1st iteration. . . . . 77 5.9 Effort distribution across Feature Inspection tasks - 2nd iteration. . . . . 77 5.10 Effort distribution across Feature Inspection tasks, without planning - 1st iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.11 Effort distribution across Feature Inspection tasks, without planning 2nd iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.12 Control Chart - Functional Requirement non-conformity densities per sub-domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81. xiv.

(17) 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21 5.22 5.23 5.24 5.25 5.26 5.27 5.28. Requirement non-conformity types distribution Pareto’s chart. . . . . . Inspection Effort - Functional Requirements specification. . . . . . . . Boxplot effort of Inspection on Functional Requirements by iteration. . Distribution of effort across Requirement Inspection tasks - 1st iteration. Distribution of effort across Requirement Inspection tasks - 2nd iteration. Distribution of effort across Requirement Inspection tasks, without planning - 1st iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Distribution of effort across Requirement Inspection tasks, without planning - 2nd iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control Chart - Use case non-conformity densities per sub-domain. . . . Use Case non-conformity types distribution Pareto’s chart. . . . . . . . Distribution of incompleteness occurrences in the items of the template. Inspection Effort - Use Cases specification . . . . . . . . . . . . . . . . Boxplot effort of Inspection on Use Cases per iteration. . . . . . . . . . Distribution of effort across Use Case inspection tasks - 1st iteration. . . Distribution of effort across Use Case inspection tasks - 2nd iteration. . . Distribution of effort across Use Case inspection tasks, without planning - 1st iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Distribution of effort across Use Case inspection tasks, without planning - 2nd iteration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 82 92 93 94 94 95 95 97 98 100 107 108 108 109 110 110. xv.

(18) List of Tables. 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 5.19 5.20 5.21. RiPLE disciplines execution. . . . . . . . . . . . . . . . . . . . . . . . Inspection activity executions. . . . . . . . . . . . . . . . . . . . . . . Feature non-conformities per sub-domain . . . . . . . . . . . . . . . . Feature non-conformities distribution: items of template by sub-domains Spearman’s rank correlation parameters: Unit of Analysis - Inspection on Features specifications . . . . . . . . . . . . . . . . . . . . . . . . . Simple Poisson regression parameters: Unit of Analysis - Inspection on Features specifications . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple Poisson regression models: Unit of Analysis - Inspection on Features specifications . . . . . . . . . . . . . . . . . . . . . . . . . . Feature non-conformities distribution: features types per type of nonconformity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mandatory Feature non-conformities distribution: items of template per type of non-conformity. . . . . . . . . . . . . . . . . . . . . . . . . . . Scope phase sub-domains analysis summary . . . . . . . . . . . . . . . Effort for performing inspection on Features specifications . . . . . . . Requirement non-conformities per sub-domain . . . . . . . . . . . . . Requirement non-conformities distribution: items of template per subdomains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Requirement non-conformities distribution: items of template by types of non-conformity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Spearman’s rank correlation parameters: Unit of Analysis - Inspection on Functional Requirements specifications . . . . . . . . . . . . . . . . Simple Poisson Regression parameters: Unit of Analysis - Inspection on Functional Requirements specifications . . . . . . . . . . . . . . . . . Requirement non-conformities distribution: requirement types per type of non-conformity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Requirement non-conformities distribution: requirement priorities per type of non-conformity. . . . . . . . . . . . . . . . . . . . . . . . . . . Requirement non-conformities distribution: requirements with and without variation points per non-conformity types . . . . . . . . . . . . . . Scoping phase sub-domains analysis summary . . . . . . . . . . . . . . Spearman’s rank correlation between features and functional requirements.. 55 55 58 62 65 66 67 68 68 74 75 80 83 83 84 85 86 86 89 90 91. xvi.

(19) 5.22 5.23 5.24 5.25 5.26 5.27 5.28 5.29 5.30 5.31 5.32 5.33. Inspection Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Use Case non-conformities per sub-domain. . . . . . . . . . . . . . . . Use Case non-conformity type distribution . . . . . . . . . . . . . . . . Template Item Occurrences distributed by sub-domains . . . . . . . . . Use Case non-conformities distribution: items of template (Pre-condition and Main flow) per types of non-conformity. . . . . . . . . . . . . . . . Spearman’s rank correlation parameters: Unit of Analysis - Inspection on Use Cases specifications . . . . . . . . . . . . . . . . . . . . . . . . Simple Poisson regression parameters: Unit of Analysis - Inspection on Use Cases specifications . . . . . . . . . . . . . . . . . . . . . . . . . Use Case non-conformities distribution between mandatory and optional use cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Numbers of defective mandatory and optional use cases distribution in the template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scope phase sub-domains analysis summary . . . . . . . . . . . . . . . Spearman’s rank correlation between requirements and use cases. . . . Effort for performing inspection on Use Case specifications. . . . . . .. 92 97 99 100 101 102 102 103 103 105 106 107. A.1 Feature specification non-conformity types . . . . . . . . . . . . . . . . 139 A.2 Inspection Checklist for Feature Specification . . . . . . . . . . . . . . 141 A.3 Feature Non-conformities Spreadsheet . . . . . . . . . . . . . . . . . . 142 B.1 Functional Requirements specification non-conformity types . . . . . . 145 B.2 Inspection Checklist for Functional Requirement Specification . . . . . 147 B.3 Functional Requirement Non-conformities Spreadsheet . . . . . . . . . 148 C.1 Use Cases specification non-conformity types . . . . . . . . . . . . . . 151 C.2 Inspection Checklist for Use Case Specification . . . . . . . . . . . . . 153 C.3 Use Case Non-conformities Spreadsheet . . . . . . . . . . . . . . . . . 154. xvii.

(20) List of Acronyms. BAST. Bug Report Analysis and Search Tool. BPR. Business Process Reengineering. BRN. Bug Report Network. BTT. Bug Report Tracker Tool. CCB. Change Control Board. C.E.S.A.R. Recife Center For Advanced Studies and Systems CIn. Informatic Center. CORE. Reuse Repository System. FPA. Function Point Analysis. FR. Functional Requirement. FRS. Functional Requirement Specification. FS. Feature Specification. GQM. Goal Question Metric. JE. Jackknife Estimator. JPL. Jet Propulsion Laboratory. LIFT. Legacy InFormation retrieval Tool. LOC. Lines of Code. MLE. Maximum Likelihood Estimator. NFR. Non-Functional Requirement. PBR. Perpective-Based Reading. RiPLE. RiSE Product Line Engineering Process. RiSE. Reuse in Software Engineering Labs. xviii.

(21) SCM. Software Configuration Management. SD. Standard Deviation. SEI. Software Engineering Institute. SI. Software Inspection. SOPLE. Software Product Lines based on Services Process. SPL. Software Product Lines. ToolDAy. Domain Analysis Tool. UFPE. Federal University of Pernambuco. UC. Use Case. UCS. Use Case Specification. VM. Variability Management. xix.

(22) 1. Introduction. A capacidade de luta que há em você precisa das adversidades para revelar-se. The ability to fight in you needs the adversities to show up. —LOURIVAL LOPES (Sementes de Felicidade). Software reuse is a key aspect for software organizations interested in improving costs, time-to-market and quality (Mili et al., 2001). However, when the reuse initiative is based on an approach focusing on small-scale, ad-hoc reuse (i.e. typically restricted to the code level), these benefits may not be perceived (Linden et al., 2007). Software Product Lines (SPL) engineering is a growing concept in software engineering and it was developed towards a systematic and prescribed way to achieve reuse (Pohl et al., 2005). SPL is a systematic and pro-active reuse strategy that explores the similarities within a set of products. The inherent reuse of software artifacts in combination with the usual iterative development, brings a set of improvements for the development in an SPL context, as pointed out by (Clements and Northrop, 2001). Nevertheless, in conjunction with the benefits, a set of challenges for quality assurance, a very important activity for SPL, can be found (Denger and Kolb, 2006). In order to ensure that the artifacts produced are in conformance with the required quality levels, techniques such as testing and software inspections can be applied. Shortly, testing is a dynamic quality assurance technique most commonly applied on executable artifacts (Tian, 2001), whereas software inspection is a static quality assurance technique which can be performed on each software artifact created in the software development lifecycle (e.g. requirements, design, test cases, code, and so on.) (Tian, 2001). According. 1.

(23) to (Wiegers, 2002), software inspection is one of the most efficient software quality assurance techniques, especially for the earlier life-cycle phases, in the earlier life-cycle phases, identify and correct non-conformities cost less than in later phases (Tian, 2001). Some studies pointed out that software inspections can lead to the detection and correction of anywhere between 50% and 90% of defects (Gilb and Graham, 1993; Wiegers, 2002). Inspection also brings important education and social benefits. Young software developers can quicker learn standards for specification and code while working as inspectors, while expert developers under pressure are less tempted to ignore standards (Pezze and Young, 2007). Another important aspect to be considered in the SPL development is the variability management. Variability is the tangible difference among the products in SPL context, it enables the development of customized products by reusing predefined and adjustable assets (Pohl et al., 2005). Variability Management (VM) encompasses the activities that represent variability in software artifacts throughout the life-cycle, managing dependencies among different variabilities, and supporting the instantiations of products from those variabilities (Schmid and John, 2004). Variability management techniques can be assessed by quality assurance techniques, such as inspection, to increase the reliability of the artifacts produced for reuse. Even with some research (Boehm, 1987; Gilb and Graham, 1993; Wiegers, 2002) highlighting the importance of software inspections in every phase of software development, few studies establish the relationship among inspections and SPL (Denger and Kolb, 2006). According to (Babar et al., 2010) there is a lack of substantial literature detailing quality assurance techniques from the point of view of variability in software engineering. Based on this scenario, we tried to understand and characterize how inspection should be handled in the SPL practice, in order to identify evidences on inspection in the SPL early phases such as Scoping and Requirements, besides possible gaps that have not been addressed by current research. This understanding was defined on the basis of a large empirical study, aimed at investigating the non-conformities found during the application of a systematic software inspection approach in a reengineering project towards the introduction of SPL practices in a Brazilian company. Details about the effort required to perform inspections in the project, the most common non-conformities and the lessons learned are further reported. The remainder of this chapter describes the focus of this dissertation and starts by presenting its motivation in Section 1.1 and a clear definition of the problem in Section 1.2, while Section 1.3 describes some related aspects that are not directly addressed by this. 2.

(24) 1.1. MOTIVATION. work. Section 1.4 presents the main contributions and, finally, Section 1.5 describes how this dissertation is organized.. 1.1 Motivation The SPL approach provides a systematic reuse which is focused on improving the development time-to-market, cost, quality, portfolio scale and scope and other business drivers, such as customer satisfaction (Northrop and Clements, 2007). Based on this potential, companies have discovered that a product line strategy can produce many benefits. The quality or quality control, as in traditional software development approach, have an important function on SPL and needs to be applied along all over the SPL life-cycle under penalty of not achieving the SPL benefits. The software quality control community has been defined and investigated many aspects about qualities techniques, such as software reviews, inspections, and testing. In order to ensure that the artifacts produced are in conformance with the required quality levels, techniques such as testing and software inspection can be applied on SPL artifacts. The Software Engineering Institute (SEI) introduces the Testing practice in the SPL framework (Northrop and Clements, 2007) for achieving success on adoption of SPL. In this framework, Testing is defined as two functions: “first, helping to identify faults that lead to failures so they can be repaired and secondly, determining whether the software under test can perform as specified by its requirements”. Nevertheless, in the SPL literature is more common to see quality approaches related to executable assets (Neto et al., 2010; Machado et al., 2011; Engström and Runeson, 2011), especially, testing in software product lines. In (McGregor, 2001), the most active research in this area (Quality Assurance for SPL), he proposes an approach to run test on executable and non-executable artifacts. For non-executable artifacts, McGregor proposes to apply guided inspection, a software inspection approach guided by test cases built from defined scenarios. In this context, the focus of this dissertation is to understand and characterize how inspection can be handled in SPL projects and provides empirical evidences of inspection on the SPL scoping and requirement engineering phase artifacts.. 3.

(25) 1.2. PROBLEM STATEMENT. 1.2 Problem Statement Motivated by the challenges presented in the previous section, the goal of the work described in this dissertation can be stated as follows: This work aims to present evidences about the effects of applying an inspection approach in the SPL early phases and their related artifacts (features, requirements and use cases specifications) based on the analysis from industrial empirical study results. The set of empirical evidences represent the effort towards understanding how inspection activities should be held at initial phases in a SPL development project. In order to achieve the goal stated above an industrial empirical case study composed of three units of analysis was performed in the SPL project, which will be presented on Chapter 4. Next it is presented the context in which this work is inserted.. 1.2.1 Context This dissertation is part of the Reuse in Software Engineering Labs (RiSE) 1 (Almeida et al., 2004), formerly called RiSE Project, whose goal is to develop a robust framework for software reuse in order to enable the adoption of a reuse program. However, it is influenced by a series of areas, such as software measurement, architecture, quality, environments and tools, and so on, in order to achieve its goal. The influence areas are depicted in Figure 1.1. Based on these areas, the RiSE Labs is divided in several projects, as shown in Figure 1.2. As it can be seen, this framework embraces several different projects related to software reuse and software engineering. They are: • RiSE Framework: Involves reuse processes (Almeida et al., 2004; Nascimento, 2008), component certification (Alvaro, 2009) and reuse adoption process (Garcia, 2010). • RiSE Tools: Research focused on software reuse tools, such as the Admire Environment (Mascena, 2006), the Basic Asset Retrieval Tool (B.A.R.T) (Santos et al., 1 labs.rise.com.br. 4.

(26) 1.2. PROBLEM STATEMENT. Figure 1.1 RiSE Labs Influences. 2006), which was enhanced with folksonomy mechanisms (Vanderlei, 2006), semantic layer (Durao, 2008), facets (Mendes, 2008) and data mining (Martins et al., 2008), and the Legacy InFormation retrieval Tool (LIFT) (Brito, 2007), the Reuse Repository System (CORE) (Burégio, 2006), the Tool for Domain Analysis (ToolDAy) (Lisboa, 2008), and and a Bug Report Analysis and Search Tool (BAST) (Cavalcanti, 2009); • RiPLE: Stands for RiSE Product Line Engineering Process and aims at developing a methodology for Software Product Lines, composed of scoping (Moraes et al., 2009), requirements engineering (Neiva, 2009), design (Cavalcanti, 2010), implementation, test (Neto, 2010; Machado, 2010), and evolution management (Oliveira, 2009). • SOPLE: Development of a methodology for Software Product Lines based on services (Cavalcanti, 2009; Eduardo et al., 2009), with some idea of the RiPLE; • MATRIX: Investigates the area of measurement in reuse and its impact on quality and productivity; • BTT: Research focused on tools for detection of duplicate bug reports, such as in. 5.

(27) 1.3. OUT OF SCOPE. (Cavalcanti et al., 2008; Eduardo et al., 2009). • Exploratory Research: Investigates new research directions in software engineering and its impact on reuse; • CX-Ray: Focused on understanding the Recife Center For Advanced Studies and Systems (C.E.S.A.R.), and its processes and practices in software development. This work is part of a broader context, as mentioned previously, in which a complete framework for software product lines development has been designed.. Figure 1.2 RiSE Labs Projects. 1.3 Out of Scope Some SPL and Software Inspection topics are not described in this work and were considered out of scope of this dissertation. The following issues are not part in this work: • SPL Inspection Process: It is known that a well-defined SPL Inspection process can bring benefits for organizations adopting SPL. However, few work cover the inspection activity in the SPL context. In this way, a SPL Inspection Process is not part of the scope of this work. This research provides a set of evidences for adopting Inspection in SPL, but it does not determine a set of systematic steps to be followed. • Tool Support: In order to support the inspection activity, some tool support may be required. However, it is out of scope to develop a tool for this finality.. 6.

(28) 1.4. STATEMENT OF THE CONTRIBUTIONS. 1.4 Statement of the Contributions As a result of the work presented in this dissertation, the following contributions can be highlighted: • The definition, planning, operation and analysis of an industrial case study, in order to identify the evidences in this natural context. • A set of empirical evidences, which aims to promote the understanding of the inspection activity in SPL early phases.. 1.5 Dissertation Structure The remainder of this dissertation is organized as follows: • Chapter 2 reviews the essential Software Product Lines concepts used throughout this work (background, activities, practice areas and variability management) and presents an initial discussion on the relationship between SPL concepts and Software Quality Assurance. • Chapter 3 reviews the foundation of software inspection and presents the state-ofthe-art on this area, based on: inspection process, reading techniques, roles and so on. • Chapter 4 presents the design of the industrial empirical study performed in order to characterizing and understanding software inspection activity in the early SPL phases and artifacts. • Chapter 5 reports the findings of the empirical study by units of analysis (features, functional requirements and use cases specifications). • Chapter 6 provides a set of conclusions based on this work, discussing the main contributions and limitations, related work, and outlines directions for future work.. 7.

(29) 2. Software Product Lines: An Overview. 2.1 Introduction Software Product Lines (SPL) reached considerable attention in the Software Engineering research community on the last years (Clements and Northrop, 2001; Bosch, 2002; Northrop, 2002; Linden et al., 2007; Habli and Kelly, 2007; Bastos et al., 2011), once SPL have proved to be a very successful approach to organizational software reuse (Bosch, 2002). On mass production product, the SPL paradigm uses a common platform to derive products that can be customized to specific customers or market segments needs (Clements and Northrop, 2001). This chapter introduces the context and foundations of SPL, including some essential SPL aspects, such as activities, variability management, practice areas and SPL quality assurance. This chapter is structured as follows: Section 2.2 contextualizes the concepts of software reuse. In Section 2.3, the concepts related to SPL are discussed. Section 2.4 describes the variability management foundations. Quality Assurance on software product lines is discussed in Section 2.5 and Section 2.6 summarizes the chapter.. 2.2 Software Reuse Software reuse ideas started to be discussed in 1968, during the NATO - Software Engineering Conference held in Germany (Naur and Randell, 1968). The conference focus was the software crisis and the problem of building large, reliable software systems in a controlled, cost-effective way. Software reuse was pointed as being the main solution of the crisis. The reference paper “Mass Produced Software Components” (McIlroy, 1968) is considered the seminal paper that spread out the software reuse research area.. 8.

(30) 2.2. SOFTWARE REUSE. By definitions software reuse is “the process of creating software systems from exiting software rather than building them from scratch” (Krueger, 1992), “the systematic practice of developing software from a stock of building blocks, so that similarities in requirements and/or architecture between applications can be exploited to achieve substantial benefits in productivity, quality and business performance” (Ezran et al., 2002). In this sense, software reuse has been a goal in software engineering, where software components should be available in families arranged according to precision, robustness, generality and performance (McIlroy, 1968). A software reuse program goes beyond simplistic software reuse approaches, characterized by low reuse and focused on source code only. Many software artifacts have potential for reuse, such as: requirements, architecture, tests, and so on, allowing reuse in all levels of abstraction. Nevertheless, a software reuse program needs significant efforts to change culture, organization, processes and infrastructure. The scope of these changes is effectively a Business Process Reengineering (BPR) of the software development process and organization (Griss, 1995). In (Jacobson et al., 1994; Frakes and Isoda, 1994) are summarized key factors and impediments facing BPR efforts and barriers to large scale reuse, considering management support, incremental adoption, explicit process change, attention to soft factors, and pilot projects. Figure 2.1 summarizes the more commons required steps towards and the degrees of changes for achieving systematic reuse.. Figure 2.1 Road to Systematic Reuse (Griss, 1995).. Since McIlroy’s seminal paper, the software engineering academic and industrial research community has developing many software reuse practices, such as: Component-. 9.

(31) 2.3. SOFTWARE PRODUCT LINES. Based Development (McIlroy, 1968); Repository Systems (Sagawa, 1990); Domain Engineering (Neighbors, 1980); Software Product Lines (Clements and Northrop, 2001); and so on Software organizations can adopt any these software reuse practices and some combinations one. However, just apply software reuse practices do not assure the reuse benefits, since the return on investment from software reuse will depends on reuse level employed on the software organization as can be seen in Figure 2.2.. Figure 2.2 Staged adoption of software reuse (Griss, 2010).. 2.3 Software Product Lines Some companies introduced the common platform concept for production of their different types of products, planning beforehand which parts will be instantiated on the different product. A systematic combination between mass customization (Northrop, 2002) and platform-based development allows reusing a common base of assets and to develop products in close accordance with customer needs. This approach is called Software Product Line Engineering, a software development paradigm (Pohl et al., 2005). SPL is defined as “a set of software-intensive systems sharing a common, managed set of features that satisfy the specific needs of a particular market segment or mission and that are developed from a common set of core assets in a prescribed way.” (Clements and Northrop, 2001). SPL engineering emerges as a viable and important development paradigm allowing companies to achieve improvements in time to market, cost, productivity, quality, and. 10.

(32) 2.3. SOFTWARE PRODUCT LINES. other business drivers (Pohl et al., 2005; Denger and Kolb, 2006). Software product lines can also enable rapid market entry and flexible response to software development organizations. As can be seen in Figure 2.3, from economic point of view, SPL requires more initial investment than single systems, but it allows lower costs per systems after the third one (Linden et al., 2007).. Figure 2.3 Economics of software product line engineering (Linden et al., 2007).. SPL techniques consolidate and capitalize on commonalities throughout the product line. In SPL, the variations among the products are formally managed and controlled through variability management activities (Pohl et al., 2005). To adopt SPL means explore the commonalities and variabilities among products to strategically reuse software artifacts, adopting an architecture-centric development, and following a two-tiered organizational structure (McGregor, 2002). This identification of commonalities (common features for the SPL members) and variabilities (difference among members) is crucial for product diversification because these concerns are essentials for product lines. The key definition of architecture-centric development determines how products are derived efficiently from the core assets1 . The product line architecture has to deal with variations of features, requirements and products in order to allow the derivation of several 1 Core Assets “are the assets that form the basis for the software product line. They often include, but are not limited to, the architecture, reusable software components, domain models, requirements statements, documentation and specifications, performance models, schedules, budgets, test plans, test cases, work plans, and process descriptions”. The core assets base also can be called the “platform” (Clements and Northrop, 2001).. 11.

(33) 2.3. SOFTWARE PRODUCT LINES. different products of scope of the product line (Rommes and America, 2006), so that, it can be configured to match a diverse range of requirements.. 2.3.1 SPL Essential Activities Basically, on SPL effort requires three essential activities, as can be seen in Figure 2.4. The activity responsible for developing reusable assets, named Core Asset Development (CAD), the Product Development (PD) is responsible for developing products based on adoption of required assets of product line platform, and the third one, Management responsible for organizational management (Linden et al., 2007). These activities are detailed as follows.. Figure 2.4 Essential product line activities (Northrop, 2002).. • Core Asset Development: Define the basis of common assets that compose the platform of product lines (Linden et al., 2007). Also known as domain engineering, the main goal is to map commonality and the variability of the product line, establishing the reusable artifacts and then production capability for products (Pohl et al., 2005). Core assets include, but are not limited to, the architecture and its documentation, specifications, software components, tools such as components or application generators, performance models, schedules, budgets, test plans, test cases, work plans, and process descriptions (Clements and Northrop, 2001). Figure 2.5 shows the Core Asset Development work.. 12.

(34) 2.3. SOFTWARE PRODUCT LINES. Figure 2.5 Core Asset Development (Northrop, 2002).. • Product Development: Also known as application engineering, has as goal to create individual products (customized) from core assets (reusing). This activity is also know as product instantiation and must be in accordance with the production plan build in the core asset development activity. By production plan is possible to produce products that meet their respective requirements, as defined in the product line scope. This activity has also the responsibility to provide feedback on any problem or deficiency encountered in the core assets. Figure 2.6 shows the Product Development. • Management: This activity is responsible for the SPL adoption success. Management involves essential activities carried out at technical and organizational levels to support the software product lines process (Ahmed et al., 2009). It is due to the fact that the core asset development and the product development activities are highly interactive, and this iteration requires carefully management (Clements and Northrop, 2001). However, Management, it is not considered as a separated activity or phase, and it is highlighted during the whole product line engineering (Pohl et al., 2005).. 2.3.2 Software Product Lines Practice Areas A practice area is a body of work or a collection of activities that an organization must master to successfully carry out the essential work of a product line. Practice areas help to make the essential activities more achievable by defining activities that are smaller. 13.

(35) 2.3. SOFTWARE PRODUCT LINES. Figure 2.6 Product Development (Northrop, 2002).. and more tractable than a broad imperative such as "develop core assets." They provide starting points from which organizations can make (and measure) progress in adopting a product line approach for software (Northrop and Clements, 2007). The SPL framework introduced by Software Engineering Institute (SEI) recommends some practice areas for achieving success on adoption of the SPL paradigm. The SPL practices areas, instituted by SEI, are organized in three categories. Figure 2.7 shows the relationship among categories of practice areas. Next, the categories and some practice areas are described.. Figure 2.7 Relationships among Categories of SPL Practice Areas (Northrop and Clements, 2007).. 14.

(36) 2.4. SPL VARIABILITY MANAGEMENT. Software Engineering Practice Areas Software engineering practice areas are the basic activities for applying the appropriate technology to create and evolve both core assets and products in the SPL context. The category Software Engineering is composed of the following practices areas (Northrop and Clements, 2007): Architecture Definition; Architecture Evaluation; Component Development; Mining Existing Assets; Requirements Engineering; Software System Integration; Testing; Understanding Relevant Domains; and Using Externally Available Software. This work (Inspection on SPL artifacts) is related to the Testing practice area, defined by SEI. According to the SEI (Northrop and Clements, 2007), the Testing practice has two main functions: first, helping to identify faults that lead to failures so they can be repaired and secondly, determining whether the software under test can perform as specified by its requirements. Technical Management Practice Areas This category involves the management practices that are necessary for the development and evolution of both core assets and products, such as: Configuration Management; Make/Buy/Mine/Commission Analysis; Measurement and Tracking; Process Discipline; Scoping; Technical Planning; Technical Risk Management; and Tool Support. Organizational Management Practice Areas Organizational management practices are those practices that are necessary for the orchestration of the entire product line effort. In this category, the following practices are included: Building a Business Case; Customer Interface Management; Developing an Acquisition Strategy; Funding; Launching and Institutionalizing; Market Analysis; Operations; Organizational Planning; Organizational Risk Management; Structuring the Organization; Technology Forecasting; and Training.. 2.4 SPL Variability Management By definition, software variability is “the ability of a software system or artifact to be efficiently extended, changed, customized or configured for use in a particular context” (Svahnberg et al., 2005). Variability is the tangible difference among the products in the SPL context and it can be revealed and distributed in every phase of development. 15.

(37) 2.4. SPL VARIABILITY MANAGEMENT. cycle of the product line. Variabilities are the key to understand SPL, since it enables the development of customized products by reusing predefined and adjustable assets (Pohl et al., 2005). During Core Asset Development, variability is introduced in all domain engineering artifacts (requirements, architecture, test cases, and so on), and it is exploited and materialized during Product Development by instantiating applications assembled to the specific needs of different customers. Variability can be represented through variation points and variants. Variation point is the representation of a variability subject (variable item of the real world or a variable property of such an item) within the core assets, enriched by contextual information; the Variant is the representation of the variability object (a particular instance of a variability subject) within the core assets (Pohl et al., 2005). In the SPL context, a set of features normally is used to model the variabilities, which usually represent reusable requirements. Kang et al. (Kang et al., 1990), define feature as a prominent and distinctive aspect or characteristic that is visible to several stakeholders, such as end users, domain experts, developers, and so on. Moreover, (Kang et al., 1990) proposed a model called feature model as part of the Featured-Oriented Domain Approach (FODA). This model is used for modeling common and variable functionalities among the products of the SPL. Furthermore, there are other approaches to model commonalities and variabilities in SPL, such as (Ardis and Weiss, 1997); (Bayer et al., 1999); (Clements and Northrop, 2001); (Atkinson et al., 2002); (Muthig and Patzke, 2002); (Gomaa, 2004); (Pohl et al., 2005); (Alves et al., 2005). The variability management involves issues, such as: variability identification, representation, binding, and control (Alves et al., 2005). The variability binding indicates the milestone that the variants related with a variation point will be realized. The literature proposes some binding times (e.g.: specification, compile, execution, and link time) and mechanisms (e.g.: inheritance, parameterization, and conditional compilation) that are appropriate for different variability implementation environment and schemes. Nevertheless, just to implement the SPL essential activities, explained previously do not ensure the success. The mechanisms of variability management require tailored software quality approach for account the SPL benefits (McGregor et al., 2004). This SPL issue is discussed in the next session.. 16.

(38) 2.5. SOFTWARE PRODUCT LINES QUALITY. 2.5 Software Product Lines Quality The Quality or Quality Assurance, as in traditional software development paradigm, has an important function on SPL and needs to be applied along all over the SPL lifecycle under penalty of not achieving the SPL benefits. The software quality assurance community has been defined and investigated many aspects about qualities techniques, such as software review, inspection, and testing. Based on them, these there are many combinations and approaches. In order to ensure that the artifacts produced are in conformance with the required quality levels, techniques such as testing and software inspection can be applied on SPL artifacts. In general, in the SPL area is more common to see quality approaches (McGregor, 2001; Neto et al., 2010; Machado et al., 2011) for executable assets. Nevertheless, software inspection can increase the reliability of the artifacts produced for reuse (Denger and Kolb, 2006). Although some authors (Boehm, 1987; Gilb and Graham, 1993; Wiegers, 2002) mention the importance of software inspections in every phase of software development, few studies establish the relationship among inspections and SPL. This issue is discussed in (Denger and Kolb, 2006), and highlighted by (Babar et al., 2010), which indicates a lack of substantial literature detailing quality assurance techniques from the point of view of variability in software engineering.. 2.6 Summary Software Product Lines is an approach to software reuse and it has proven applicability in a broad range of situations (Neto, 2010), producing impressive results (Weiss et al., 2006). The major distinction between SPL engineering and traditional software engineering is the presence of variation in some or all of the software assets. This chapter presented and discussed the foundations for Software Product Lines Engineering. It contextualized the SPL field and showed the main definitions and peculiarities. Variability management is far from trivial or easy task. It is a discipline on the whole development life cycle, which has the important and difficult mission to determine the most suitable practices, strategies to be adopted, and monitoring the results. Next chapter presents an overview about software inspection, discussing the main concepts, techniques, roles, approaches and so on, in order to define the bases for this dissertation.. 17.

(39) 3. Foundations on Software Inspection. 3.1 Introduction The Software Inspection ideas were introduced in 1976, by Fagan (Fagan, 1976). According to (Wiegers, 2002), software inspections are one of the most efficient software quality assurance techniques, especially for the earlier life-cycle phases, when identifying and correcting non-conformities cost less than in later phases (Tian, 2001). It is a static verification and quality assurance technique, performed on each software artifact created during the software development (e.g. requirements, design, test cases, code, and so on) (Tian, 2001). Fagan (Fagan, 1986) pointed out that software inspections are less expensive if defects are detected and repaired at the stage where they are inserted as opposed to when they are found and removed during testing or operational stage. Software inspection is also a type of software review technique. The terminology used for different types software review techniques is often imprecise and it can leads to confusion. In this work, we adopted the terminology defined by IEEE standard 1028-1997 (IEEE, 1998) as described following: • Software Review: “a process or meeting during which a software product is presented to project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval”. • Software Inspection: “a visual examination of a software product to detect and identify software anomalies, including errors and deviations from standards and specifications”. Software inspection is considered a well-structured technique, but to reach successful results, and promised benefits, the approach depends on three core dimensions: Technical,. 18.

(40) 3.2. TECHNICAL DIMENSION SOFTWARE INSPECTION. Organizational and Managerial. This chapter presents an overview regarding to these three conceptual dimensions of software inspection: The technical dimension of inspection shows the concepts and discusses the main inspection processes and reading techniques. The organizational dimension points out and discusses the aspects related to the inspection team, structure and environment. The last dimension treats the main points for assessing the results of software inspection approaches. This chapter is structured as follows: Section 3.2 introduces the technical dimension of software inspection. In Section 3.3 is described the main processes for software inspection. Section 3.4 details some reading inspection techniques. Section 3.5 introduces and discusses the Organizational dimension of software inspection. Section 3.6 discusses the managerial dimension of software inspection and presents some reports about one and, finally, Section 3.7 summarizes the dimensions pointed out and discussed in this chapter.. 3.2 Technical Dimension Software Inspection The software inspection technique originally began on hardware logic and moved to design and code, test plans and documentation (Fagan, 1986). Software inspection can be characterized in terms of its objective, steps or tasks, roles of participants, size team, work product type, output products and the process discipline (Wheeler et al., 1996). In general, the software inspection team examines the documents individually to learn about the work product. The participants attend a meeting with the intended purpose of effectively and efficiently identifying defects on the work product. Next, the defect reports and recommendations are sent to the author of the documents to be repaired. Then, someone representative of the inspection team validates the corrections applied by the author. Depending on the results and software inspection approach established, the work product can be re-inspected. For a systematic approach of software inspection is necessary making some decisions for defining the software inspection technical plan. It is composed by decisions and definitions related to: The products under inspection; Team roles and size; Software inspection process; and the Inspection reading technique (Laitenberger and DeBaud, 2000). Figure 3.1 shows the relation among the aspects of technical dimension (products, roles, processes and reading techniques). The products or work products are the input for software inspection. The term product refers not only to the final delivered software system, but also to any documentation. 19.

(41) 3.2. TECHNICAL DIMENSION SOFTWARE INSPECTION. Figure 3.1 Technical Dimension of Software Inspection (Laitenberger, 2002).. produced in the context of a software development project (Laitenberger, 2002). Besides the work product is the input for software inspection, it determines many of technical plan decisions. The technical plan needs to be aligned with the type, size, and quality level of the work product to be inspected. Some studies (Boehm, 1981; Briand et al., 1998a) support the statement that the use of software inspection on early life-cycle documents can avoid more rework than on later life-cycle artifacts. Software Inspection requires a team who plays defined roles. It is important that people performing the inspection are familiar with the product and knowledge about the inspection process or otherwise they must be trained. Inspection team size is another important issue in a software inspection approach, since it involves teamwork where a group of individuals work together to analyze the product in order to identify and remove defects (Aurum et al., 2002). The complexity of the products has become the driving force behind the creation of teams. It has been found that the quality of the product improves as it is inspected from multiple viewpoints (Laitenberger and DeBaud, 1997). In general, inspection teams are formed from small groups of developers. In (Laitenberger, 2002), for example, the author advocates that two inspectors are more effective than one. Fagan (Fagan, 1976) suggests that four people constitute as a good-sized inspection team. This suggestion also ties with the industrial work done by Weller (Weller, 1993), who reports that four people teams are more effective and efficient than three people ones. In general, it is common teams composed of 4-5 people (Wheeler et al., 1996).. 20.

(42) 3.3. SOFTWARE INSPECTION PROCESSES. 3.3 Software Inspection Processes An important criterion to start an inspection is to adopt a well-defined software process with clear exit criteria based on the software project context and work products (Bisant and Lyle, 1989). The literature reports the Fagan’s software inspection process as the first one, which was proposed with the goal to evaluate code and design work products. However, there are many variations from original software inspection process (Parnas and Weiss, 1985; Bisant and Lyle, 1989; Martin and Tsai, 1990; Knight and Myers, 1993; Gilb and Graham, 1993; Thelin et al., 2004). Next, Fagan’s process is described, as well as, some other ones.. 3.3.1 Fagan’s Inspection Process Fagan’s inspection process was developed at IBM and it defines a formal, efficient and economical method to finding errors in design and code (Fagan, 1976). This process can be applied to any phase of the software development life-cycle, such as: requirements, test, or management as well as to design and code. Fagan advocates that software inspections involve teamwork and are organized by a moderator. Essentially, a set of participants reviews a document aiming to discover defects. He states that the inspection will be successful if all team members play out their roles fully in the process. The steps that compose the process are described following (Fagan, 1976): • Planning: An inspection team is composed and the roles are assigned to each team members; • Overview: An optional stage where the inspection team is informed about the product; • Preparation: Individual reviewers inspect the material independently. The inspection material may be a product of analysis, design (such as entity-relationship diagrams, data-flow diagrams, state-transition diagrams and text specification) or coding activities. It aims to learn the material and to fulfill their assigned roles; • Examination: It is also known as inspection meeting. It does not aim to discuss or evaluate the solution, but to find and organize the defects together. The moderator, who guides the meeting, takes notes and prepares a list of defects. The meeting. 21.

(43) 3.3. SOFTWARE INSPECTION PROCESSES. should not take more than two hours. After the meetings the moderator produces a written report of the meeting to ensure that all issues identified will be addressed in the rework and follow-up; • Rework: The defects, to be verified by the moderator, are corrected by the author. Some products must be reworked and re-inspected several times if necessary; and • Follow-up: The moderator checks and verifies each correction. The original Fagan’s process also defines four roles, which are following described: • Moderator: It is the key person in the inspection who manages the team and coordinates the inspection process. • Author (designer or coder): The programmer who produces the program design and who may also translates the design into code. • Reader: The programmer who paraphrases the design or code during the meeting. • Tester: The programmer who reviews the product from testing point of view.. 3.3.2 Inspection Process Variations Some variations (Parnas and Weiss, 1985; Bisant and Lyle, 1989; Martin and Tsai, 1990; Knight and Myers, 1993; Gilb and Graham, 1993; Thelin et al., 2004) were proposed for the former software inspection process defined by Fagan. The software inspection processes variations addressed different relevant issues, such as: team roles, team size, number of teams, coordination strategy within the team and multiple teams. The main variations proposals are pointed out and described as follow. From the original Fagan proposal, some approaches to inspection suggest other roles that have consensus among some authors (Ackerman et al., 1989; Russell, 1991). Theses suggested roles are not conflicting with the original inspection process (Fagan, 1976), and are described following: • Organizer: The organizer plans all inspection activities within a project or even across projects. • Recorder: The recorder is responsible for logging all defects in an inspection defect list during the inspection meeting.. 22.

(44) 3.3. SOFTWARE INSPECTION PROCESSES. • Collector: The collector consolidates the defects found by the inspectors if there is no inspection meeting. On the other hand, some authors disagree with the inspection process defined by Fagan and they propose other inspection processes. Active Design Review The Active Design Review was proposed in 1985 (Parnas and Weiss, 1985), and it is based on the review process. Basically, the difference for the Fagan inspection is that it performs several brief reviews instead of only one review sequence, where each review focuses on certain part of the product. Different reviewers who have particular expertise and specific responsibilities perform each review, and every review consists of three stages. In the first stage, the reviewers are presented with a brief description of the main product. In the next stage, reviewers study the material following the guidelines. Then, in the third stages the issues raised by the reviewers are discussed in small meetings where only the designer and the particular reviewers met. The reviews are designed according to the properties of the product that the reviewer is examining. Reviewers are also received with a list of questions to be used as a guideline during the review process. In order to guide the reviewers, the issues are classified in terms of inconsistency, inefficiency and ambiguity. Two-Person Inspection Bisant and Lyle (Bisant and Lyle, 1989) defined a formal method for software inspection, which has no moderator, the central required role defined in Fagan’s process. The inspection team consists of two persons: an author and a reviewer. Their general view is that inspections involve too many resources. They conducted experiments to assess the approach productivity and determining whether their productivity improves with this technique during the design or coding phase of product development. The experiments results showed that two-person inspections had immediate benefits in the program quality and program productivity. It pointed out the significant improvements in individual productivity, in particular with novice programmers and less productive programmers. They suggested the use this method in small organizations where the team sizes are relatively small. It can also be applied for transition from small to large team techniques that will have the benefits of establishing consistent data to monitoring the process.. 23.