Conclusion

speed does not clearly determine the ability to detect defects.

At this stage of the improvement initiative there are two relevant factors to con-sider. Firstly, the ability to link the expected improvement in performance to higher level quality or performance goals and secondly the ability to conduct software exper-iments in an industrial setting. The first factor stresses the measurement capabilities of the organization and additionally the ability to define meaningful operational high level goals of the organization. Only when these higher level goals are available and monitored it is possible to rationalize about any possible impact of process changes in high level quality or performance goals and thus drive effective process improvement initiatives. The second factor relates to the ability to plan and execute software exper-iments in line with the experiment design constrains so that inference is not impaired by validity threats. In an industrial setting there are strong limitations ensuring the desirable environment to validate a change to a process.

In the third stage of the improvement initiative the process improvement was de-ployed to the organization and data was collected on the execution of the updated inspection process. The improvement impact on performance corroborated the result from the pilot experiment. A significant increase in the ability to detect defects was achieved when the average review speed decreased to the recommended speed. Ad-ditionally, the impact of the change on higher level project goals was monitored and evaluated. In the perspective of inspection process performance, data collected from the deployment phase shows that for review speeds below 500LOC/hourthe review speed impact on defect detection ability is less conclusive. A’shotgun effect’is present which may indicate that the determinant factor of defect density for review speeds be-low 500LOC/houris other than the review speed.

Once again at this last stage the evaluation of the improvement solution highly demands on measurement capabilities. Process execution data needs to be collected systematically and additional data for monitoring higher level quality and performance objectives also needs to be available. Additionally, the deployment stage of an im-provement initiative demands greatly on change management capabilities to guide a successful adoption of a process change by the organization.

6.5 Conclusion

Software process improvement initiatives in line with the analytical paradigm highly stress the capability of an organization to implement measurement programs in soft-ware engineering. Measurement is present in all stages of a softsoft-ware process improve-ment initiative. Additionally, analysis of process performance is only possible if a process is performed systematically. This implies that processes subject of

improve-ment are well established and executed by the organization. The analytical paradigm also stresses the need to have knowledge on statistics. Being able to apply statistical knowledge and tools is required in all stages of a software process improvement initia-tives, namely when establishing baselines and performance models, when validating results of software experiments and when evaluating the impact of deployment of a process change.

Success of software process improvement also requires that improvement efforts are driven by higher level organizational business goals. These can take the form of quality or process performance goals. Only when improvement efforts are clearly linked to business goals there is the possibility to compare the effectiveness of different improvement solutions and validate the real impact on organizational performance.

An industrial setting is characterized by limitations in ensuring desired levels of control of the operational environment and thus conducting sound software experi-ments is challenging. Resources are most of the times limited, whether by opportunity or by number. Opportunity issues occur when the desired resources are not avail-able, e.g., the need to have a project using a specific programming language, having a experienced software developers available or having enough projects available for per-forming a specific software life-cycle phase within a desired time window. Limitation in number is when the desired number of subjects are not available or is not possible or feasible to have the minimal number of repetitions for a specific experiment. As result software engineering experiments in an industrial setting are most of the times limited by threats to validity that limit the soundness of possible claims resulting from engineering experiments. This is highly relevant if the goal is to derive solid software engineering knowledge. However, the focus of software process improvement is in fact improving the process and experiments provide a mechanism to decrease the risk of introducing changes that may fail to have the desired impact on performance.

Software experiments, even if limited by some validity threats, are preferable to the scenario where no experimentation is considered, however there is a cost associated to experimentation. In the end a decision to perform an experiment is one of a trade off between risk reduction and cost.

Chapter 7

Modelling Large Software Process Systems

Contents

7.1 Process modelling . . . 170 7.2 Related work . . . 172 7.3 Software process design and implementation demonstration case 180 7.4 Analysis and discussion of results . . . 187 7.5 Conclusion . . . 188

Improvement models typically prescribe a wide set of practices and the decision to adopt such models typically results in an Organizational Standard Set of Processes that are dif-ficult to manage. Abstraction is a technique used in Computer Science to manage com-plexity of systems. In this chapter a meta-model is proposed to support the definition of process architectures along with a set of transition rules, to refine architectures into low level, ready to enact, process specifications by allowing the transition between two levels of abstraction. A demonstration of the use of the meta-model is documented for creating a process architecture along with the transition to low-level process specifications.

169