Sub-Characteristic Accuracy Quality
Attribute Correctness
Goal Evaluates the percentage of the results that were obtained with precision Question Based on the amount of
tests executed, how much test results return with
Usability Sub-Characteristic Configurability
Quality Attribute Effort to Configure
Goal Evaluates the time necessary to configure the component.
Question How much time is needed to configure the component in order to work correctly in a system?
Metric Time spent to configure correctly
Objective Metric
Subjective
Each technique can be
measured in different ways and complexity, using
different tools, techniques, methods and processes.
and Modularity analyzes
EMM level I
Technique Coupling, Cohesion, Simplicity, Reusability, Modularity analyzes using Checkstyle tool[2]
Goal Evaluates the internal source code of the component
Question Is the Checkstyle tool efficient enough to measure those attributes?
Metric Analysis of the results and coverage of the Tool
Interpretation If the tool can mine these kinds of information from the source code and present them to be analyzed, it is good to evaluate the component. On the other hand, if it is not enough to evaluate some kind of attributes, other tool should be use to complement or to substitute this one. If this tool is good to evaluate the component, an analysis of the metrics collected in the tool can be used to define those attributes from the component. The idea is that the component should have: less coupling, high cohesion, high modularity, [2] Checkstyle – http://checkstyle.sourceforge.net
The idea is to obtain feedback from those metrics in order to improve the activities and
steps to assure the efficiency and efficacy of the process
Goal Adequately evaluate the software component embedded
Question Could the evaluation team evaluate
everything they planned to execute using the documents developed during the process activities?
Metric Total documented functionalities / Total component functionalities (or Total measurement accomplished)
Interpretation 0 <= x <= 1; which closer to 1 is better Component Certification Process
Goal Analyze the usability of the templates provided
Question Has the template helped during the certification development?
The evaluation team should define metrics as much as they think interesting in
framework for software reuse (Almeida et al., 2004).
It define a set of activities in order to guide the evaluation team during the component evaluation, and
It could be repeatable and reproducible, each activity contains a well-detailed description:
inputs and outputs,
mechanisms to execute, and
to control.
A set of works from literature which includes processes for software product evaluation and processes for software component
assessment aid during the definition of this process (McCall et al., 1977) , (Boegh et al., 1993), (Beus-Dukic et al., 2003), (Comella-Dorda et al., 2002)
(Ross, 1997).
The evaluation team is the main responsible for executing this
process and should be carefully defined in order to assure that the This information is important in the case that the
component is approved and, much more interesting if the component is rejected.
• short time to market, and
• high quality.
Assessment and evaluation of software components has become a compulsory and crucial part of any CBSD Assessment and
evaluation of software components lifecycle.
To properly enable the evaluation of embedded software components, supplying the real necessities of the embedded system design of building system fast, cheap and high quality systems, an Embedded software Component Quality
addressed by CBD approach
Definition (problem, objective and goals), Planning (design,
instrumentation and threats), Operation (measurements are collected)
, Analysis and Interpretation (data are analyzed and evaluated), Presented and Packaged (results are presented and packged) Definition, Planning activities will be presented.
The complete experiment study will be accomplished and described in next year.
Analyze the capacity to evaluate the quality of embedded software component for the propose of evaluating embedded software
component quality verification framework with respect to the efficiency of the framework from the point of view of the researchers, software and quality engineers (customers, evaluators) in the context of the embedded software component quality area.
Training - The training of the subjects using the process will be conducted in a classroom at the university
Pilot Project - Before the study, a pilot project will be
conducted, aiming to detect problems and improve the planned material.
Selection of Subjects - Ten students of pos-graduation at UFPE were selected by convenience sampling.
Subjects - According to its skills and technical knowledge to evaluate the embedded software components.
Instrumentation - the subjects will receive a questionnaires about
his/her education, experience and satisfaction using the framework, and
(ii) EMM; and (iii) subjects difficulty to use the framework).
Hypothesis is to know and to formally state what is going to be evaluated - Null hypotheses, H0: determine that the framework is not efficient and it is very difficulty to use:
Ho’: coverage of the component quality attributes proposed in the CQM X the quality attributes used during the component evaluation < 90%
Ho’’: coverage of the evaluation techniques proposed on the SCMM for the quality attributes defined during the component evaluation < 90%
Ho’’’: %Subjects that had difficulty to understand, follow and use the Software Component Quality Framework > 20%
- Alternative hypotheses: determine that the framework is efficient and it is easy to use:
Ho’: coverage of the component quality attributes proposed in the CQM X the quality attributes used during the component evaluation >= 90%
Ho’’: coverage of the evaluation techniques proposed on the SCMM for the quality attributes defined during the component evaluation >= 90%
and the usability of the framework
Qualitative Analysis - aims to evaluate the difficulty of the application of the framework and the quality of the material used in the study
Randomization - This technique can be used in the selection of the subjects, however, the subjects were selected by convenience sampling
Blocking - not identified the necessity of dividing the subjects into blocks Balancing - it is not necessary to divide the subjects, since there is only
one group
Internal Validity - this study is supposed to have at least between four to
Construct Validity - A relative well-know project will be used (i.e. the experimenter have about three years of experience). This choice
avoids previous experience of making a wrong interpretation of the impact of the proposed framework.
Validity of the Conclusion of the Study - This conclusion will be drawn by the use of descriptive statistic.
The project used in the experimental study was the embedded software components used in cars tracking system developed in a partnership
between industry, academy and a research institute.
• Possible,
• Practically viable, and
• Applicable in the embedded system industry.
To show the challenge and specific requirements of the use CBD approach for embedded systems in different domain;
the realization of a survey related to the state-of-the-art in embedded software component quality and certification research;
To enable the quality component verification and certification for a Robust Software Framework Reuse (Almeida et al.2004), and
To Assess and To evaluate quality of embedded software components
• Practically viable, and
• Applicable in the embedded system industry.
To show the challenge and specific requirements of the use CBD approach for embedded systems in different domain;
the realization of a survey related to the state-of-the-art in embedded software component quality and certification research;
To enable the quality component verification and certification for a Robust Software Framework Reuse (Almeida et al.2004), and
To Assess and To evaluate quality of embedded software components
Component Certification Center - component certification standard for Software Factories
Tool Support - it is needed a tool support in order to aid the usage of the proposed processes, methods, techniques, etc
Risk and Cost/Benefit Management Model the risk and cost/benefit Model analyze if the costs are acceptable and viable or not.
Activity Deadline Delivera
ble Suggest and review metrics for
remaining quality attributes Jan 16 - Jan Define a document/form model for
register embedded component evaluation
30 - Jan
Finalization of case of study for
proposed framework evaluation 13 - Feb Completion of doctoral thesis 1 - Mar Doctoral thesis defense 27 - Mar