SAFE COMPOSITION VERIFICATION FOR CONFIGURATION KNOWLEDGE-BASED SOFTWARE
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 39 A∧(B ⇒A)∧(C ⇒A)
. According to this FM, the 4 valid product configurations that satisfy the domain constraints are the following:
Product Configuration 1. A∧¬B∧¬C
Product Configuration 2. A∧B∧¬C
Product Configuration 3. A∧¬B∧C
Product Configuration 4. A∧B∧C
B0'57$0-!./$0112%3
!""#$"%&'()*+,#,-! ./0""!
1 !"2#3$1
. !"2#3$.
1%"%. !"2#3$104,.
5#67+(#, 88
!"2#3$.9%./0""!
./0""!
!"2#3$19%!"2#3$.
!"
!
1 .
#$
Figure 3.6 Toy SPL Example, to illustrate propositional logic codification rules.
Figure3.6also presents the CK for the toy SPL. It has one item for each feature, and a final item associating the feature expressionB∧CtoAspectBandC. The CK can also be represented by a propositional formula. Implementation constraints are represented by the provided and required interfaces. Thus, for each item of the configuration knowledge, we generate one proposition for the provided column and another proposition for the required column, as illustrated in Figure3.7. For example, for the feature expressionB∧C of the CK in Figure3.6, the provided proposition is given by (B∧C)⇒AspectBandC.
Therefore, following the rules of Figure3.7, the complete mapping of feature expres-sions to provided interfaces for a CK is the conjunction of the provided propositions for each CK item. For the configuration knowledge of the toy SPL (Figure3.6), this is given by the formula below:
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 40
!""#$"
!"!"# $
% &'"(
)*+,-./.
!
"#$%
0/12-*/.
"#%
&'&(
)*+,-./."!"!"#3$$$$$$)*$!"+,$⇒$!
0/12-*/."!"!"#3$$$$$$)*$!"+,$⇒$)"$#$%, )*+,-./."%3$$$$$$$$$$$-$⇒$)"$#$!, 0/12-*/."%3$$$$$$$$$$$-$⇒$4*2/
)*+,-./."563$))-$⇒$)"$#$!,,$#"))*$!"+,$⇒$!,, 0/12-*/."563$))-$⇒$4*2/,$#"))*$!"+,$⇒$)"$#$%,,,
5+784*9-748#:3$)*+,-./."56$⇒$0/12-*/."56
Figure 3.7 Propositional logic codification rules for CK.
provided: (A⇒ClassA)∧(B ⇒AspectB)∧(C⇒AspectC)∧((B ∧C)⇒ AspectBandC).
So, for the example of the CK item in the fourth row in Figure3.6, if (B∧C) is evaluated to true for a product configuration, this proposition is simplified to AspectBandC. If evaluated to false, the proposition is simplified to true. Given a product configuration, the formula above is simplified to the conjunction of the provided interfaces for the CK items where feature expressions evaluate to true. The CK items whose feature expressions evaluate to false are simply removed from the proposition, since they are simplified to true.
We map the required interfaces similarly. For example, going back to the CK item on the fourth row in Figure3.6, we see that it states that if bothBandCfeatures are jointly selected in a product configuration,AspectBandAspectCare required. So, the required proposition for that item is given by (B∧C)⇒(AspectB∧AspectC). In the same way that we do for the provided interfaces, the complete mapping of feature expressions to required interfaces for a CK is the conjunction of the required propositions for each CK item. In the case of the CK of the toy SPL (Figure 3.6), this is given by the following
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 41 formula:
required: (A⇒true)∧(B ⇒(AspectC∧ClassA))∧ (C⇒ ClassA)∧((B ∧C)⇒(AspectB∧AspectC)).
This way, as with the provided proposition, given a product configuration, this formula is simplified to the conjunction of the required interfaces for the items whose feature expressions evaluate totrue. The items whose feature expressions are evaluated to false are removed from the proposition, since they are simplified to true.
Finally, we relate both propositions to represent the constraints of the configuration knowledge (constraintsCK). This formula is given by the provided proposition implicating in the required proposition. It follows the idea that, for a product to be considered valid, required interfaces should be a subset of the provided interfaces.
constraintsCK :provided⇒required.
For example, the constraints for the configuration knowledge of Figure3.6 are translated into the following propositional formula:
((A ⇒ ClassA) ∧ (B ⇒ AspectB) ∧ (C ⇒ AspectC) ∧ ((B ∧ C) ⇒ AspectBandC))
⇒
((A ⇒ true)∧(B ⇒ (AspectC ∧ClassA))∧(C ⇒ ClassA)∧((B ∧C) ⇒ (AspectB∧AspectC))).
So, the intuition is that a product is considered valid when, given a product configuration, this formula is satisfied.
For example, for the product configurationA∧B∧C, theconstraintsCK proposition is simplified to
(ClassA∧AspectB∧AspectC ∧AspectBandC)
⇒
(true∧(AspectC ∧ClassA)∧(ClassA)∧(AspectB∧AspectC)).
In this proposition we see that all aspects and classes that are required for this product (appearing on the right side of the implication) are being provided (appearing on the left side of the implication). Therefore, it evaluates totrue.
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 42 On the other hand, if we evaluate the product configuration A∧B ∧¬C, the con-straintsCK proposition is then simplified to
(ClassA∧AspectB∧true∧true)
⇒
(true∧(AspectC ∧ClassA)∧true∧true).
For this case, we see that AspectC, which is required for this product and therefore appears on the right side of the implication is not being provided — it does not appear on the left side of the implication. So, we can conclude that this proposition is impossible to be proved. We have AspectC in the consequent, but not in the antecedent. As the antecedent is not false, we cannot prove.
Recalling the definition presented on the previous section, a valid SPL is one where all products are valid. The semantics of the FM (semanticsFM) denotes the set of valid product configurations and the CK represents the implementation constraints with required and provided interfaces (constraintsCK). Therefore, we relate both models to perform the verification of safe composition for a CK-based SPL. This way, a FM that satisfies all implementation constraints is a valid SPL. This is equivalent to verifying if each product satisfies implementation constraints. The proposition we write to perform this verification is the following:
semanticsF M ⇒constraintsCK
If this proposition is valid, the SPL is valid. If not, there is a problem with at least one of the SPL products.
3.3.2 Formalization
The previous section describes the overall idea of our approach to automatically verify safe composition of CK-based SPLs. Based on the propositional logic codification, there are many ways that we can actually perform the verification using SAT solvers. In this work, we have developed an encoding using Alloy 4 [Jac06]. We chose Alloy due to having a formal semantics, and by our previous experience using it in the context of FMs and SPLs [GMB06]. Besides, the Alloy Analyzer provides tool support that allows automatic
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 43 analysis be performed, that can be performed with many off-the-shelf SAT solvers. This section describes how we encode our approach using Alloy.
To keep specifications close to logical propositions, we use an idiom, illustrated on Listing 3.4. We define an abstract signature Bool, and two signatures extending it, TrueandFalse, which are disjoint signatures, each one representing one atom. By using an abstract signature for Bool, we guarantee that there is noBool that is neitherTrue orFalse. The predicates assign a given valueb of typeBooltoTrueorFalse. We need to specify such an idiom because Alloy does not handle boolean types natively.
Listing 3.4 Alloy Idiom used in our codification.
abstract sig Bool {}
one sig True, Falseextends Bool {}
pred isTrue[b: Bool] {b in True} pred isFalse[b: Bool]{ b inFalse }
Therefore, when using this idiom, formula encoding in Alloy is straightforward from the propositions, as discussed in the following subsections.
3.3.2.1 Feature Models
To encode a FM, we need to first declare features as subset signatures of Bool, so they can assume valuesTrueorFalse. This follows the concept of a feature being selected or not, in a product configuration. After that, we create a new predicate, which represents the semantics of the feature model. The rules for creating this predicate are analogous to the propositional logic codification rules presented in Section 2.1.1. The main difference is the addition of the isTrue[] predicate surrounding each feature, to comply with the idiom shown on Listing 3.4. An example for the toy FM in Figure 3.6 is illustrated in Listing 3.5.
Listing 3.5 Alloy codification of the feature model in Figure3.6.
one sig A, B, Cin Bool {}
pred semanticsFM[] {
isTrue[A] and (isTrue[B]⇒ isTrue[A]) and (isTrue[C] ⇒isTrue[A]) }
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 44 3.3.2.2 Configuration Knowledge
The CK, likewise, is encoded by first declaring the assets names as subset signatures of Bool. To keep specifications elegant, we break the definition of the predicate representing the constraints of the configuration knowledge using two predicates: one for provided propositions and other for required propositions. This also helps to use the formulae in other places, if needed. As with the FM, codification rules work in the same way as in Figure3.7. An encoding of the toy SPL CK of Figure3.6 is given on Listing 3.63.
Listing 3.6 Alloy codification of the configuration knowledge in Figure3.7.
one sig ClassA, AspectB, AspectC, AspectBandCin Bool {}
pred provided[] {
( (isTrue[A])⇒ (isTrue[ClassA]) ) and ( (isTrue[B]) ⇒(isTrue[AspectB]) ) and ( (isTrue[C]) ⇒ (isTrue[AspectC]) )and
( (isTrue[B] and isTrue[C])⇒ (isTrue[AspectBandC]) ) }
pred required[]{
( (isTrue[A])⇒ (isTrue[True]) ) and
( (isTrue[B]) ⇒(isTrue[AspectC] and isTrue[ClassA]) ) and ( (isTrue[C]) ⇒ (isTrue[ClassA]) )and
( (isTrue[B] and isTrue[C])⇒ (isTrue[AspectB] and isTrue[AspectC]) ) }
pred constraintsCK[]{ provided[] ⇒ required[]
}
3.3.2.3 The validSPL predicate
Finally, we need to verify if a SPL is valid or not. A valid SPL is one in which all products are valid. A product is valid when all required assets are provided. The predicate validSPL[]represents the proposition that must be verified to check safe composition of
3We assume that signatures for features A, B and C are previously declared, as in the previous listing.
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 45 a CK-based SPL. The semantics of the FM implying on the CK constraints means that all SPL products must satisfy implementation constraints, as already discussed.
Listing 3.7 Alloy predicate for verifying safe composition of a CK-based SPL.
pred validSPL[] {
semanticsFM[] ⇒ constraintsCK[]
}
Listings presented in this section, when composed together, form an Alloy specifica-tion for a SPL. In this case, the toy SPL specificaspecifica-tion presented in Figure3.6. Chapter5 presents evaluation of this verification approach using an approach to automatically gen-erate this specification from the MobileMedia SPL assets [FCS+08]. With such a specifi-cation, we are able to determine whether a SPL is valid or not. However, when a SPL is invalid, it is desirable to obtain information about where and how errors happen, beyond only stating that a SPL is valid or not. The next section describes how we actually verify if a SPL is valid or not, and how can we obtain more detailed information when performing this verification.
3.3.3 Verification in Practice
The previous section describes the formalization of the models that describe a SPL into Alloy 4. In Section 3.3.2.3, we discuss the validSPL[] predicate, which is used to verify whether a given SPL is valid or not. Nevertheless, in addition to that, when it is deter-mined that a SPL is invalid, it is useful to provide further detail. Information such as the list of products that break safe composition of a SPL, and how can we determine what is the error for each of these products are desirable. This section discusses implementation of a tool that provides detailed information when verifying safe composition of CK-based SPLs based on the Alloy encoding described in Section 3.3.2.
3.3.3.1 Verifying Safe Composition of a SPL
To determine whether a SPL is valid or not, we must check thevalidSPL[]predicate. We discuss, in Section2.3, that developers must specify a scope to perform analysis using the Alloy Analyzer, since it uses finite scope check. This means that the analysis is complete
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 46 up to the scope defined, although there is no guarantee about the property holding for larger scopes. In our case, however, when using the idiom presented in Section3.3.2, the scope is defined by the parent signature, which is Bool. Every subsignature is either True or False. Notice that all other signatures are subset signatures of True orFalse.
Therefore, the scope of Bool does not need to be bigger than 2, in any scenario, since the specification is based on propositional logic, which is decidable.
To perform the verification, we declare an assertion verifySPL. The body of this assertion consists of the validSPL[] predicate. We then check it for the scope of 2. In our case, since the scope is well delimited, the analysis is sound and complete. Thus, Alloy works as a theorem prover in this context. The result of this check is always a proof. If a counterexample is given, it is false, which means that the SPL is invalid, according to Definition 4. If no counterexamples are given, it is true, which means that the SPL is valid, according to Definition 4. Listing 3.8 illustrates how we use Alloy to perform this verification.
Listing 3.8 Alloy assertion for verifying safe composition of a SPL.
assert verifySPL { validSPL[]
}
check verifySPL for 2
When performing theverifySPLcheck for the toy SPL specification from Figure3.6, a counterexample is found — meaning that the verified SPL is invalid. Figure 3.8 illus-trates a possible counterexample for the toy SPL. This counterexample is an assignment of signatures that violates thevalidSPL[]predicate. The rectangle in the left, represents signatures assigned to False and the one in the right, represents signatures assigned to True. If we examine the feature assignment, we notice that this is the product configu-ration
A∧B∧¬C,
since A and B are assigned to True and C to False. From previous section, we learn that the set of provided interfaces for this product configuration is
{ClassA,AspectB},
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 47 and the set of required interfaces is
{AspectC,ClassA}.
Figure3.8shows thatAspectCis assigned toFalse, when it should be assigned toTrue, for this product configuration.
Figure 3.8 Possible counterexample when performing theverifySPL check for the toy SPL specification.
Next we explain how can we obtain further information about which products are invalid.
3.3.3.2 Retrieving invalid products
As we discuss in Section2.3, when Alloy checks an assertion, it looks for a counterexample of the given assertion. In our case, a counterexample is an assignment of features and assets to subsignatures of Bool, namely True and False. Based on a counterexample, we can obtain a product configuration that breaks safe composition for a given CK-based SPL. For instance, on the counterexample of Figure 3.8, we extract the product configuration as previous section explains.
So, given that we know an invalid product, we evaluate the CK for the sets of required and provided interfaces. With both sets, we compare them against each other, to find what dependency has not been satisfied. For the example of the toy SPL from Figure3.6, the problem is that there exists an unsatisfied dependency with the asset AspectC. With this information, we can look in the CK for which feature expression provides that de-pendency, and find out whether it was a mistake when modeling the domain, building the CK or an implementation problem. In this case, we notice that AspectCis provided by Conly. A possible solution is to change the feature expression of this CK item to B
∨ C.
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 48 Since assets are also modeled asBool, it is possible that there is more than one coun-terexample for the same product configuration. For example, Figure 3.9 shows another possible counterexample for the product configurationA∧B ∧¬C. If we analyze both counterexamples, we notice that the only thing changing, for this case, is the value that some assets assume, whetherTrueorFalse. In the example from Figures3.8and3.9, the product configuration does not change. However, AspectBandC is False in Figure 3.8 and True on Figure 3.9.
Figure 3.9 Another possible counterexample when performing the verifySPL check for the toy SPL specification.
To find all product configurations that break safe composition of a SPL, we can loop through all counterexamples. However, as the example above demonstrates, it is possible that redundant counterexamples might happen while looping. It is important to notice that these are indeed different counterexamples. They are redundant to our context, since we are only interested in product configurations. So, counterexamples are redundant for a specific product configuration. The number of these depend on the size of the SPL. A SPL with more assets will have more redundant counterexamples. The quantity of assets can lead to thousands, or even millions of these redundant counterexamples. We are interested only in which products break safe composition of a SPL. With this in mind, instead of looping through all counterexamples, we can use an iterative approach to retrieve all invalid product configurations of a SPL. So, given a SPL specification such as the one we present in Section 3.3.2, we follow a simple algorithm to retrieve invalid products.
The general intuition is that we run the assertion check perform repeatedly, updating the validSPL[] predicate to remove the already identified invalid product configurations.
For example, for the toy SPL of Figure 3.6, we first check theverifySPL assertion.
If there are no counterexamples, the SPL is valid, and no further steps are necessary.
However, this is not the case for the toy SPL. Given that we find a counterexample when checking the verifySPL assertion, such as the one from Figure3.8, we extract its product configuration. Then, we do not loop through the remaining counterexamples.
3.3 VERIFYING SAFE COMPOSITION OF CK-BASED PRODUCT LINES 49 Since we are not interested in other counterexamples for the same product configuration, as Figure3.9demonstrates that can happen, we need to remove it from the set of products we are interested in. To achieve this, we first generate a new predicate to represent this product configuration, such asproduct1[] below.
pred product1[]{
isTrue[A] and isTrue[B]and not(isTrue[C]) }
Then, the validSPL[] predicate must be updated to exclude this product configura-tion from the set of products we are interested in checking. So, instead ofsemanticsFM[]
⇒ constraintsCK[] we now have:
pred validSPL[]{
(semanticsFM[] and not(product1[]) )⇒ constraintsCK[]
}
The updated validSPL[] predicate guarantees that the product configuration that we already identified as invalid is not taken into account anymore. After updating this predicate, we check the assertion again. Checking the assertion again for the toy SPL gives us no counterexamples. Then, the algorithm is finished. The list of products that make the SPL invalid is given by the product predicates generated. In the case of the toy SPL example, there is only one invalid product, given by product1[]. If there is a counterexample, we go back to:
1. extracting the product configuration;
2. generating a new predicate for it;
3. updating the validSPL[] predicate, appending the negation of the product predi-cate to the existing assertion.
Following these steps, it is possible to get all products that break safe composition of a given SPL encoded with our approach. This process is fully automated in Java, using the Alloy API, and we use it to evaluate our approach, as Chapter 5 discusses.
It can loop through all invalid products, then it executes according to the quantity of product configurations that are invalid, according to Definition 3. Or we can perform