Factors influencing free and open-source software adoption in developing countries—An empirical study

(1)

Journal of Open Innovation: Technology, Market,and Complexity 9 (2023) 21–33

Contents lists available atScienceDirect

Journal of Open Innovation: Technology, Market, and Complexity

journal homepage:www.sciencedirect.com/journal/joitmc

Factors influencing free and open-source software adoption in developing countries—an empirical study

Domingos G. Silva

^a,^⁎

, Carlos Coutinho

^b

, Carlos J. Costa

^c

aInstituto Universitário de Lisboa (ISCTE-IUL), ISTAR, 1649-026 Lisbon, Portugal

bInstituto Universitário de Lisboa (ISCTE-IUL), ISTAR, 1649-026 Lisbon, Portugal / Caixa Mágica Software, Lisbon, Portugal

cAdvance/ISEG (Lisbon School of Economics & Management), Universidade de Lisboa, 1200-109 Lisbon, Portugal

A R T I C L E I N F O Keywords:

FOSS characteristics software open source adoption models public sector developing countries

A B S T R A C T

Free Open-Source Software (FOSS) is essentially seen by many organizations as a key pillar for accelerating technological innovation. This study aims to evaluate the extent to which factors such as cost and quality of the system, as well as usability, interoperability, and security, influence the intention to use FOSS. Based on the data obtained from the survey, we propose a new theoretical model that we test and evaluate using the SEM-PLS method. The results suggest a significant relationship between FOSS adoption and factors such as low cost, performance expectation, social influence, and system quality, which was explained on a large scale by safety, interoperability, and usability as the most important factors in adoption decisions. Thus, this paper presents a new model for the adoption of FOSS in countries with high growth potential and aims to be a significant contribution to the scientific community on the state of the art in FOSS organizations in developing countries.

1. Introduction

Free Open-Source Software (FOSS) is essentially seen as a key pillar for many public and private sector organizations due to potential benefits that generally include cost reduction, improved security and interoperability, as well as a substantial increase in system quality and ability (Sanchez et al., 2020). Thus, as with many governments (AlMheiri et al., 2018; Silic and Back, 2017) in developed countries, the authors’ motivation is to more or less disruptively disseminate knowledge about FOSS in developing countries as a viable alternative to improve the development of technological solutions and promote innovation and consequently improve their efficiency and productivity levels, especially at a time when digital transformation has become a concept and a path that can lead to many opportunities. Different ap- proaches consider the demands of providing better service in a rapidly changing digital world. FOSS (e.g., Debian GNU/Linux) is generally defined as a type of free available resource, free of licensing fees or other restrictions, giving users complete freedom to perform, copy, distribute, study, edit and improve the software (Kilamo et al., 2020;

Raymond, 1999; Thankachan and Moore, 2017).

Many studies agree that FOSS and its lack of licensing fees allows for the faster adoption of technology and accelerates the innovation

ecosystem (Umm-E-Laila et al., 2021). Following that, (Sanchez et al., 2020; Pezer et al., 2017) assert that the adoption of FOSS has the potential to provide greater contract freedom and supplier independence, as well as to contribute to the creation of the local software industry, allowing for the development of local software that is qualified for the creation of FOSS-based systems, thereby promoting entrepreneurship and, as a result, the local economy. Similarly, (AlMheiri et al., 2018;

Patino-Toro et al., 2022) consider FOSS to be a valuable and promising resource for reducing digital exclusion, particularly in developing countries, as well as in less industrialized environments that lack the technological and financial resources to promote innovation and invest in efficient public-sector digital infrastructure.

However, despite the fact that FOSS is widely used in developed countries (Europe and North America) (AlMheiri et al., 2018; Patino- Toro et al., 2022), its acceptance has been an unsustainable journey in the majority of developing countries’ public sector organizations.

Thus, the central idea of the article is to assess to what extent factors such as system cost and quality, as well as usability, interoperability, and security, influence the intention to use FOSS. To address this question, the authors propose a theoretical model of FOSS adoption that tests the influence of the aforementioned factors from the perception of ICT (Information and Communication Technology) managers in the

https://doi.org/10.1016/j.joitmc.2023.01.002

Received 15 November 2022; Received in revised form 13 December 2022; Accepted 16 December 2022 Available online xxxx

creativecommons.org/licenses/by/4.0/).

]]

^]]

]]]]]]

⁎ Corresponding author.

E-mail addresses:domingos_silva@iscte-iul.pt(D.G. Silva),carlos.eduardo.coutinho@iscte-iul.pt(C. Coutinho),cjcosta@iseg.ulisboa.pt(C.J. Costa).

(2)

public sector. The research is based on data from questionnaires applied in Angola, an example of a developing country with a low level of literacy regarding FOSS.

The resulting theoretical model was empirically validated using the Structured Equation Model-Partial Least Square (SEM-PLS) method (Ringle et al., 2010). All structures demonstrated satisfactory reliability as well as convergent and discriminatory validity. The findings indicate a positive and significant relationship between FOSS adoption and factors such as low cost, performance expectations, and social influence, as well as system quality, which was explained primarily by system security, interoperability, and usability. The main contribution is the presentation of a theoretical model of FOSS adoption. Further- more, the study is considered innovative because it investigates and proposes for the first time a theoretical adoption model that reflects the factors that influence FOSS adoption and use in a developing country with significant growth potential, shedding light on a new digital innovation and revolution in the public sector.

The rest of the paper is organized as follows.Section 2is a background of the literature and discusses the proposed research model and hypotheses, whileSection 3describes the empirical study. The results and discussions are presented inSection 4, and the study is concluded inSection 5.

2. Literature review and hypotheses development 2.1. Technology acceptance theoretical background

The adoption of FOSS offers a clear cost reduction for developing countries (Syzdykova et al., 2017), as well as some other benefits that have motivated several developed countries to consider their adoption (AlMheiri et al., 2018). However, understanding how and why some technological innovations are accepted while others are rejected has been one of the most challenging and recurring issues in the literature of information systems (IS) (Dwivedi et al., 2019, 2020), in which, as a consequence of this phenomenon, a wide variety of complementary theories were proposed to address cognitive concerns, which are emo- tional and contextual factors inherent in the decision of individuals to accept or reject the adoption of a particular innovation (Rogers et al., 2014; Davis et al., 1989; Venkatesh et al., 2003).

Based on the literature review, the decision-making process or acceptance of an innovation is commonly referred to as a systematic model that begins when an “individual or an organizational unit is exposed to the existence of an innovation that then gains an understanding of how it works (knowledge), starting to form an opinion, regarding the decision to adopt or reject the given innovation” (Rogers et al., 2014). This narrative on the theory of the diffusion of innovation defined innovation as an idea, practice, or object, is not necessarily new, and is only perceived as new by an individual or other unit of adoption. In addition, Rogers considered that the adoption of an innovation depends significantly on the perception of individuals in relation to the potential attributes of it. Innovations or technology pro- ducts that offer better attributes over relative advantage, compatibility, and lower complexity predict a broader and faster acceptance rate compared to other innovations.

In another theoretical approach, (Davis et al., 1989). sought to estab- lish a theoretical basis to explain and predict how an individual’s per- ceptions of technological innovation affects their behaviour in relation to the adoption and effective use of technology. Davis, in his theoretical basis, included and tested two specific beliefs that he believed could predict the results of an effective use of technology: The first belief—perceived ease of use, was defined as the “degree to which a person believes that using a given system would be effort-free”. The second belief, perceived utility, was defined as “the degree to which a person believes that the use of a given system would improve their performance at work”.

Similarly, (Venkatesh et al., 2003). also proposed a theoretical basis, derived from an extensive review and synthesis of alternative

theoretical models, which has since been widely used by researchers to understand individual adoption and the use of technology. They theo- rized that the expectation of performance and the expectation of effort and social influence have significant relationships with the intention to use technologies. Recently, a series of studies have used these principles as a theoretical basis to explain the adoption of various technologies in different contexts including mobile payment (Al-Saedi et al., 2019), e- commerce (Haryanti and Subriadi, 2020), and mobile health (Dwivedi et al., 2020).

2.2. Factors affecting the adoption of technology

As mentioned earlier, the adoption of FOSS offers several benefits for the organizations and individuals who adopt it (AlMheiri et al., 2018; Syzdykova et al., 2017). More recently, a growing number of studies have shown that the low cost of acquisition and maintenance (total cost of ownership—TCO) has been one of the main motivators by which organizations adopt FOSS, and not only (Sanchez et al., 2020;

Dhir and Dhir, 2017). The reference (Ajila and Wu, 2007) noted that the cost of FOSS gives organizations an opportunity to experiment and fail quickly without significant financial risk. Thus, many researchers agree that the low cost of FOSS adoption facilitates innovation, and this benefit may be important, especially for developing countries (Patino- Toro et al., 2022). According to (Jokonya et al., 2013), developing countries that do not produce software end up paying for the import of software licenses or resorting to piracy. Therefore, the same study (Jokonya et al., 2013) noted that the low cost of adoption of FOSS is an important driver to avoid legal issues related to software piracy. An- other significant and deeply engaging study of the use of ICT in academic institutions conducted by both (Sooryanarayan et al., 2014) and (Pezer et al., 2017), observed that the use of FOSS in the academic context could offer quality and economic solutions which would help the local economy.

In addition to the low-cost factor, the quality of the system was also considered a relevant attribute for the adoption of information systems and, therefore, a widely researched theme (Aparicio and Costa, 2012;

Chen and Chengalur-Smith, 2015; Shukla et al., 2021). articulate that the quality of the system has a substantial impact on the assessment of ICT adoption issues. Similarly, (Alrawashdeh et al., 2019) found that the quality of the system influences the behavioural intent of users to adopt FOSS systems. The “system quality” refers to the desired quality of an information system (Delone and McLean, 2003), and can generally be driven largely by interoperability factors as well as the security and usability of the system (Alwadi et al., 2018).

Traditionally, interoperability is characterized as the ability of one system to process, integrate, and use information on behalf of (or to) another heterogeneous system in a meaningful, safe and effective manner, without any inconvenience to both parties (Jindal et al., 2022;

Neinstein et al., 2016). Reports from different industrial areas point to interoperability as a fundamental requirement to improve productivity, transparency and accelerate innovation (Jindal et al., 2022; Neinstein et al., 2016; Chalyvidis et al., 2016; da Rocha et al., 2020).

Interoperability is a fundamental need for flexible ICT and to improve business agility (Leal et al., 2019). For example, with regard to the cloud computing industry (Bouzerzour et al., 2020), the authors addressed the lack of interoperability in cloud environments and highlighted the importance of achieving interoperability to avoid supplier entrapment. Research by (Tshering and Anutariya, 2022) revealed the importance of interoperability in electronic governments. They suggest that an interoperable electronic government system can improve efficiency, accountability, transparency, and gain access to services at minimal cost and reduced capital risk caused by technology or supplier obsolescence.

Interoperability has also been discussed in the health industry.

Researchers such as (Adams et al., 2017) considered interoperability as fundamental to improving health care delivery. Results from other

(3)

studies indicate that interoperability can minimize delayed action in a patient, reduce repetitive examinations, simplify the process of searching for complex information, and reduce medical errors (Hidayat and Hermanto, 2020). The research (Sfakianakis et al., 2007) presented their insights into the impact of FOSS on the interoperability of health information systems. The same authors noted that FOSS is a great alternative to investing in interoperability between different technologies and applications. And for this reason, many companies adopt FOSS to face the challenges related to convergence blockades. FOSS contributes to interoperability, ensuring that data and systems can be interpreted independently of the tools that generated them (Lundell et al., 2021).

Safety is also a key predictor for assessing the quality level of a system and therefore its acceptance. The authors (Alharbi et al., 2017) investigated the impact of security on the adoption of electronic government. The findings also reveal that the safety perception of individuals has a significant impact. Furthermore, the authors (Umbas et al., 2022;

Amron et al., 2022) found that safety perception significantly influences the acceptability of ICT use. This is similar to the study (Tomić et al., 2022) that found that factors such as transaction security have a positive impact on the user’s intention to accept electronic payment systems.

According to the results of the studies, it is concluded that people prioritize the safest systems. People will not use unsafe systems.

Several studies also consider usability as an important system quality attribute that affects the adoption of information systems (Berendes et al., 2022; Llerena et al., 2019; Sagar and Saha, 2017).

According to (ISO, 2001), usability concerns the ability in which a software product can be understood, learned, used effectively and ef- ficiently in a specified usage context. Usability tests have led to some research sensitivity. In the research (Dawood et al., 2019), usability refers to software sustainability. The same authors found that the usability of the system influences the acceptance of users to adopt an information system, as well as its sustainability. The authors (Darmawan et al., 2021), in their study, found that the system usability was an important factor for the successful implementation of the Smart Regency Mobile-Apps application. Another interesting study on usability was conducted by (van der Nat et al., 2022). The authors tested patient-centred usability and found that the adoption of a personal health record (PHR) depends on its usability.

2.3. Research model proposal

Fig. 1describes the proposed conceptual model and combines several constructs to increase the explanatory power regarding the intention to use FOSS systems, namely: behavioural intentions (BI), performance expectation (PExp), effort expectation (EExp), and social influence (SI) which were adapted from (Venkatesh et al., 2003), while system quality (SysQual), security, interoperability (Interop) and cost related to work (Alrawashdeh et al., 2019) as well as usability (Usab) (Kamau and Sanders, 2013).Table 1lists the references and their de- finitions of the constructs.

According to the review of the literature, these factors are widely used in exploratory research on information system adoption, making it the appropriate, valid, recent, and reliable theoretical model proposed by accommodating a high percentage of variance (R²) in behavioural intention (Hoque and Sorwar, 2017; Kalavani et al., 2018; Wang et al., 2020). Studies by (Alrawashdeh et al., 2019; Dawood et al., 2019;

Safadi et al., 2015) on these features are critical to improving the perception and sustainability of FOSS.

Previous research has established a link between financial strain and propensity to use. The authors of (Aparicio and Costa, 2012; Kamau and Namuye, 2012) referred to FOSS as having great potential to reduce acquisition (CAPEX) and operation (OPEX) costs, as well as to bridge the innovation difficulty and digital divide at a relatively low cost that would otherwise be significantly difficult to achieve, particularly in developing countries.

For the author of (Smith, 2006), financial hardship is one of the major impediments in the pharmaceutical sector, whereas (Alrawashdeh et al., 2019) demonstrated the importance of financial burdens in FOSS adoption. According to this theoretical paradigm (Bhatiasevi and Krairit, 2013), FOSS is a good stimulus for innovation because its software maintenance costs are relatively low when compared to proprietary software.

According to the present research, if the benefits of adopting FOSS systems and technologies are judged to be larger than the financial cost associated with their adoption, potential decision makers will be more inclined to accept it.

The following hypotheses are proposed by the theoretical model:

Fig. 1. Research Model.

(4)

Hypothesis 1. (H1).Cost has a positive effect on behavioural intention to utilize FOSS.

In the framework of this study, it is assumed that the individual believes that using FOSS systems and technology will be simple, understandable, and effortless (Venkatesh et al., 2003). This construct is seen as critical in predicting behavioural intention, particularly in first contacts with a technology, because the perceived ease of use of that technology has a significant impact on the belief/behaviour of accepting to use the system/technology (Cimperman et al., 2016).

For users to agree to utilize a FOSS technology system, it must first be helpful and then simple to use. Potential users in the healthcare sector, for example, are likely to have little or no expertise with ICT. As a result, individuals may be frustrated early on due to a lack of official assistance and training (Bhatiasevi and Krairit, 2013). As a result, we can make the case that the technology that demands the least amount of effort will most likely be selected by users with the least amount of experience (Davis et al., 1989; Venkatesh et al., 2003).

Given the considerations presented above, the theoretical model proposes the following hypothesis:

Hypothesis 2. (H2). Expectation of effort influences behavioural intention to utilize FOSS.

The capacity of two or more software components to communicate despite variations in language, interface, and execution platform is referred to as interoperability (Wegner, 1996). In the context of the study, it refers to the degree to which an individual believes that by using FOSS, they will be able to interact with other external systems and store their documents in various formats, thereby promoting interoperability and establishing links between service providers and researchers (Muinga et al., 2018). When researching the function of FOSS in the interoperability of health information systems, (Sfakianakis et al., 2007), when studying the role of FOSS in the interoperability of health information systems (eHealth), took a different approach to this feature (eHealth). According to their findings, FOSS is the primary enabler of interoperability in eHealth.

Interoperability in the context of FOSS system acceptance has the potential to improve system quality and patient safety, safeguard against data loss, and improve care efficiency (Neinstein et al., 2016).

As a result, the theoretical model suggests the following hypothesis:

Hypothesis 3. (H3).Interoperability has a positive effect on system perceived quality.

Performance Expectancy is defined here as the degree to which an individual believes that utilizing FOSS would be beneficial in meeting operational goals and improving job performance. Previous research in various adoption situations (Ahmed et al., 2020; Zhou et al., 2019) revealed that performance expectancy has substantial predictive power about technology usage intention.

According to Davis (Davis et al., 1989), the degree to which an individual believes that using a given system will assist him in his work will contribute greatly to the system’s acceptance. Following this rea- soning, it is argued that the adoption of technologies seen as favourable (with high quality, security, stability, flexibility, low acquisition cost, and no vendor lock-in) as FOSS is considered, will necessarily lead to faster adoption (Gwebu and Wang, 2011).

In the context of our study, an example of this construct is when an individual realizes that using FOSS systems and technology, for example, would be useful to them, as well as more efficient in terms of productivity and speed in performing his tasks, and would allow them to enter the digital age much more easily with the desired outcome and services. As a result, the theoretical model suggests the following hypothesis:

Hypothesis 4. (H4).Performance expectancy has a positive effect on behavioural intention to use FOSS.

Social influence is assumed to be the degree to which the individual perceives that important people in his social/corporate environment, believe that it would be important for him/her to use the system in question to obtain better results. These people influence them toward the behavioural intention to use the technology.

According to preliminary UTAUT research (Venkatesh et al., 2003), social impact in the process of accepting new technologies is defined as a relevant predictor of the desire to use the system, especially in contexts where adoption is mandatory. In contrast, in voluntary use situations, social influence simply influences the perception of the system but has no substantial impact on its use. Several researchers on the usage of health-related technology have found that social influence has a considerable predictive value when it comes to using linked devices for health requirements (Wang et al., 2020). In this case, the individual’s use of technology is stimulated by the opinions of people close to or associated with reference groups (colleagues, managers, and other mechanisms), which increases the individual’s intention to use the technology, making social influence an equally important construct in our choice for determining the acceptance of new technologies.

Given the considerations presented above, the theoretical model proposes the following hypothesis:

Hypothesis 5. (H5). Social influence has a positive impact on behavioural intention to use FOSS.

The perceived security of technology refers to how much trust companies and consumers have in the software’s security. Unlike the traditional paradigm (referred to as “closed source”), FOSS is openly published and, through Freedom 1—free access to the source code, allows users to study and adapt to their operational needs (Marsan and Paré, 2013), as well as report vulnerabilities and propose complementary software improvements (Barcomb et al., 2019).

Table 1

Model Constructs definition.

Constructs Concepts Reference

Behavioural Intentions (BI) Degree to which an individual associates with the impact of using the system on his or her work. (Venkatesh et al., 2003) Performance Expectation (PExp) Degree to which an individual believes that using the system will help him/her make gains in his/

her job performance.

Effort Expectation (EExp) Degree of effort or ease associated with using a system.

Social Influence (SI) Degree to which an individual perceives those significant others believe that he or she should use the new system.

Cost Degree to which an individual perceives the benefits of using a system, even if it has a monetary cost

associated with it. (Alrawashdeh et al., 2019)

Interoperability (Intero) Degree to which a system reveals that it could openly exchange information with systems.

Security Degree to which individuals and organizations believe in software security.

System Quality (SysQual) Degree that a system discloses having desirable product characteristics, such as availability, reliability, performance, usability, and functionality.

Usability (Usab) Degree to which a product can be used by specified users to achieve specified goals with

effectiveness, efficiency, and satisfaction in a specified context of use. (Kamau and Sanders, 2013)

(5)

In their investigations, the authors of (Raymond, 1999; Hansen et al., 2002) concluded that FOSS has much greater levels of security than “proprietary and commercial software”, since the source is available and is evaluated by more individuals. Access to the source code enables FOSS researchers, developers, and advocates to enhance the degree of security due diligence far more quickly than proprietary systems, as argued in (Raymond, 1999).

According to (Witten et al., 2001), having source code available generally benefits system security. Despite FOSS’s technological su- premacy, several studies contend that this development paradigm (open standards) offers a number of security issues for organizations (Crain, 2017). Similarly, access to the FOSS source code gives hackers and other stakeholders greater analytical power to attack software vulnerabilities (Cowan, 2003; Schryen and Rich, 2010).

As a result, it is stated that the perception of safety has a vital impact and can potentially increase the system’s quality.

Hypothesis 6. (H6). Security has a positive impact on the system’s perceived quality.

The quality of the system is another essential criterion that stands out significantly in the adoption literature. System quality is defined by DeLone and McLean (Delone and McLean, 2003) as the total performance index of a system as evaluated by potential users. For this study, it is defined as a desirable feature of a FOSS system from the perspective of a potential user, often evaluated along the dimensions of system security, usability, and interoperability.

According to (Ajila and Wu, 2007; Wahyudin et al., 2008), certain FOSS projects have quality levels comparable to “proprietary and commercial software” development. Similarly, (Alrawashdeh et al., 2019) discovered a substantial association between system quality and FOSS component use. That is, the authors believe that the intention to utilize FOSS has explanatory value in this dimension.

According to (Chen and Chengalur-Smith, 2015; Costa et al., 2016), system quality substantially impacts evaluating ICT adoption issues.

This data suggests that system quality will influence an individual’s pleasure through the system performance expectation. As a result, we propose the following hypothesis:

Hypothesis 7. (H7). Quality of the system has a positive impact on performance expectation.

The authors of (Dawood et al., 2019) investigated the effect of usability on systems. The same authors concluded that usability is one of the most important factors influencing user acceptability and longevity of FOSS.

Usability is an important feature in software that has been characterized in terms of effectiveness, efficiency, and satisfaction in a specific context of usage (ISO, 2018). In technological acceptance models, e.g., (Venkatesh et al., 2003) and the information success model, usability is a desirable attribute (Delone and McLean, 2003).

Despite FOSS’s popularity, usability issues have received insufficient attention (Dawood et al., 2019).

Despite the popularity of FOSS, usability issues do not appear to have received adequate attention (Kamau and Sanders, 2013).

Keeping this in mind, we propose the following hypotheses:

Hypothesis 8. (H8). Usability has a positive impact on effort expectations.

Hypothesis 9. (H9).Usability has a positive impact on system quality.

3. Empirical study

3.1. Structure research approach

A preliminary exam was administered to two professors and an information system professional at the outset. Small changes to the level

of the questions were made. As seen inAppendix A, previously validated and tested scale items were used.

3.2. Participants

As a result, it became a suitable environment to bring together IT professionals and other entities involved in software acquisition pro- cesses within public organizations to find acceptance factors and then propose the FOSS acceptance survey model that can be used to develop possible recommendations.

3.3. Instrument

The questionnaire was created with tool Google Forms and was distributed electronically to each respondent’s WhatsApp and email addresses. The measurement instrument’s item development was ap- propriately adapted from trustworthy scales, validated from past empirical studies, and modified to fit the goal of the current study. The final measurement equipment used to test the structural model is listed inAppendix A.

This questionnaire was divided into four components. Our first section, as proposed by (Lo et al., 2015), helped give the target audi- ence a quick overview of FOSS systems. The purpose was to guarantee that participants understood the fundamental ideas of FOSS (definition and value proposition) and could answer the questionnaire clearly. The questionnaire’s second portion was developed to support demographic- type items such as gender, age, job category, years of experience, and academic level. Finally, the third and fourth sections of the questionnaire employed a five-point Likert scale with statements ranging from 1 (strongly disagree) to 5 (strongly agree) based on their FOSS experience and relevant qualities.

It is worthy of note that the questionnaire was only available for two months, from 27 November to the end of January. A total of 262 re- sponses were obtained for the sample. Because this is a convenience sample that is not representative of the population, the results are non- probabilistic and only pertain to the sample. As shown inTable 2, the sample comprised 76.7% males and 23.3% females, with most males aged 31.8 years and females aged 26.4 years. It is also possible to confirm that the professional profile distribution is nearly identical for responders with consulting duties (40.1%) and IT support (45%), with professional experience ranging from 2.7 years for females to 5.7 years for males.

3.4. Data analysis

Subsequently, SEM-PLS was used to test and validate the proposed structural model defined inFig. 1, because it allows for the inclusion of Table 2

Sample characteristics.

Population Male 201 76.7%

Female 61 23.3%

Age Male 31.8

Female 26.4

Professional Profile CIO 2 0.8%

CEO 5 1.9%

CTO 1 0.4%

IT Support 118 45%

Consultant 105 40.1%

Software Acquisition 17 6.5%

Manager 14 5.3%

Years of Experience Male 5.7

Female 2.7

Instructional level Technical background 126 48.1%

Bachelor’s degree 98 37.4%

Post Graduate studies 15 5.7%

Master's 21 8%

PhD 2 0.8%

(6)

reflective and formative measurement models and is recommended in the literature as an appropriate method in the early stages of theory revelation to test and validate exploratory models for theory building purposes (Hair et al., 2011; Henseler et al., 2009).

4. Results and discussion 4.1. Measurement model assessment

In order to ensure the internal consistency of the dimensions, that is, to validate the reliability of the measurement instrument, which in this study is represented by the research questionnaire, the indicator Cronbach’s alpha for internal consistency and Composite Reliability (CR) for indicator correlation, should be more than 0.6 and 0.7, respectively. As indicated inTable 3, all indicators have values that are equal to or greater than 0.91, meaning that all items are similarly de- pendable.Table 3also reports convergent validity using Average Var- iance Extracted (AVE), with 0.5 being the minimum value as corrected by (Hair et al., 2011). The results, however, show that all items con- verge and share a high amount of variance, implying that the dimensions explain more than half of the variance of their indicators on average. This consistency demonstrates that the outer loadings of the dimensions have a lot in common when it comes to measuring each of the latent variables (LV). The discriminant validity test findings de- monstrate how one dimension is sufficiently distinct from another. As a result, the cross-loadings table (see Appendix B) indicates that the outer loadings of the indicators are greater than all other dimensions’ loadings (Gefen and Straub, 2005).

The Fornell-Larcker criterion was also examined. This criterion is typically based on the assumption that a dimension shares more var- iation with its associated indicators than any other dimension. The criterion compares the AVE’s square root with the LV’s correlations to accomplish this (Hair et al., 2011; Fornell and Larcker, 1981). This condition is met, as shown inTable 4, with all model dimensions ver- ified and the different construct measures being distinct from one another.

In addition, the Heterotrait-Monotrait (HTMT) criterion was applied to evaluate discriminant validity. According to (Henseler et al., 2015), the HTMT criterion can assume values of 0.8 and 0.9. Based on our results, using thresholds suggested by (Henseler et al., 2015), the HTMT ratio value for all constructs is less than 0.9 > 1, indicating the es- tablishment of discriminant validity of the research model (see Table 5).

4.2. Assessment of the structural model

Once all dimensions were tested for multicollinearity, we proceeded to study the results of the structural model by calculating the Variance Inflation Factor (VIF). The results were more noticeable and there were no multicollinearity problems since all the VIF obtained were less than 4.266, as shown inTable 6, which means that it was well below the threshold of 10 (Diamantopoulos and Siguaw, 2006; Zainodin and Yap, 2013).

The structural model’s quality was evaluated using bootstrapping, a resampling process that uses multiple samples derived from the original dataset. In this scenario, 5000 samples were used to estimate the significance of the correlations between dimensions (path significance) inside the structural mode (Henseler et al., 2009).Fig. 2depicts the outcome of the bootstrapping technique. Following the validation of the structural model, the structural routes were examined to validate the study hypotheses. Cost (β̂ = 0.377,p< 0.001), Effort Expectation (β̂

= 0.077,p< 0.10), Performance Expectation (β̂ = 0.345,p< 0.05), and Social Influence (β 0.176,p< 0.05) explain 75.5% of the variance in Behavioural Intention. Effort Expectancy is explained (22.3%) by Usability (β̂ = 0.473,p< 0.001) and Performance Expectancy is explained (39.0%) by System Quality (β̂ = 0.624,p< 0.001). System Quality, on the other hand, is explained (80.4%) by Security (β̂ = 0.265,p< 0.001), Interoperability (β̂ = 0.322,p< 0.05) and Us- ability (β̂ = 0.376, p< 0.001). As seen in Path: * significant at p< 0.10; * * significant atp< 0.05; * ** significant atp< 0.001, the model reveals that all associations have at least a slight predictive influence. Except for EExp, the four dependent latent variables explain Table 3

Construct reliability and validity.

Constructs Cronbach’s Alpha Composite Reliability (rho_a) Composite Reliability (rho_a) Average Variance Extracted (AVE)

BI 0.897 0.897 0.936 0.83

Cost 0.932 0.932 0.957 0.88

EExp 0.943 0.96 0.963 0.898

Interop 0.953 0.953 0.977 0.955

PExp 0.933 0.935 0.957 0.882

SI 0.91 0.916 0.943 0.847

Security 0.933 0.933 0.968 0.937

SysQual 0.942 0.942 0.972 0.945

Table 4

Fornell-Larcker criterion.

BI Cost EExp Interop PExp SI Security SysQual Usab

BI 0.911

Cost 0.786 0.938

EExp 0.667 0.579 0.947

Interop 0.776 0.753 0.544 0.977

PExp 0.817 0.744 0.757 0.632 0.939

SI 0.714 0.609 0.627 0.5 0.751 0.92

Security 0.679 0.611 0.464 0.781 0.586 0.462 0.968

SysQual 0.771 0.714 0.497 0.832 0.624 0.577 0.819 0.972

Usab 0.665 0.582 0.473 0.806 0.57 0.492 0.804 0.849 0.925

(7)

more than half of the variations. Behavioural Intention (BI) with R²

= 0.756, Performance Expectancy (PExp) with R²= 0.390 and System Quality (SysQual) with R²= 0.804 have significant values. In addition, the indicator Q2 referring to the predictive ability of the model was measured (Stone, 1976; Geisser and Eddy, 1979). As a result, BI (Q²= 0.708), EExp (Q²= 0.211), PExp (Q²= 0.393) and SysQual (Q²= 0.789), so that the model satisfied the desired predictive relevance. The Goodness-of-Fit (GoF) index was also calculated. Specifically, the GoF can be understood as a means of validating a PLS trajectory model at the global level (Henseler and Sarstedt, 2013). This is obtained based on

the mean correlation of the square root of the AVE and the R². Our results show a GoF > 0.3 satisfies model validation.Table 7.

Finally, we calculated the size of the f²effect in relation to the variable exogenous to endogenous ratios, which are frequently supported by the values 0.150 and 0.350, respectively (values > 0.350 represent large effects, values between 0.150 and 0.350 represent medium effects, and values between 0.150 and 0.350 represent small effects).

According toTable 8, this measurement had a significant and favourable influence on all hypotheses.

Table 5

Heterotrait-monotrait results.

BICost 0.859

EExp 0.716 0.611

Interop 0.838 0.799 0.57

PExp 0.893 0.799 0.802 0.672

SI 0.784 0.658 0.669 0.533 0.808

Security 0.741 0.655 0.491 0.828 0.629 0.5

SysQual 0.838 0.762 0.52 0.878 0.665 0.622 0.874

Usab 0.711 0.613 0.487 0.841 0.599 0.524 0.847 0.89

Table 6 Inner VIF.

BICost 2.271

EExp 2.389

Interop 3.326

PExp 4.266

SI 2.367

Security 3.301

SysQual 1

Usab 1 3.68

Fig. 2. Structural model. Path: * significant atp< 0.10; * * significant atp< 0.05; * ** significant atp< 0.001.

(8)

4.3. Discussion

All of the hypotheses offered for the adoption of FOSS systems were empirically supported. For example, Cost (H1), Performance Expectation (H4), and Social Influence (H5) all significantly influenced Behavioural Intent (BI) (p0.001), but Cost (H1) had a medium effect and Performance Expectation (H4) and Social Influence (H5) both had a smaller effect. For the same relationship (BI), however, (H2) (Effort Expectations) is not statistically significant (p0.10), and the impact is too small to be adequately explained. On the other hand, previous work has shown that the expectation of effort performance had a significant positive effect on the acceptance of FOSS (Ajila and Wu, 2007;

Alrawashdeh et al., 2019).

In terms of the hypotheses that influence System Quality (H3, H6, and H9), it can be established that all have a highly significant influence (p0.001), H3 has a strong explanatory effect (f²> 0.350), and H6 and H9 have a small effect (0.150 > f²> 0.020). H7 illustrates the effect of System Quality on Performance Expectations. This ratio is positive, highly significant (p0.001), and has a large effect (f²> 0.350) on performance expectations (Alrawashdeh et al., 2019). It has also been stated that the quality of the system has a substantial impact on

FOSS acceptability. The other line of research (Ajila and Wu, 2007) also showed strong correlations between FOSS adoption and product quality gains.

H8 has an average bond (0.350 > f²> 0.150), an expectation of effort, and a strong statistical significance (p 0.001). Although it is likewise favourably and strongly associated with the model, H8 (Usability) is the least rewash-independent latent variable for FOSS adoption. In fact, the outcome is comprised of the findings of (Raza et al., 2012) in their study on the usability of FOSS.

5. Conclusion

As the literature review notes, the adoption of FOSS offers several benefits for organizations that adopt it. However, few studies are conducted in developing countries, so this study aims to fill this gap by proposing a new theoretical model to assess which factors influence the intention to use FOSS. The research was based on data from questionnaires applied in Angola as an example of a developing country.

Our results validate the assumptions that the specified dimensions of our study model (system quality, security, interoperability, usability, costs, effort expectancy, performance expectancy, and social influence) Table 7

Hypothesis Testing Results.

Ind. Variable Dep. Variable Findings Conclusion

H1 Cost → BI Positive relationship * **

(β̂ = 0.377,p< 0.001) Supported with large effect

H2 EExp → BI Positive relationship *

(β̂ = 0.077,p< 0.10) Supported with small effect

H3 Interop → SysQual Positive relationship * *

(β̂ = 0.322,p< 0.05) Supported with medium effect

H4 PExp → BI Positive relationship * *

H5 SI → BI Positive relationship * *

H6 Security → SysQual Positive relationship * **

H7 SysQual → PExp Positive relationship * **

H8 Usab → EExp Positive relationship * **

H9 Usab → SysQual Positive relationship * **

(β̂ = 0.376,p< 0.001) Supported with large effect Notes: Path Coefficient β̂: NS = not significant; * significant atp< 0.10; * * significant atp< 0.05; * ** significant atp< 0.001. Effect size: > 0.350 large; >

0.150 and ≤ 0.350 medium; > 0.150 and ≤ 0.350 small (Chin, 2022; Cohen, 2013).

Table 8

Summary of the Hypothesis Evaluation.

Hypothesis f² Effect Size β^ pValues

H1 Cost - > BI 0.257 Medium 0.377 0

H2 EExp - > BI 0.01 Small 0.077 0.365

H3 Interop - > SysQual 0.159 Medium 0.322 0.003

H4 PExp - > BI 0.115 Small 0.345 0.001

H5 SI - > BI 0.054 Small 0.176 0.012

H6 Security - > SysQual 0.109 Small 0.265 0

H7 SysQual - > PExp 0.639 Large 0.624 0

H8 Usab - > EExp 0.288 Medium 0.473 0

H9 Usab - > SysQual 0.196 Medium 0.376 0

(9)

have a strong influence on the intention to use FOSS. System quality is arguably the most significant latent variable in the model, being largely explained by the factors of security, interoperability, and usability. In addition, the low cost of FOSS, performance expectation, and social influence also proved to be important factors in influencing the acceptability of FOSS. On the other hand, respondents did not consider effort expectation as a determining factor for FOSS acceptability.

This is the first study on FOSS conducted in Angola. Theoretically, this research contributes to the current literature on FOSS by proposing a new theoretical model of adoption that includes interoperability in system adoption. Furthermore, the model identifies that cost and social influence have a significant weight, specifically in the adoption of this type of system. And this, in practice, may enable decision makers to promote FOSS in developing countries.

This study, like all empirical research, has limitations. Initially, all analyses are based on survey data. In addition, our questionnaire was applied only in large cities. Although the results are statistically significant, additional studies with a broader geographic scope would improve the explanatory ability of the model. The proposed model suggests further investigation of the strength of the system’s influence on other regions. In summary, there could be more studies on this type of subject.

CRediT authorship contribution statement

Conceptualization, D.G.S., C.C. and C.J.C.; methodology, D.G.S., C.C. and C.J.C.; validation, D.G.S., C.C. and C.J.C.; formal analysis, D.G.S., C.C. and C.J.C.; investigation, D.G.S., C.C. and C.J.C.; resources, D.G.S., C.C. and C.J.C.; data curation, D.G.S., C.C. and C.J.C.; writing—original draft preparation, D.G.S., C.C. and C.J.C.;

writing—review and editing, D.G.S., C.C. and C.J.C.; visualization, D.G.S., C.C. and C.J.C.; supervision, D.G.S., C.C. and C.J.C.; project administration, D.G.S., C.C. and C.J.C.; funding acquisition, C.C.and C.J.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Fundação para a Ciência e a Tecnologia, I.P. (FCT) [ISTAR Projects: UIDB/04466/2020 and UIDP/

04466/2020]. Likewise, we gratefully acknowledge the financial support via ADVANCE-CSG from the Fundação para a Ciência and Tecnologia (FCT Portugal) through research grant number UIDB/

04521/2020.

Institutional review board statement Not applicable.

Informed consent statement Not applicable.

Data availability statement

Not Applicable, the study does not report any data.

Conflict of interest

The authors declare that they have no conflicts of interest.

(10)

Appendix A SeeTable 9.

Table 9 Questionnaire.

Constructs Concepts References

Behavioural Intentions (BI) BI1: I intend to adopt Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR in the future.

BI2: I will try to adopt Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR in the coming months.

BI3: I am willing to use Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR frequently

(Venkatesh et al., 2003)

Performance Expectation (PExp) PE1: I would find Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR, useful in my work.

PE2: Adopting Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR, will allow me to accomplish my tasks faster.

PE3: Adopting Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR, would increase my productivity.

Effort Expectation (EExp) EE1: My interaction with Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR, would be clear and understandable.

EE2: It would be easy for me to become skilled in using Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR.

EE3: I would find Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR, easy to use.

Social Influence (SI) SI1: I will adopt Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR if the people who influence my behavior think I should use it.

SI2: I will adopt Free and Open-Source Software like Linux Ubuntu, Libre Office, and OpenEMR, if the people who are important to me think I should use it.

SI3: I will adopt Free and Open-Source Software like Linux Ubuntu, Libre Office, and OpenEMR, if the instructors of my apprenticeship are helpful in using such a system.

Cost Cost1: The total cost of Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR is reasonably low.

Cost2: The total cost Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR is cost-effective.

Cost3: In general, Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR is a good alternative to reduce financial burden.

(Alrawashdeh et al., 2019)

Interoperability (PI) Int1: Free and Open-Source software such as Linux Ubuntu, Libre Office and OpenEMR may provide services and accept services from other systems.

Int5: In general, Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR can interact with other systems and exchange data with them

Security (Sec) Sec3: I believe that no one can access my private data stored on Free and Open-Source Software systems like Linux Ubuntu, Libre Office and OpenEMR without my permission.

Sec4: Free and Open-Source Software such as Linux Ubuntu, Libre Office and OpenEMR do not share my personal information with others

System Quality (SysQ) SysQ3: Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR decreases the number of errors when using the computer.

SysQual4: Free and Open-Source Software like Linux Ubuntu, Libre Office and OpenEMR require less maintenance

Usability Usab1: Free and Open-Source Software systems like Linux Ubuntu and Libre Office are easier to use than proprietary software like Microsoft Office and Windows

Usab2: Free and Open-Source Software systems like Linux Ubuntu and Libre Office have familiar icons that are easily recognizable than proprietary software like Microsoft Office and Windows Usa3: Free and Open-Source Software systems like Linux Ubuntu and Libre Office have better help facilities, tutorials and support than proprietary software like Microsoft Office and Windows Usab4: I generally like the user interface of Free and Open-Source Software like Linux Ubuntu and Libre Office than proprietary software like Microsoft Office and Windows

Usab5: Navigating while doing other tasks on Free and Open-Source Software systems like Linux Ubuntu and Libre Office is easier than proprietary software like Microsoft Office and Windows

(Kamau and Sanders, 2013)

(11)

Appendix B SeeTable 10.

References

Adams, K.T., Howe, J.L., Fong, A., Puthumana, J.S., Kellogg, K.M., Gaunt, M., Ratwani, R.M., 2017. An analysis of patient safety incident reports associated with electronic health record interoperability. Appl. Clin. Inform. 08, 593–602.https://doi.org/10.

4338/aci-2017-01-ra-0014

Ahmed, M.H., Bogale, A.D., Tilahun, B., Kalayou, M.H., Klein, J., Mengiste, S.A., Endehabtu, B.F., 2020. Intention to use electronic medical record and its predictors among health care providers at referral hospitals, north-West Ethiopia, 2019: Using unified theory of acceptance and use technology 2(UTAUT2) model. BMC Med.

Inform. Decis. Mak. 20, 207.https://doi.org/10.1186/s12911-020-01222-x Ajila, S.A., Wu, D., 2007. Empirical study of the effects of open source adoption on

software development economics. J. Syst. Softw 80, 1517–1529.https://doi.org/10.

1016/j.jss.2007.01.011

Al-Saedi, K.; Al-Emran, M.; Abusham, E.; El Rahman, S.A. Mobile Payment Adoption: A Systematic Review of the UTAUT Model. In Proceedings of the 2019 International Conference on Fourth Industrial Revolution (ICFIR), Manama, Bahrain, 19–21 February 2019; pp. 1–5.https://doi.org/10.1109/icfir.2019.8894794.

Alharbi, N., Papadaki, M., Dowland, P., 2017. The impact of security and its antecedents in behaviour intention of using e-government services. Behav. Inf. Technol. 36, 620–636.https://doi.org/10.1080/0144929x.2016.1269198

AlMheiri, N.H.; Rajan, A.; Akre, V. Framework for Open Source Software implementation in the Government Sector of Dubai. In Proceedings of the 2018 Fifth HCT Information Technology Trends (ITT), Dubai, United Arab Emirates, 28–29 November 2018; pp.

71–76.https://doi.org/10.1109/ctit.2018.8649506.

Alrawashdeh, T.A., Elbes, M.W., Almomani, A., ElQirem, F., Tamimi, A., 2019. User acceptance model of open source software: an integrated model of OSS characteristics and UTAUT. J. Ambient Intell. Humaniz. Comput 11, 3315–3327.https://doi.org/10.

1007/s12652-019-01524-7

Alwadi, A.; Nahhas, A.; Bosse, S.; Jamous, N.; Turowski, K. Toward a Performance Requirements Model for the Early Design Phase of IT Systems. In Proceedings of the 2018 Sixth International Conference on Enterprise Systems (ES), Limassol, Cyprus, 1–2 October 2018, pp. 9–16. https://doi.org/10.1109/es.2018.00009.

Amron, M.T.; Ibrahim, R.; Abu Bakar, N.A. The Impact of Technology Readiness on User Acceptance of Government Cloud Computing Services. In Proceedings of the 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), Ankara, Turkey, 9–11 June 2022; pp. 1–7.https://doi.org/10.

1109/hora55278.2022.9799952.

Aparicio, M.; Costa, C.J. Macroeconomics leverage trough open source. In Proceedings of the Workshop on Open Source and Design of Communication, Lisboa, Portugal, 11 June 2012; pp. 19–24.https://doi.org/10.1145/2316936.2316941.

Barcomb, A.; Stol, K.-J.; Riehle, D.; Fitzgerald, B. Why Do Episodic Volunteers Stay in FLOSS Communities? In Proceedings of the 41st International Conference on Software Engineering, Montreal, Quebec, Canada, 25–31 May 2019; pp. 948–959.

https://doi.org/10.1109/icse.2019.00100.

Berendes, S., Hilpert, S., Günther, S., Muschner, C., Candas, S., Hainsch, K., van Ouwerkerk, J., Buchholz, S., Söthe, M., 2022. Evaluating the usability of open source frameworks in energy system modelling. Renew. Sustain. Energy Rev. 159, 112174.

https://doi.org/10.1016/j.rser.2022.112174

Bhatiasevi, V., Krairit, D., 2013. Acceptance of open source software amongst Thai users:

an integrated model approach. Inf. Dev 29, 349–366.https://doi.org/10.1177/

0266666912465880

Bouzerzour, N.E.H., Ghazouani, S., Slimani, Y., 2020. A survey on the service interoperability in cloud computing: Client‐centric and provider‐centric perspectives.

Softw. Pr. Exp 50, 1025–1060.https://doi.org/10.1002/spe.2794

Chalyvidis, C.E., Ogden, J.A., Johnson, A.W., Colombi, J.M., Ford, T.C., 2016. A method for measuring supply chain interoperability. Supply Chain Forum Int. J 17, 246–258.

https://doi.org/10.1080/16258312.2016.1247655

Chen, Y.-H., Chengalur-Smith, I., 2015. Factors influencing students’ use of a library Web portal: applying course-integrated information literacy instruction as an intervention.

Internet High. Educ 26, 42–55.https://doi.org/10.1016/j.iheduc.2015.04.005 Chin, W.W. Commentary: Issues and Opinion on Structural Equation Modeling. MIS Q.

1988, 22, vii–xvi. Available online:〈http://www.jstor.org/stable/249674〉(accessed on 30 January 2022).

Cimperman, M., Brenčič, M.M., Trkman, P., 2016. Analyzing older users’ home telehealth services acceptance behavior—Applying an Extended UTAUT model. Int. J. Med Inform. 90, 22–31.https://doi.org/10.1016/j.ijmedinf.2016.03.002

Cohen, J., 2013. Statistical Power Analysis for The Behavioral Sciences. Routledge, Oxfordshire, UK.

Costa, C.J., Ferreira, E., Bento, F., Aparicio, M., 2016. Enterprise resource planning adoption and satisfaction determinants. Comput. Hum. Behav. 63, 659–671.https://

doi.org/10.1016/j.chb.2016.05.090

Cowan, C., 2003. Software security for open-source systems. IEEE Secur. Priv. 1, 38–45.

https://doi.org/10.1109/msecp.2003.1176994

Crain, S.P., 2017. Open source security assessment as a class project. J. Comput. Sci. Coll 32, 41–53.

Darmawan, A.K.; Hamzah, M.A.; Bakir, B.; Walid, M.; Anwari, A.; Santosa, I. Exploring Usability Dimension of Smart Regency Service with Indonesian Adaptation of The System Usability Scale (SUS) and User Experience Questionnaire (UEQ). In Proceedings of the 2021 International Conference on Computer Science, Information Technology, and Electrical Engineering (ICOMITEE), Banyuwangi, Indonesia, 27–28 October 2021; pp. 74–79.https://doi.org/10.1109/icomitee53461.2021.9650086.

Table 10 Cross-Loadings.

BI1 0.866 0.791 0.591 0.896 0.712 0.575 0.8 0.875 0.821

BI2 0.931 0.668 0.599 0.603 0.745 0.674 0.512 0.615 0.503

BI3 0.936 0.688 0.633 0.619 0.775 0.701 0.541 0.615 0.491

Cost1 0.737 0.915 0.566 0.657 0.718 0.613 0.489 0.61 0.44

Cost2 0.73 0.96 0.542 0.746 0.661 0.535 0.63 0.704 0.611

Cost3 0.745 0.939 0.52 0.715 0.714 0.565 0.6 0.696 0.586

EExp1 0.705 0.613 0.955 0.587 0.765 0.631 0.478 0.529 0.477

EExp2 0.64 0.543 0.956 0.475 0.736 0.611 0.427 0.486 0.482

EExp3 0.529 0.472 0.931 0.475 0.636 0.528 0.406 0.379 0.369

Interop1 0.757 0.714 0.533 0.977 0.588 0.468 0.773 0.814 0.796

Interop5 0.759 0.757 0.531 0.977 0.646 0.51 0.752 0.812 0.779

PExp1 0.76 0.73 0.712 0.642 0.902 0.647 0.538 0.525 0.495

PExp2 0.771 0.671 0.73 0.582 0.968 0.703 0.562 0.595 0.558

PExp3 0.769 0.698 0.692 0.56 0.946 0.762 0.552 0.635 0.55

SI1 0.613 0.562 0.579 0.436 0.654 0.914 0.434 0.544 0.474

SI2 0.621 0.515 0.542 0.421 0.637 0.947 0.388 0.509 0.457

SI3 0.722 0.597 0.604 0.512 0.766 0.899 0.449 0.537 0.431

Sec3 0.669 0.594 0.435 0.739 0.586 0.463 0.967 0.784 0.774

Sec4 0.646 0.589 0.463 0.773 0.55 0.433 0.969 0.802 0.783

SysQ3 0.757 0.702 0.483 0.808 0.618 0.573 0.788 0.972 0.828

SysQ4 0.741 0.687 0.484 0.81 0.596 0.549 0.805 0.972 0.823

Usab1 0.548 0.477 0.398 0.698 0.494 0.432 0.711 0.754 0.947

Usab2 0.612 0.53 0.44 0.733 0.493 0.462 0.701 0.786 0.915

Usab3 0.526 0.504 0.358 0.717 0.461 0.346 0.69 0.724 0.92

Usab4 0.668 0.557 0.504 0.759 0.566 0.497 0.806 0.815 0.928

Usab5 0.699 0.61 0.469 0.81 0.606 0.522 0.797 0.834 0.916