Formative indexes

An index is a standard tool in questionnaire research. It helps researchers investigate complex constructs, and at the same time it reduces some of the problems of validity and reliability that can affect quantitative studies. The general idea behind this tool is to take a complex construct that one is interested in, say, whether a person has a healthy lifestyle, and break that down into several parts, e.g., whether or not they get enough sleep, whether or not they exercise etc. Now, instead of asking just one question (“Do you have a healthy lifestyle?”), you ask a question for each of the aspects or variables that you have broken the complex construct into. From the answers to each of the questions, one then computes the answer to the overall question one wants answered (more details on that below).

In this study, the views on etiology described in chapter 2 and their compo- nent parts form the basis of three indexes. Instead of asking participants one question, such as “Do you subscribe to the Voice of Competence view regarding the evidential use of intuitive judgements in linguistics?”, they are asked several questions, each of which covers just one aspect of the three views that are under investigation.

The individual questions in the questionnaire are called the items of the index. The indexes in this study are built with so called formative items (Dia- mantopoulos and Winklhofer, 2001). This means that the components that go

into the index are seen as individual aspects of the construct in question, and they are seen as collectively constituting that construct (e.g., the concept of a healthy lifestyle, or one of the three views on the etiology question investigated here).² When one collects data with a formative index, the results for each item are not necessarily expected to intercorrelate as some criteria for the construct may be fulfilled while others are not (this is in contrast to reflexive scales).

According to Petersen (2010), the kind of indexes described here are used in survey studies as they often give more valid measurements than using single questions, i.e., they may make it more likely that you are measuring the thing you actually want to measure (if well-constructed). The idea is that if par- ticipants are given a number of questions about specific aspects of a complex construct instead of one question about the complex construct in general, there is a greater chance that they will actually understand and answer the questions in the way they were intended to be understood.

Compared to asking just one question about a complex construct, using in- dexes furthermore potentially provides benefits for the reliability of a study.

This is because asking a number of questions instead of only one means that the random variation in participants’ answers is distributed over the multiple questions. If the variation is really random, and not systematic due to some un- considered variable, this means that the overall negative effects are diminished, and the result is more reliable.

When creating a formative index, all aspects of the modelled concept need to be covered, and each aspect should be included only once. The selection of aspects to include as items relies primarily on the theory/literature, and one should be very careful with the selection process as it cannot be tested statistically whether an aspect/item belongs to the concept or not (this can be done for reflexive scales, on the other hand).

The indexes of this study are summative, which means that for each item in the questionnaire, the scores are added up to form the total score. In the example used above, this would mean that instead of answering the question

“Do you have a healthy lifestyle?”, participants would indicate their position on several items that together form the complete construct, and those answers would then be summed to form the total score for the construct of “having a healthy lifestyle”. There are, however, other ways to calculate the scores for indexes as well, see Hellevik (2002). The total score of an index can primarily be interpreted in relation to something else, e.g., the score of one person as related to the score of another person or as related to the total possible score.

2Another type of index can be made with so called reflexive items, and those “indexes” are usually called scales (see Spector 1992). For reflexive scales, the items are seen as tracking something that is the result of the complex construct, not tracking constituent parts of the construct. For instance, being tired often could be seen as the result of an unhealthy lifestyle.

The three indexes of this study were constructed to map the three views mentioned in chapter 2. That is, each view is turned into a separate index.

Participants will thus get a score on each of the indexes, and the one they score the highest on will be interpreted as the view they agree with the most.

The questionnaire contains six items which are all part of the three indexes.

A number of additional items are included to cast light on the Experimental Syntax debate, and there are questions pertaining to participants’ backgrounds as well (see the full questionnaire in appendix A). See an overview of how the items build into the indexes in table 5.1 above.

5.3.2.1 Overlapping indexes

The three indexes used in this study have a certain degree of overlap between them (see table 5.1 above). By this, I mean that for some (most, in fact) of the questions in the questionnaire, choosing one particular response will result in getting a point on more than one of the indexes. The literature that specifically deals with overlapping formative indexes is sparse. Based on the literature on overlapping reflexive scales, I have identified some potential problems. Here, I will only mention the two that seem the most problematic (see Spector 1992 and Nicholls et al. 1982 for more discussion of overlapping reflexive scales).

The first problem that I want to mention is that when different sub-scales correlate with another criterion variable, it will not be possible to determine whether the similar relation between the two scales and the variable are because both scales are in fact related to the third variable or whether it is because the scales overlap. As an example, say that two of the views look like they are related positively to, for instance, the participants’ geographical location. In that case, we will not know whether that is because they are each actually related to location or because the two indexes share some items. However, the constructs in this case, the three views on the etiology question, overlap themselves. In chapter 2 we saw that on several of the areas of disagreement (variables), two of the views would have the same component view. That is, the three overall views are not different becauseall their component parts are different, instead they are different because they each have different configurations of component views that are shared with the other views. I think the indexes need to reflect this fact although it makes the interpretation of the results less straightforward.

The second problem is that with overlapping scales, the score for one con- struct may be inflated because of answers resulting from other constructs. Using an example from Spector (1992, 40), I want to argue that this is mainly a prob- lem for reflexive scales, not necessarily for formative indexes. Suppose we have two constructs, anger and anxiety, and that we have built a reflexive scale for

each of the constructs with a number of items on each, corresponding to the aspects of the construct that we want to investigate. “I feel tense” is a compo- nent of both scales. Assume that the participant who fills out the questionnaire is angry, but not anxious. Their positive response to “I feel tense” will falsely inflate their score on the anxiety scale.

This would be problematic for a reflexive scale, but for a formative index the components are independent of each other, so the fact that the participant responds positively to “I feel tense” should add to the scores for both constructs.

In conclusion, I will proceed with the indexes presented above even though there are overlaps between them.