The Achievement Emotions Questionnaire: Validation for Pre-adolescent Students
Francisco Peixoto, Lourdes Mata, Vera Monteiro and Cristina Sanches ISPA - Instituto Universitário
Reinhard Pekrun University of Munich
The published manuscript is available at
http://www.tandfonline.com/doi/abs/10.1080/17405629.2015.1040757?journalCode=pedp20 doi: 10.1080/17405629.2015.1040757
Correspondence concerning this article should be addressed to Francisco Peixoto, Department of Educational Psychology, ISPA - Instituto Universitário, Rua Jardim do Tabaco, nº34, 1149-041 Lisbon, Portugal.
E-mail: fpeixoto@ispa.pt
Phone: +351 218 811 700 Fax: +351 218 860 954
Abstract
The Achievement Emotions Questionnaire is a self-report instrument developed to measure the emotions of students in academic situations. The main purpose of this research was to adapt and validate this questionnaire to assess pre-adolescent class-related and test-related emotions towards mathematics. The participants were 1515 Portuguese students from grades 5 and 7 (age range 10-13 years). Confirmatory factor analyses and descriptive statistics confirm the reliability and internal validity the Achievement Emotions Questionnaire for Pre-Adolescents (AEQ-PA), providing evidence that the AEQ-PA is an effective instrument to assess pre-adolescent achievement emotions towards mathematics classes and tests.
Emotions are central to the understanding of student learning and achievement (e.g. Govaerts & Grégoire, 2008; Pekrun, 2006; Weiner, 2010). Despite the importance of emotions in academic settings, research on this topic is relatively recent, with some
exceptions including research on test anxiety (Zeidner, 1998) and Weiner’s attribution theory (Weiner, 2010). Results of several qualitative studies (e.g. see Pekrun, Goetz, Titz & Perry, 2002) highlight that, in their academic activities, students experience a wide range of
emotions, which can appear periodically at different frequencies depending on the academic situations. This emotional diversity must be considered in order to deeply understand
student’s affective life.
Pekrun's (2006) control-value theory provides a social-cognitive perspective on achievement emotions providing a theoretical taxonomy of the internal structure of academic emotions. It maintains that control appraisals (competence beliefs, expectancies, attributions) and value appraisals (perceived value of activities or outcomes) are critical antecedents of achievement emotions. This theory organizes emotions according to three dimensions: valence (positive vs. negative), activation level (activating vs. deactivating), and object focus (activity vs. outcomes). It has been the theoretical framework for the development of
instruments adapted to different grade levels which allows us to understand emotions and emotional change in different school situations.
Pekrun, Goetz and Perry (2005) conceived the Achievement Emotions Questionnaire (AEQ) based on exploratory studies identifying the emotions reported by students in
academic situations (see Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011). The AEQ includes nine different emotions (enjoyment, hope, pride, relief, anger, anxiety, hopelessness, shame, and boredom), classified according to the three dimensions above. Items focus on emotions experienced in three contexts: class attendance, learning, and doing tests (Pekrun et al, 2005).
Research with the AEQ supports the idea that achievement emotions are better understood when taking into consideration “both the differences between discrete emotions and the differences between emotions that occur in different achievement settings” (Pekrun et al. 2011, p. 44).
Given the lack of empirical data among elementary school students, Lichtenfeld, Pekrun, Stupnisky, Reiss, and Murayama, (2012) constructed the AEQ – Elementary School (AEQ-ES) which assesses three emotions (enjoyment, anxiety and boredom) in three
educational contexts (class, learning, test). The results corroborated the hierarchical structure of AEQ showing that elementary students distinguish between both different emotions and emotions in different achievement contexts.
In addition, Pekrun, Goetz, and Frenzel (2005) developed the AEQ-Mathematics (AEQ-M) which measures seven achievement emotions (enjoyment, pride, anger, anxiety, shame, hopelessness, boredom) in upper elementary, middle and high school students. This instrument has been used in longitudinal and cross-cultural studies (e.g., Frenzel, Goetz, Lüdtke, Pekrun, & Sutton, 2009; Frenzel, Pekrun, & Goetz, 2007a, 2007b; Frenzel, Thrash, Pekrun, & Goetz, 2007), but so far there has been no study specifically conducted to validate the AEQ-M and examine its structure and psychometric properties with pre-adolescents.
Current Research
The main purpose of this research is to adapt and validate the AEQ in order to assess pre-adolescent emotions towards Mathematics. Our focus was on those emotions that occur in the school context, that is, class and test-related emotions, and measuring them separately. Not taking into consideration learning-related emotions allowed us to have a relatively short instrument which is satisfactory for this age group, while at the same time measuring all of the discrete emotions addressed by the AEQ. The original AEQ was used as a departure point
instead of the AEQ-M because it is more comprehensive. The AEQ measures nine emotions and includes relief which is not measured by the AEQ-M. Relief is a positive deactivating emotion and without it this category of emotions would not be taken into consideration. To represent the 2 (valence) x 2 (activation) taxonomy of achievement emotions (Pekrun, 2006), all four cells must be represented (see Table 1), which is the case with the AEQ but not the AEQ-M.
(Table 1 about here)
Adapting the AEQ to pre-adolescent students is important since it enables the study of achievement emotions in different academic subjects (here the focus was on Mathematics) and with students of different ages. In pre-adolescence students face an important transition between elementary and middle school, which presents multiple changes and challenges, so it is important to study their emotions towards achievement situations.
Method Participants
The participants were 1515 Portuguese students (51.2% girls) aged 10 to 13 years (M = 11.3 years, SD=1.13), attending the 5th (N=886; Mage= 10.54; SD=.74; 51% girls) and 7th grades (N=629; Mage= 12.41; SD=.49; 52% girls). These students were from 13 public schools in the Lisbon area selected by convenience. The class-related version was initially completed by 1670 students and the test-related version by 1624. In each version, nearly 10% of the participants were excluded due to their age (>13 years old) or because they had missing values on more than five items. Therefore the statistical analyses of the class-related version were conducted with 1469 participants and those of the test-related version with 1515 participants. The difference between the number of participants who completed the
class-related version and the test-class-related version was due to the students that were missing classes when the two questionnaires were administered.
Measures
The AEQ-PA scales to assess class and test-related emotions were adapted from the AEQ (Pekrun et al., 2005) and translated into Portuguese. Several interviews with students were conducted to ensure that the items were understood by pre-adolescent students as well as to check any ambiguity in the items (some of them were rephrased accordingly). Factor structures and reliability were also examined in small scale pilot studies. Since we were adapting the AEQ to pre-adolescents and the original instrument is too extensive, the number of items assessing each emotion was reduced to six by choosing those which presented higher factor loadings in the studies referred to. The version used in this study was a 48-item
questionnaire (6 items for each emotion) both for class and test versions (see Appendix for examples). The class version assesses boredom, hopelessness, anger, anxiety, enjoyment, pride, hope, and shame. The test version assesses the same emotions, except boredom which was replaced by relief. For both versions, items are ordered in three blocks (before, during, and after class or test). Items are answered on a 5-point Likert scale (completely disagree to
completely agree). Procedure
Data for the AEQ-PA class version was gathered in the second term and data for the test version was gathered in the third. Written parental consent was obtained. Students were assured of the confidentiality of their responses and participation was voluntary.
Data Analysis
Confirmatory factor analyses (CFA) were carried out using Amos Version 21.0 (Arbuckle, 2012) to test the internal structure of the scales. Analyses were conducted using
the maximum-likelihood estimation. The goodness-of-fit of the models was assessed through: GFI, CFI, TLI, whose values should be ≥.90 (Kline, 2011) and RMSEA, whose values should be ≤.05 along with a small confidence interval (Raykov & Marcoulides, 2006). Reliability was analyzed using Cronbach’s alpha and Composite Reliability indices were computed following the approach of Geldhof, Preacher and Zyphur (2014).
Results
Preliminary CFAs showed either poor fit to the data or identification problems. This analysis also showed very strong correlations (>.90) between Hope and Pride, Hope and Enjoyment, and Anxiety and Shame, suggesting there were multicollinearity problems between these variables, implying that they cannot be differentiated. Selecting one emotion for each of these pairs makes the instrument shorter without substantial loss of information and avoids problems for subsequent analyses that can be produced by multicollinearity.
These results made us decide to exclude shame and hope but to retain enjoyment and anxiety. The two emotions were retained because they are well-researched, which makes it possible to compare the study findings to the existing evidence. Since reliability scores were >.80 for all emotion scales, in order to have a shorter version of the questionnaire, only the four items displaying the highest factor loadings were retained for each emotion.
(Table 2 about here) (Figure 1 about here)
Two models were separately tested for class and test-related emotions: the first with two factors - negative and positive emotions (Model 1); and the second with six inter-related factors (Model 2). The model with six interrelated emotions is rooted in previous research showing that a multidimensional structure is adequate for representing achievement emotions (Govaerts & Grégoire, 2008; Lichtenfeld et al., 2012; Pekrun et al., 2011). The model with
two factors (positive and negative emotions) is an alternative and plausible model to the six interrelated emotions, leading to the idea of non-differentiation between discrete emotions and also tested in previous research (Lichtenfeld et al., 2012).
The model with six interrelated factors (Figure 1) presented the better fit to the data both for class-related and test-related emotions, showing acceptable goodness-of-fit indexes (see Table 2). The factor loadings and the latent factor correlations for this model can be seen in Figure 1. Intercorrelations between emotions highlight that positive and negative emotions relate negatively and emotions of the same valence are positively associated, both for class and test situations. The sole exceptions are the positive correlations of relief with
hopelessness and anxiety. The final version of the questionnaire is presented in the Appendix. Table 3 shows descriptive statistics, item-total correlations and reliabilities. Skewness and kurtosis values suggest normal distributions, which are reinforced by the small
differences between median and mean for each emotion. The reliability values were good (>= .75) and the item-total correlations were medium to high (Cohen, 1988). These values show that the scales have adequate homogeneity and that each item adequately represents the dimension where it has been included.
(Table 3 about here) .
Discussion
The results obtained in the present study suggest that the AEQ-PA adequately
assesses seven different achievement emotions related to class attendance and taking tests and exams. CFA and descriptive statistics support the reliability and internal validity of this instrument. Despite having fewer items per dimension and being used with younger students, the AEQ-PA presented reliability values similar to the original AEQ (present study: .75 to .93; Pekrun et al., 2011, study: .77 to .93).
The organizational structure used in the present research is different from that found in the AEQ (Pekrun et al. 2011), as we tested the CFA models separately for test and class situations. Despite this and the slight differences in the number and type of emotions assessed, our CFA findings are in line with those in the literature that concluded that a
multidimensional structure is more satisfactory for representing achievement emotions than a single factor (Pekrun et al, 2011) or a simple positive versus negative emotion structure (Govaerts & Grégoire, 2008, Lichtenfeld et al., 2012). Moreover, the pattern of relationships between emotions is similar to findings in Pekrun et al. (2011), with positive relationships between emotions of the same valence and negative relationships between emotions of different valence with the exception of relief. Additionally, and considering construct
validity, preliminary analyses (Peixoto, Mata, Monteiro & Sanches, 2015) on the relationship between the emotions assessed by AEQ-PA and intrinsic motivation or academic
achievement showed a similar pattern of results to those obtained in previous research using the AEQ (Pekrun et al. 2011).
Our findings allowed us not only to understand that pre-adolescents differentiate emotions in each situation, but also opens up the possibility of obtaining different profiles of emotions experienced by students for each situation. In short, the AEQ-PA shows satisfactory psychometric properties indicating that it can be a useful tool for teachers, researchers and psychologists in assessing and understanding pre-adolescent student emotions towards mathematics in both class and test situations.
Some limitations need to be considered. As this adaptation was made with a
convenience sample of Portuguese pre-adolescents in mathematics, more research is needed to test the generalizability of the findings across cultures and other academic subjects. Future research should also address issues such as the hierarchical organization of achievement
emotions as assessed by the AEQ-PA as well as their structural invariance over time and across genders and age groups.
Acknowledgements
This research was supported by the FCT - Science and Technology Foundation under Grant PTDC/CPE-CED/121358/2010.
References
Arbuckle, J. (2012). Amos (Version 21.0) [Computer Program]. Chicago: SPSS.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Frenzel, A. C., Goetz, T., Lüdtke, O., Pekrun, R., & Sutton, R. (2009). Emotional
transmission in the classroom: Exploring the relationship between teacher and student enjoyment. Journal of Educational Psychology, 101, 705–716. doi: 10.1037/a0014695 Frenzel, A. C., Pekrun, R., & Goetz, T. (2007a). Girls and mathematics – A “hopeless” issue?
A control-value approach to gender differences in emotions towards mathematics.
European Journal of Psychology of Education, 22, 497–514. doi: 10.1037/a0014695
Frenzel, A. C., Pekrun, R., & Goetz, T. (2007b). Perceived learning environments and students' emotional experiences: A multilevel analysis of mathematics classrooms.
Learning and Instruction, 17, 478–493. doi: 10.1016/j.learninstruc.2007.09.001
Frenzel, A. C., Thrash, T. M., Pekrun, R., & Goetz, T. (2007). Achievement emotions in Germany and China: A cross-cultural validation of the Academic Emotions
Questionnaire-Mathematics (AEQ-M). Journal of Cross-Cultural Psychology, 38, 302– 309. doi:10.1177/0022022107300276
Geldhof, G., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72-91.
doi:10.1037/a0032138
Govaerts, S., & Grégoire, J. (2008). Development and construct validation of an academic emotions scale. International Journal of Testing, 8, 34–54. doi:
Kline, R. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.
Lichtenfeld, S., Pekrun,R., Stupnisky, R., Reiss, K., & Murayama, K. (2012). Measuring students' emotions in the early years: The Achievement Emotions Questionnaire-Elementary School (AEQ-ES). Learning and Individual Differences, 22, 190–201. doi:10.1177/0022022107300276
Peixoto, F., Mata, L., Monteiro, V., & Sanches, C. (2015). Emotions, motivation and
academic achievement in pre-adolescent students. Paper accepted to be presented in D. Raccanello (Chair), Achievement emotions and correlates from primary school to
university. Symposium to be conducted at the 16th Biennial EARLI Conference,
Cyprus, Cyprus University of Technology
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational
Psychology Review, 18, 315–341. doi: 10.1007/s10648-006-9029-9
Pekrun, R., Goetz, T., & Frenzel, A. C. (2005). Achievement Emotions
Questionnaire-Mathematics (AEQ-M). User’s manual. Munich, Germany: University of Munich,
Department of Psychology.
Pekrun, R., Goetz T., Frenzel, A., Barchfeld, P., & Perry, R. (2011). Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ).
Contemporary Educational Psychology, 36, 36–41. doi:
10.1016/j.cedpsych.2010.10.002
Pekrun, R., Goetz, T., & Perry, R. P. (2005). Achievement Emotions Questionnaire (AEQ) -
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of quantitative and qualitative research.
Educational Psychologist, 37, 91–106. doi:10.1207/S15326985EP3702_4
Raykov, T., & Marcoulides, G. A. (2006). A first course in structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Weiner, B. (2010). The development of an attribution-based theory of motivation: A history of ideas. Educational Psychologist, 45, 28–36. doi:10.1080/00461520903433596 Zeidner, M. (1998). Test anxiety: The state of the art. NY: Plenum Press.
Appendix Table A
Achivement Emotions Questionnaire for Pre-Adolescents (AEQ-PA): Scales and Items of the final version Emotions Item number Item Class Version
Boredom 7 I feel like going out during the Math class because it is so boring. (d) 9 I get bored during the Math class. (d)
13 The Math class bores me. (d) 19 I find the Math class fairly dull. (d)
Hopelessness 1 It’s pointless to prepare for the math class because I don’t understand the material anyway. (b)
2 Even before entering the Math class, I already know that I won’t understand the material. (b)
5 I’d rather not go to the Math class because there is no chance of understanding the material anyway. (b)
16 I have lost all hope of understanding the Math material. (a) Anger 10 I feel anger welling up in me during the math class. (d)
15 Because I’m angry, I get restless in the Math class. (d)
18 Thinking about all the useless things I have to learn in Mathematics, makes me irritated. (d)
21 After the Math class I am angry. (a)
Anxiety 4 I worry about the difficulty of the things I might be asked to do in the Math class. (b)
12 I feel nervous in the Math class. (d)
17 I get scared that I might say something wrong in the Math class, I’d rather not say anything. (d)
races. (d)
Enjoyment 3 I am motivated to go to the Math class because it’s exciting. (b) 6 I enjoy being in the Math class. (d)
11 I feel good when I am in the Math class listening to the teacher talk. (d)
23 I’m glad it paid off to go to the Math class. (a)
Pride 8 I take pride in being able to keep up with the material in the Math class. (d)
14 I am proud of the contributions I have made in the Math class. (d) 22 I think that I can be proud of what I know about Mathematics. (a) 24 Because I take pride in my accomplishments in Mathematics, I am
motivated to continue. (a) Test Version
Relief 16 When I finish the Math test, I finally can breathe easy again. (a) 17 After the Math test, I feel very relieved. (a)
20 After the Math test I feel like I lifted a weight off my shoulders. (a) 23 After the Math test, I feel freed. (a)
Hopelessness 3 Before the Math test, I feel sad / upset because I feel that I don’t have much chance of succeeding. (b)
7 During the Math test, I feel like giving up. (d)
10 During the Math test, I get so discouraged that I have no energy to do the test. (d)
12 During the Math test, I have given up believing that I can answer the questions correctly. (d)
Anger 14 After the Math test, I get angry about the teacher’s grading standards. 15 After the Math test I am fairly annoyed. (a)
18 After the Math test, I wish I could tell the teacher off. (a) 21 After the Math test, I feel very angry (a)
Anxiety 2 Before the Math test, I feel so nervous that I just want to run away. (b)
6 I am very nervous during the Math test. (d)
8 I get so nervous I can’t wait for the test to be over. (d)
9 I feel in panic (nervous and afraid) when I start the Math test. (d) Enjoyment 1 Before the Math test, I am willing to show my knowledge. (a)
4 I enjoy doing the Math test. (d)
11 I am happy that I can cope with the Math test. (d) 13 For me the Math test is a challenge that is enjoyable. (d) Pride 5 During the Math test, I feel proud of my knowledge. (d)
19 After the Math test, I feel so proud I feel on top of the world. (a) 22 After the Math test, I am very satisfied with myself. (a)
24 After the Math test, I am very proud of myself. (a)
Note: b/d/a = before/during/after the situation of attending class, or taking tests and exams,
Table 1
Emotions classification considering valence and activation Activation
Valence Activating Deactivating
Positive Enjoyment Relief Hope Pride Negative Anger Hopelessness Boredom Anxiety Shame
Table 2
Goodness-of-fit indexes for the models tested
χ2 df p TLI CFI RMSEA [90% CI] AIC BIC
Class Model 1 3480.1 251 <.001 .82 .85 .094 [.091, .096] 3626.1 4012.4 Model 2 1138 237 <.001 .95 .96 .051 [.048, .054] 1312 1772.4 Test Model 1 4369.1 251 <.001 .73 .77 .104 [.101, .107] 4515.1 4903.7 Model 2 1243.8 237 <.001 .93 .94 .053 [.050, .056] 1417.8 1880.9 Note. df = Degrees of freedom; TLI = Tucker-Lewis index, CFI = Comparative fit index; RMSEA = Root mean square error of
Table 3
Descriptive statistics and reliability for each dimension of the AEQ-PA
M SD Med Skewness Kurtosis M ri(t−i)a Alpha
Composite Reliability Class Boredom 2.03 1.03 1.74 .69 -.56 .83 .93 .92 Hopelessness 1.46 .72 1.33 1.2 .94 .69 .85 .85 Anger 1.42 .65 1.41 .94 .50 .62 .81 .81 Anxiety 1.68 .66 1.59 .45 -.53 .55 .75 .75 Enjoyment 2.93 .79 3.03 -.67 -.02 .74 .88 .88 Pride 2.93 .65 2.98 -.88 .77 .67 .84 .83 Test Relief 2.54 .72 2.67 -.46 -.29 .63 .81 .81 Hopelessness 1.80 .80 1.69 .54 -.51 .68 .84 .84 Anger 1.46 .61 1.34 .70 .09 .57 .77 .77 Anxiety 2.09 .89 1.93 .31 -.90 .70 .86 .86 Enjoyment 2.52 .71 2.64 -.47 -.35 .63 .81 .82 Pride 2.58 .76 2.63 -.31 -.40 .69 .85 .85
Figure 1. Six-factor model for both class-related and test-related emotions. In test-related
emotions Boredom was replaced by Relief. B = Boredom; R = Relief; H = Hopelessness; Ang = Anger; Anx = Anxiety; E = Enjoyment; P = Pride. Read from left to right the digits represent factor loadings, and latent factor correlations, respectively for class-related/test-related emotions
