The way students’ internalize assessment criteria on inquiry reports

The

way

students

’

internalize

assessment

criteria

on

inquiry

reports

Inês

Bruno

*

,

Leonor

Santos

,

Nilza

Costa

a

EscolaSecundáriaJoséSaramago,AvenidaCidadedeLeiman,2640-470Mafra,Portugal

b

InstitutodeEducaçãoeCentrodeInvestigaçãoemEducação,UniversidadedeLisboa,AlamedadaUniversidade,1649-013Lisboa,Portugal

c

DepartamentodeEducaçãoePsicologia,CentrodeInvestigaçãoemDidáticaeTecnologianaFormaçãodeFormadores,UniversidadedeAveiro,Campus UniversitáriodeSantiago,3810-193Aveiro,Portugal

ARTICLE INFO

Articlehistory: Received12March2016

Receivedinrevisedform9September2016 Accepted12September2016 Availableonlinexxx Keywords: School-basedevaluation Evaluationmethods Studentevaluation Assessmentcriteria Self-regulatedlearning Highschool ABSTRACT

Thispaper,basedontwocasestudies,presentsaninterpretativeresearchontheprocessesusedbyhigh schoolstudentstointernalizetheassessmentcriteriaofPhysicsandChemistryinquiryreports.Findings support thatunderstandingtheassessment criteriaiscomplex,mainlybecauseof itsterminology. Howeverthediscussionofexemplarscanhaveacrucialroleinthisprocess.Inordertooperationalizethe assessmentcriteria,studentsusedstrategiesstemmingfromthesocialcontext(teacher,peersandother didactic sources)aswell asfromtheirindividual experience (errorsmade, engagementin critical thinkingandthecreationofafavorableenvironment).Howeveritalsoshowedthatsomedifferencesin operationalizingtheassessmentcriteriawererelatedtodifferentstudents’profile.Theresultspointtoa multi-strategy pedagogical approach to enhance students’ internalization of assessment criteria. Neverthelessreducing thetension betweenteachers’expectations and students’ownstandards of qualityshowedtobeacomplexprocess.

ã2016ElsevierLtd.Allrightsreserved.

1.Introduction

Modernsocietiesrequireindividuals,andthereforestudentsin particular,tobeabletocontinuouslyacquirenewknowledgeand competences.Itisveryunlikelythatstudentssucceedunlessthey reflecton themselves,onwhat and howthey aredoing (Dann, 2002).Thus,studentsneedtohaveanactiveandconstructiverole inthelearningprocess(Nunziati,1990;Zimmerman,2002).Inthis sense, self-regulation is an essential competence because it includes“themechanismsoforientation,control,andadjustment ofthecognitive, affectiveand socialactivitiesthat promotethe transformationofstudents’competencies”(Allal,2007,p.9).The assessmentcriteriaplayacrucialroleinthisprocessbecausethey canguidestudentstoachievetheobjectivesofatask(Nunziati, 1990).

However,each individualestablishes his/her own standards, criteria or self-representations, so it is essential to use a pedagogicalapproach that fosterstheinternalizationof assess-mentcriteria,assuggestedbyAndradeandDu(2007),Santosand

Gomes (2006) and Semana and Santos (2010). According to

Vygotsky (1934/1978), the internalization of the assessment

criteria means that students integrate them in their own knowledge.Inotherwords,itimpliesthat they:i)interpretthe criteria,understandtheirmeaning,reachacommon understand-ingandappropriatetheteacher’sexpectations(Vial,2012),andii) operationalize the criteria, in other words putting them into practice, because “criteria assume meaning only when used” (Woolf,2004,p.488).

In this articlewe assumethat understandinganassessment criterionpresupposesthatthestudentgivesthepropermeaningto allthecontentofthecriterionstatement(matchingtheteacher’s interpretation) and can explain it in his/her own words. Operationalization of a criterion meansthat students useit in speci_ficsituations.

Accordingtoseveralstudies(e.g.Kirby&Downs,2007;Rust, Price, & O’Donovan, 2003; Tillema, 2014), informing students abouttheirassessmentcriteriaisusuallyinsuf_ficientfor internali-zationtooccur.Infact,theinternalizationis acomplexprocess (Andrade&Du,2007;Clark,2012).Forthisreason,andaccording toseveralstudies(e.g.Tierney,2013),theinternalizationprocess needstobedeeplyinvestigated.Thepresentpaperaimstofillagap instudiesfocusingonthisprocess.Theresearchwasundertakenin order to understand how high school students internalize assessment criteria. It was guided by the following research questions:

* Correspondingauthor.

E-mailaddresses:inesbruno@gmail.com(I.Bruno),mlsantos@ie.ul.pt(L.Santos) ,nilzacosta@ua.pt(N.Costa).

http://dx.doi.org/10.1016/j.stueduc.2016.09.002

0191-491X/ã2016ElsevierLtd.Allrightsreserved.

ContentslistsavailableatScienceDirect

Studies

in

Educational

Evaluation

1.How do high school students understand the assessment criteriaofPhysicsandChemistryinquiryreports?

2.Whichstrategiesdostudentsusetosupportthe operational-izationofassessmentcriteriawhentheydevelopPhysicsand Chemistryinquiryreports?

3.Whatistheroleofthepedagogicalapproachinthe internaliza-tionoftheassessmentcriteria?

2.Conceptualframework

Self-regulation canbede_finedas _“amultilevel, multicompo-nentprocessthattargetsaffect,cognitions,andactions,aswellas featuresoftheenvironmentformodulationintheserviceofone’s goals” (Boekaerts, Maes, & Karoly, 2005, p. 150). Therefore, it assumesthatstudentsareactivelyinvolvedinthelearningprocess. According tothe model proposed by Zimmerman(2002), self-regulationincludesthreephases:theforethought(wherestudents definespecificgoalsandplanstrategies),theperformance(where studentsmonitortheirperformance)andtheself-reflectionphase (wherestudentsreflectonthemethods,knowledgeacquiredand theimportanceoftheprocesstheyusedtoachievethegoals).

Self-regulationfostersanawarenessofwhenandhowstudents learn, or not (Black, Harrison, Lee, Marshall, & Wiliam, 2002;

Swaffield,2011).Itimpliestheyconstructtheirownlearningpath also through the understanding of assessment criteria, self-assessmentandself-correction(Nunziati,1990).Thisisessential forunderstandinghowsuccessfultheirworkwas,aswellasfor identifyingandcorrectingtheirmistakesanderrors(Dann,2002;

Papaleoutiou-Louca, 2003; Zimmerman, 2002). But for this to

happen,itisnecessarytomakeexplicitwhatusuallyisimplicit, promotingtheunderstandingofassessmentcriteria.

Assessment criteriaarerepresentationsof variousaspectsof the task (Nunziati, 1990); therefore its significance must be interpretedsothatstudentsunderstandwhatisactuallyrequested (Vial,2012).Thisprocesscanoccurunilaterallyorbilaterally,with theteachereither informing or negotiating assessment criteria with students, respectively. In fact, the usefulness of the assessment criteria depends not only on their understanding, butalsoonthedegreeoftheiracceptance(Hadji,1989).Itiscrucial that,fromthepointofviewofthestudents,theymakesense,that is, they must beaccepted asimportant and legitimate. So, the bilateral process has the advantage that responsibility in the assessmentprocessissharedwiththestudents(Gipps,1999).As such, an open and constructive dialog between teacher and students can help them to understand and consider the task requirements(Woolf,2004).

Althoughassessmentcriteriaoftenrefertostudents’tasks,they areabstract(e.g.rigor,clarity),thusitisessentialthatstudentsare awareaboutthequalitiestheirworkshouldhave.Ausefulwayof doing this is by using rubrics (Brookhart, 2013). These are documentsthatdemonstratethecharacteristicsthatareexpected fromaparticulartask,indicatingwhatleadstoahighmarkand describinglevelsof quality, forexamplefrom excellenttopoor (Andrade,2000).Rubricscanbeanalytical,whentheperformance isdescribedoneachcriterionseparately,orholistic,ifthecriteria aretreated together.They alsocan be generic when a general performanceisdescribedandinthiswaytheycanbeusedwitha familyofsimilartasksorbetask-specificifthedescriptionofthe performanceis specifictoa singletask.AccordingtoBrookhart (2013),usinganalyticrubricsismoreadvantageousforformative assessmentbecauseitcanhelpstudentstoidentifywhataspectsof theirworkneedattention.Thehighestlevel,whichindicatesthe characteristicsofanexcellentwork,makesstudentstoengageina processofconstructivelearningbasedonself-regulation(Hafner& Hafner,2003).

Asseveralstudiespointout(Andrade,2001;Green&Bowser, 2006;Santos&Semana,2015),itisnecessarytoexplorerubrics withstudents inordertoreachasharedunderstanding.Aclear explanationofwhatisexpectedhelpsstudentstounderstandthe moreunclearorless obvious items(whichwouldotherwise be ignored),andovercomethevague,impreciseandsubjectiveparts ofsomeindicators.Inthecaseofmoreabstractcriteria,itisuseful to give concrete examples for discussion, such as previous assignments of thestudents or otherstudents’ assignments, as suggestedbyAndrade,Du,andMycek(2010),aswellasbyHendry,

Armstrong,andBromberger(2012).Sometimesstudentsneedto

seehowassessmentcriteriacanbeapplied.Thisisausefulwayto helpthemunderstandandoperationalizethosecriteria,orinother words,tointernalizethem.However,itisimportanttodissuade studentsfromusingmechanisticstrategiesorthinkingthatthese examplesarestandard(Norton,2004).

Feedbackcanalsobeusedtopromotetheunderstandingofthe assessment criteriabecauseambiguitiesand misunderstandings maybediminishedthroughthisprocess(Taras,2003).According toNicolandMacfarlane-Dick(2006)andNicol(2010),feedback, understood as a dialog process, may: i) help clarify what constitutesagoodperformance,whichiscrucialforstudentsto set goals and guide them in the process of self-regulation; ii) encourage students to identify the criteria and standards that applytotheirworkandmakejudgmentsbasedonsuchstandards; iii)givehighqualityinformationtostudentsabouttheirlearning. This information should help them improve their own perfor-manceandcorrectpossiblemistakes.Itshouldleadstudentstoact sothatdiscrepancybetweenintentionsandresultsisdiminished (Black, Harrison, Lee, Marshall, & Wiliam, 2003). In this way feedback can also foster students’ operationalization of the assessment criteria.Therefore,it must bestrictlyrelatedtothe objectives,standardsandcriteriaandstudentshavetounderstand thisrelationship(Wharton,2003).

Feedbackalsogivesstudentstheopportunitytoengageinother waysoflookingatthetask,sothatotherpathscanbeconsidered (Taras,2001).However,inthelongrun,theamountanddetailof feedbackshouldbeminimizedinordertopromoteanautonomous learningandthedevelopmentofskillsthatenablethestudentsto analyzetheirworkindependently(Black&Wiliam,1998;Crisp, 2012;Santos&Pinto,2009;Swaffield,2011).

The dialog between peers can also be beneficial for the internalization of assessment criteria to occur (Price, Handley, Millar,&den Outer,2007)since thelanguageusedby peersis easiertounderstandandthecontactwithotherpointsofviewand alternative strategies help students to review and build new apprenticeships(Blacketal.,2003).

Previous studies considered the potential of enhancing the internalization of assessment criteria for increasing learning capacity. Students can identify more effectively the strengths and weaknesses of their processes and products, have better performances,bemoremotivatedandless anxiousanddevelop skills inherent toself-regulated learning (Andrade &Du, 2007; Dann,2002;Rustetal.,2003).

3.Methodology

Thisstudyexaminestheprocessesusedinthedevelopmentof inquiry reports guided by assessment criteria. Therefore, and takingintoaccountthat theresearchquestionsare designedto address the understanding of the process of internalization of thosecriteria,wehavechosenaninterpretiveparadigm(Lichtman, 2006).Aqualitativecasestudywasdesignedforthisinvestigation. According toMerriam (1988), thequalitative case study‘is the idealdesign for understandingand interpretingobservationsof educational phenomena’ (p. 2). As a result, it was possible to

understandhowhighschoolstudentsinternalizetheassessment criteriaofPhysicsandChemistryinquiryreports.

3.1.Participants

ThestudytookplaceinaschoolinPortugal,andinvolvedone classofChemistryandPhysics,overtwoschoolyears.Inthefirst year,thefirstauthorofthispaperwastheteacheroftheclass,and inthesecondyear,anotherteachertookoverbutworkedclosely withthefirstoneonthepedagogicalinterventiondevelopedinthe study. These teachershad respectivelysix and twentyyears of professionalexperienceandboth werefamiliar withtheuseof inquirytasksintheclassrooms,but hadalimitedexperiencein usingpedagogicalapproaches topromotetheinternalizationof assessmentcriteria.Thestudents(15/16-years-old)wereinformed about the nature and objectives of the study. Consents were obtainedfromallstudentsandtheirparents.Twostudentswere selected for case studies: Sara and Gustavo (participants were givenfictitiousnamestoensureconfidentialityandanonymity). Theseparticularstudentswerechosenbecause,whiletheyhada goodabilitytoexpressthemselves,theyhaddifficultyinreflecting abouttheirownwork.Inaddition,theywereofdifferentgender, performance level (between average and good) and they were availabletobeinterviewed.

3.2.Pedagogicalcontext

Fourinquirytaskswereimplementedoverthe10thand11th gradesofthesameclass(fromNovember2008toJanuary2010).As preconizedintheNationalScienceEducationStandards(National ResearchCouncil,1996),thesetasksinvolvedmakingobservations, examiningsourcesofinformationtoseewhatisalreadyknown, planning investigations, using tools to gather, analyze and interpretdata,proposinganswers,explanationsandpredictions, andcommunicatingtheresults.Thesetaskscanbeclassifiedas guidedinquiryactivities(Bell,Smetana,&Binns,2005)giventhat the inquiry questions were introduced by the teacher but the answersweregivenbystudentsafterdesigningaprocedureand analysingthecollecteddataindependently.Theguidelinesgivento students’ work in the third task is presented as an example (AppendixA).

Writing requires students “to reflect, consolidate, elaborate, reprocessconcepts and ideascentral tothe topic,hypothesize, interpret,synthesize, and persuade, and hence develop higher-orderthinkingandtheconstructionofadeeperunderstandingof scienceconcepts”(Yore,Bisanz,&Hand,2003).Therefore,students were requested to develop reports containing five sections: introduction,planning,procedure,resultsandconclusions.Three of the reports included the development of a second version consideringthewrittenfeedbackprovidedbytheteacher.

Studentswerealsorequestedawrittenre_flectionabouteach reportthat consistedin assigningagrade leveltoeach section consideringthe levels of therubric (Appendix B) and giving a justi_fication(pointingoutitsstrengthsandweaknesses).

Eachinquirytaskwasdevelopedina135minuteslesson(1st lesson). The reports and written reflections of the second and fourthtasksweredoneduringthenexttwo 90minuteslessons (2ndand3rdlessons)andthereportofthethirdtaskaswellasthe revisionofthefirstversionofthereportofthefourthtaskwere doneintheschoollibraryaftertheschoolwork.

Thepedagogicalapproachusedin thisstudytopromotethe internalizationofassessmentcriteriaincluded:

i) the discussion of the assessment criteria and the use of an analyticalrubric.Beforethefirsttask,theteachergavestudents someguidelinesaboutthestructureofthereportthat included somequestionsforeachsection,suchas:“Whatconclusionsdid

you reached? Whichwerethelimitations of thestrategies you used?Wouldyouchangethewayyousolvedtheproblem?How? And why?” Based on these guidelines, students were askedto collaborate in the selection of the most relevant assessment criteria,describingthecharacteristicsofaqualityreport.Lateron, the teacher developed a rubric taking into account students’ suggestions.Thiswasdeliveredtothestudentswhentheystarted thefirsttask.AssuggestedbyGriffin(2009),somechangeswere made after the teacher/researcher had analyzed the first task reports andtheclassdiscussion(inthelightof therubric) that followedthedeliveryofthefinalversion(wherestudentsanalyzed the quality of their performance considering each assessment criterion and made somesuggestions aboutthe contentof the rubricdescriptors).Thisnewrubricwasusedbystudentsinthe nextphasesofthestudy(AppendixB).

ii)thewrittenandoralfeedback.Bothtypesoffeedbackwere usedinaconstructiveway.Thecomments,usuallyprovidedinthe formofquestionsorhintsbytheteacher,aimedtogivestudent informationabouttheirperformanceinrelationtoanassessment criterion inordertohelpthemtoidentifythecauseofthegap betweenthedesiredandtheactuallevel,aswellastofillthatgap and to understand how to successfully operationalize that assessmentcriterion(Black&Wiliam,1998).

iii)thediscussionofanexemplarreport.Duringthediscussionof thereportsofthesecondtask,anexemplarreportwasgiventothe students,assuggestedbyseveralauthors,likeAndrade,Du,and Mycek(2010).Thisexemplarreportwasthecompilationofthe bestpartsofthereportsdonebythestudentsofthisclass.Itwas carefully analyzed with the students through an open dialog between them and the teacher. Regarding each assessment criterion,studentswereaskedto:i)explainthemeaningofsome wordsintherubric,wheneveritseemednecessary(e.g.synthetic), and ii) do a reasoned judgment about the quality of its’ operationalizationintheprovidedexample.

iv)thedialogbetweenpeers.Onlythefirsttaskwasdeveloped individuallyforthepurposeofgatheringdatatoselectthecase studies(thoughinteractionbetweenthestudents wasallowed). The other tasks as well as the respective reports and written reflectionsweredonebygroupsofthreeorfourstudents.These groups compositionchangedfromtasktotaskandtheselected students,SaraandGustavo,werealwaysintegratedintodifferent groups. The interaction between students was systematically encouraged.Theteachergenerallygavefeedbackonlywhenshe wasaskedtoandaftergroupdiscussionaboutthatissue. 3.3.Collectionofdata

Observation,interviewanddocumentalanalysiswereusedfor collecting data. Observations and interviews were both audio-tapedandlaterfullytranscribed.Thefirstauthormadeparticipant observation(Cohen,Manion,&Morrison,2000)duringthelessons in which students performed the tasks and developed the respectivereportsandwrittenreflections.

Semi-structured interviews (Burns, 2000) were carried out with the two case-study participants after submission of the lastversion of eachreport andat theend of thestudy.These allowed students to reveal feelings, intentions, meanings and thoughts about a topic or situation (Lichtman, 2006). The questions sought toencourage the analysis of each section of the report in order to understand the processes used for internalizingtheassessmentcriteriaandthereasonsbehindthe less successfulitems.

Documents provide access tothoughts, ideas and meanings expressedbytheirauthorsandmayraisequestionsthatcanbe taken up in subsequent moments of datacollection (Lichtman,

consistedmostlyofstudentreportsandwrittenreflections.These assignmentsreflectamoreorlesssuccessfulinternalizationofthe assessmentcriteria,sotheiranalysisraisedquestionsthatguided furtherdatacollection.Examplesofstudentassignments,aswell asexcerptstranscribedfromtheirreports weretranslatedfrom Portuguese.Theexcerptsofthereportsareidentifiedbytheletter Rfollowedbythenumberofthetaskandthenumberoftheversion (V).

3.4.Dataanalysis

Dataanalysiswasconductedintwophases.The_first,whichwas developed during the data collection, raised issues to be considered in subsequent data collection. The second, more thoroughandorganized,occurredafterthefieldworkandaimed toanswer theresearch questions of the study. The method of questioningandconstantcomparison(Strauss&Corbin,1998)was applied,usingcategoriesdefinedbothduringtheanalysisprocess andonthebasisofthetheoreticalframework.

Asstatedbefore,theinternalizationoftheassessmentcriteria includes the understanding and the operationalization of the criteriainaspecificsituation(Vial,2012;Woolf,2004).Thus,the following categories were considered: i) understanding and ii) strategiesusedinthe operationalizationofthe assessmentcriteria. Regardingthesecondcategory,somesubcategoriesemergedfrom theanalysis:discussingwithpeersandaskingforhelp,searchingfor information, learning with the errors, critical interaction with themselvesandcreating afavorable environment.Therole ofthe pedagogicalapproachtofostertheinternalizationofthe assess-mentcriteriawas studiedina transversalway,focusing onthe discussionoftheassessmentcriteriaandtheuseoftherubric,the feedback,theuseofanexemplarreportandthedialogbetween peers.

4.Results

4.1.ThecaseofSara

Sarais astudentwitha goodperformanceinthesubjectof Physics and Chemistry. Her final grade was fourteen out of twenty1 on both 10th and 11th high school years. She is a hardworkingstudent,committedtoachievehighmarks,butshe oftenfeelsinsecureaboutherperformance.Sheoftenaskedher peersand teachers for help andsometimes shepreferred not performingthetaskratherthanmakingmistakes:“Iknewitmore orless,butIdidn’tknowifIwasright,sinceIdidn’twanttofail,I didn’twrite anything” (1stInterview). At thebeginningof the study,shehad limitedexperience incarryingout inquirytasks and developing lab reports as well as assessing her own assignments: “Usually teachers tell me what to do, help me andnowIhavetothinkanddoitbymyself.Thatiswhyitwas difficult”(1stInterview).

4.1.1.Understandingtheassessmentcriteria

InSara’sview,theinitialdiscussionoftheassessmentcriteria, which lead to the establishment of the rubric, helped her to understandwhatwasrequiredtodoineachsectionofthereport: Researcher—Wehavediscussedtheassessmentcriteriaofthe reports.Doyouthinkthiswashelpful?

Sara—Yes, because, as it often happens, when I’m doing an assignment,I’mnotsureaboutwhatIneedtoincludeornotand inthiswayIunderstoodwhatIshouldorshouldnotdo.Yes,I

thinkithelped.(1stInterview)

However,theevidencesuggeststhatthisinitialdiscussionand the reading of the rubric, by itself, did not promote the understandingofsomecriteria,mainlythosethatincludedaspects orexpressionsshewasnotfamiliarwithandthecriteriainvolving theuseofcomplexskills.

Forexample,oneoftheassessmentcriteriawas“definingclear and synthetic objectives consistent with the task”. Initially, it appearsthatSaraandhercolleaguewerenotabletogiveaproper meaningtothewordsynthetic:

StudentA—Whatissynthetic?Synthetically?Clearandsimple, isn’tit?

Sara_—Yes,Ithinkso.(2ndTask;3rdLesson)

However,theanalysisoftheexemplarreportappearstohave promotedtheunderstandingthatthegoalsshouldbedirectand concise.Thisideaisbasedon,firstly,thefactthatonthesecond interview(which occurredafterthe discussionof theexemplar report), Sara pointed out that the objectives were one of the weaknessesofherreportbecausetheywerenotconcise[sentences 1and2below];secondly,inthereportofthethirdtask(R3),she was able to formulatea synthetic goal and in the subsequent interviewshedescribeditscharacteristics[sentences3and4]:

1.Sara—Weshouldexplainourgoalsexplicitlyandwewere... Basically,insteadofsayingtheessential,wewereverbose,let’s say.

(...)

2.Sara—Then,withthereporttheteachergave,whichwasan example(...)Inoticedthattheywentstraighttothepoint anddidn’trambleabout.Theyjustsaywhatitisandthiswas notquitewhatwehavedone.(2ndInterview)

3.Researcher—Whatisyourunderstandingofasyntheticgoal? 4.Sara—Notverylongandnotscatteredthroughoutthetext (...)saidinaplainlyway.(3ndInterview)

Theobjectivesofthisactivityweretocollectexperimentaldata toknowwhatwasthemosteffectivewaytocoollemonade.(R3) Regarding the indication of alternative strategies to solve the problem,Sarainitiallyhadsomedifficultytounderstandwhatwas expected.So,duringthefirsttask,theteachergaveheranexample that could be solved using different separation techniques (somethingthatwasfamiliartoherbecausethissubjecthadbeen previouslystudied).But,thediscussionintheclassroom concern-ingthedifferentinstrumentsthatcouldbeusedtomeasurethe volumeofsandseemed tohaveledSaratoconsiderthat these differentoptionsweredifferentstrategiestosolvetheproblem.In the report, she identified several instruments and presented argumentstojustifyherchoice:“Tohaveagreateraccuracyinthe resultsIchosetouseameasuringcylinderbecausethebeakerhasa low precisionand thepipetteis notsuitablefor measuringthe volumeofsand(asolid)becauseitisusedtomeasurethevolume ofliquids”(R1_V2).

Theevidence suggeststhatthediscussionaboutthewritten reports(whenthesecondversionwasdeliveredtothestudents) helpedhertocreateawiderunderstandingofwhatanalternative strategyis.Duringtheinterview,thatoccurredthedayafterthis class,Sarawasabletocriticizeherassignment,identifyingwhy alternative strategies were missing and why she previously thoughttheywerepresent:

Sara—Ididn’twroteseveralstrategies,Ionlywroteone(...)I thoughtit wastheinstrument thatwe usedtomeasurethe volume,thoughtitwasthebeaker,thepipette ... andIdidnot knowthat it was expecteda reallydifferentstrategy,then I assessedmyselfwithlevel3,butitisnotcorrect.

1

InthePortugueseeducationalsystemgradesrangefrom0to20,with10or higherbeinga“positive”passingmark.

Researcher—Okay,sohereyou’ve consideredthatpresenting several instruments to measure meant to present different strategies,that’swhyyouassessedyourselfwithlevel3. Sara—AndbecauseIalsoexplainedwhy,IthoughtIwasdoing ok.(1stInterview)

Anotherassessmentcriterionthatwasnotinitiallyunderstood bySarawasthepresentationofevidencetosupporttheconclusions. Astheteacherdrewattentiontotheimportanceofpresentingfacts thatsupporttheconclusions[sentence3],Sarasuggestedtotake thisoralfeedbackintoconsideration,butsheincludedthevaluesof thecontrolledvariables[sentence4],insteadofthefindingsthat could justify her conclusions. Besides, the comment that was writteninthe_firstversionofthereportofthesecondtask(_“In theseconditions,howmanymolecules/atomsdoyouhave?You shouldalwaysjustifyyourconclusionsbasedonyourresults”)was noteffectiveaswell.

Duringthefollowinginterview,Sararevealedthatshehadnot previouslyunderstoodthisassessmentcriterion[sentences10–13] andcouldnotfinditontherubricaswell[sentences8and9]:

1.Researcher—So,ifweareinthesameconditionsofpressure, temperatureandthevolumeremainsconstant...

2. Sara—It will always have the same number of atoms or molecules.

(...)

3. Researcher—Exactly. And now you should justify that conclusionpresenting thefindings of yourexperiment, your simulation.

(...)

4.Sara—Youhavetoinsertthevalueofpressurebecauseyou havetoincludeyourresults.

5.StudentA—Isthatso?

6.Sara—Yes.(2ndTask;1stSession)

7. Sara—We didn’talways include ourfindings. We didthat regardingsomeconclusions,butnotallofthem.

8.Researcher—Butwherecanyoufindthisorientationinthe rubric?

(...)

9.Sara—Itisnothere[intherubric].Itisabouttheconclusions, butitisnotwritten.Itisnothere,butIknowthatweneedto includeit.

10.Researcher—Sowhatdoesthis mean:theconclusionsare explicitlybasedonevidence?

11.Sara—(silence).Ithinkitis:Idrawconclusionsofwhatwe foundwithwhatreallyis... HowcanIsayit?Whatwefound withthereality,inthiscasewithAvogadro’sLaw.Ifourresults wereinlinewiththeAvogadro’sLaw.

12.Researcher—Andwhatdoesthis mean:thatareexplicitly [emphasisonexplicitly]basedonevidence?

13.Sara—Well,Idon’tknowthat.(2ndInterview)

Inthethirdtask,Saraexplicitlyreferredtothefindingstojustify theconclusions,thussupportingthefactthatthediscussionand analysisof the exemplarreport helped Saratounderstand the teacher’sexpectationsaboutthisassessmentcriterion.

Finally,itwaspossibletorealizethatSaradidnotunderstand whatwasmeantaboutexplaininghowtheidentifiedsourcesoferror affecttheresults.Whenaskedaboutthepresenceofthisiteminthe report,Sarainitiallyconsidereditwaspresent[sentences1—4].But thensheconsideredthatitwasnotpresentandthatshehadnot understoodwhatwasrequired[sentences5–10]:

1.Researcher—Didyouexplainhowyourresultswereaffected bytheerrorsyouidentified?Howdoesthefactthatthesystem isnotisolatedaffectyourresults?

2.Sara—Yes.

3.Researcher—Wheredidyouexplainthat?

4.Sara—Itishere,forexample... (...)

5.Researcher—So,didyouidentifytheerrorsandexplaintheir effectsontheresults?

6. Sara—No. (...) Becausewe explained the errors but we didn’texplainhowtheseerrorswould...

7.Researcher—Affecttheresults? 8.Sara—Yes

9.Researcher—So,basicallyyouhavenotyetrealizedwhatthis means,isthatit?

10.Sara—No,IguessIhadnot.(3thInterview)

4.1.2.Strategiesusedintheoperationalizationoftheassessment criteria

Sarausuallystartedbyreadingthehigherlevelofqualityofthe rubriceventhoughshedidnotseemtodoanin-deepanalysisof thedescriptors:“whenIreadit ... sometimesIdon’tpaymuch attentiontothemoredifficultsentences.Idon’tunderstand,and thenIpretendthatithasalreadybeendone”(1stInterview).

Afterreadingsheusedanumberofstrategiestooperationalize someoftheassessmentcriteria,suchasdiscussingideaswithpeers andaskingforhelp.Itseemsthattheinteractionssheestablished with peers and the teacher not only aimed at overcoming difficultiesandidentifyingerrors,butalsoasaformofvalidation: Researcher—Howdidyouovercomethedifficultiesthathave arisen?

Sara—First,Icomparedtheresultswithmycolleagues,asked themhowtheyweregoingtodo.(...)Thentheysaidwhat they were going to write, and I tried to do my best. (1st Interview)

Sara—Teacher,intheprocedure,canwewrite:Firstly,secondly, thirdly?(1stTask;2ndSession)

Sara—Wehavetoindicateanameforthetable.(...)Wecan writedatarecord,right? Thenyou write:Table1.(...)Ah, whatdowewrite?Ah,measurementsmade?

Student B—Wait, we just measured the mass and the temperature.(3rdTask;1stSession)

Sara—Don’twehavetoindicatethevalues?Whatdoyouthink? (...)Weaddiceinthemeasureof...

StudentB—No,weadd53gofice.(3rdTask;1stSession) Sara—Ithinkthereissomethingwrong,Iamsure.Isthisokay likethis?(3rdTask;1stLesson)

The discussions and requests for assistance occurred in all sectionsofthereport,buttheyweremorefrequentduringthedata analysis.Regardingthisaspect,althoughthecalculationsusually had been correctlydone, the reasoning was not always under-stood:“NowIknow,butIdidn’trealizethisbefore,whywesaid thattheheat ... ah,whyweequal(...)Icouldn’tunderstandit well”(3thInterview).

Inordertoproduceamorecompleteassignmentandovercome difficulties, Sara also searched for information on the web, the school textbook, the former reports and the exemplar report. Usually, she incorporated the information almost verbatim (in particularwithregardtothedefinitions):

Sara—WeusedtheInternettoseethedefinition.Wedidn’twant towritewhatweknewotherwiseitwouldbepoorlyexplained, butthenwhenwetriedtofitit...

Researcher—WhatdefinitiondidyousearchontheInternet? Sara—Thevolume,themolarvolume.Then,whenwetriedto addthis,Ithinkitendedupwithsomerepetitions.But,then,if we had changed it, it would become even worse. (2nd Interview)

Thefearofmakingmistakesandnotbeingrigorous,aswellas thelackofconfidenceinherselfseemstohavebeenoneofthe

reasonswhySara:i)fullycopiedsomeinformationavailableonthe Internet:““Ithinkthispartisverywelldonewhilethisotheroneis notsogoodbecausewewroteourownthoughtsonthepaper.But itwasnotthesamecasehere,wesearchedforandinsertedthe information(...)Thewayweexpressourselvesisnotgoodatall” (2ndInterview);ii)transcribedthebeginningofsomesentencesof theformerreports:“Wesawwhatwehadwritten,howwehad begunsentences” (2ndInterview);iii)usedthestructureofthe procedureoftheexemplarreport,despitetheproceduretheyhad previouslydonealsobeingcorrect: “We did[the procedure]in topics,butthenasthatexemplarreportwasdoneasanarrative,we thoughtitwasmoreorganizedanddecidedtodothesame.Sowe justhadtoswitchtothenarrativeformat”(3thInterview).

However,regardingthedrawingoftheexperimentalsetup,the exemplarreportwasnotusedinamechanisticwayandseemsto havebeencrucialfor theoperationalizationof thiscriterion. In spiteofthefactthattheteacherhadalreadyactedtopromotethe internalizationofthiscriterion,shewasonlyabletooperationalize itwiththesupportoftheexemplarreport:

Includingadiagramofthesimulationcouldhelpus compre-hendtheprocedure,don’tyouthink?(writtenfeedbackinthe firstversionofthesecondtask’sreport)

Sara—Yes,Ithinkthatadiagramwouldhavehelped(...)We didn’tthinkthatwecoulddoitasLuisadid[referringtothe exemplarreport](...)And,becausewewerenotabletodoit asitshouldbedone,itwouldbeaninvention,wepreferrednot tomakeadiagram.(2ndInterview)

While doing the report of the third task, Sara told her colleagues:“Look, we shoulddo it [the diagram]theway it is here[theexemplarreport].Ithinkitisverywelldone”(3rdTask; 1st Session) and then she managed to develop an adequate diagram(Fig.1).

LearningwiththeerrorswasanotherwayforSarato operation-alizetheassessmentcriteriamoresuccessfullyovertime.Inthe secondtask,attentionhasbeendrawn(throughoralfeedback)for not rounding numbers when making calculations. From that momenton, shebecamecarefulabouttheaccuracyofthedata analysis,beingawareofthereasonsforthisprocedure:

StudentB—Areyougoingtointroducealldigits?

Sara—Iwill,otherwisewewouldhaveinaccurateresults.(3rd Task;1stLesson)

Furthermore, concerningthescientificterminology, Saradid not repeat some of the mistakes on subsequent reports. For instance,insteadofusingincorrectlythewordweight(pointedout bywrittenfeedbackonthe1streport:“Doesthescalemeasures weight?”)shestartedtousethewordmass.Herspeechsupports theideathatshetriedtotakeintoaccountthefeedbackprovidedin theformerreports:

Researcher—Were there some aspects that you paid more attentiontointhereportofthistaskbecauseofthereportyou haddonepreviously?

Sara—Yes, Iremembered that Ihad not putthe title onthe tables.ThiswasthefirstthingIremembered:toputthetitleon thetables.(2ndInterview)

Identifyingthelesssuccessfulaspectsinthepreviousreports (throughthefeedbackandsupplementedwiththediscussionand analysisofthem)alsoseemstohavecontributedtoan improve-mentintheintegrationofconceptsinthetheoreticalframework. Forexample,inthereportofthefourthtask,theconceptswere explainedinamoreabstractwayanddidnotconstituteadirect description and explanation of the physical phenomena that occurredinpractice.Expressionslike“wehavetocalculate ..."or “wemultiplythisvalueby ..."nolongerexisted.

Withregardtotheplanning,therewasanimprovementinthe descriptionofthecalculationsthatwouldhavetobemadeinorder toanswertheproblem—itbecameclearerandmorecorrect.For example,inthesecondtask,Saraoftenwrotethattheylinkedthe variable x with y, but usually it was not explained how these variables were linked: “Explaining what we calculate is more difficult.(...)Wewerenotabletoexplainwhatwehaddone.” (2ndInterview).Butinthetwolastreports,Saraincluded,inthe description,the mathematical expressions,which facilitated its comprehension.

Withrespecttotheconclusions,thesourcesoferrorsthatwere identifiedbecamemorespecificovertime.While,inthereportof thefirsttask,Saraonlywrotethataccidentalerrorsoccurred,inthe followingreportsthesourcesoferrorsbecamemorespecifictothe taskandmoredetailed:

The largest temperature difference occurred in the ice experimentbecausemoreerrorswerecommitted.Oneofthem is related with the fusion of the ice. Since the outside temperature is higher than the ice temperature (so theice meltsquickly)andwetooksometimetoaddthethermometer inthebeakerandtomeasurethetemperature,wehavemadean error.Wealsomadeothererrors,suchasmeasurementswith thescalesinceafewdropsofwatermayhavebeenoutofthe beaker... (R3)

Lastly,itisimportanttoaddthatsometimestakingintoaccount theerrorsmadepreviouslywasnotenough.Thisstrategyhadtobe complementedwithothers,e.g.discussingwithpeers.Thiswas clear when Sara knew what to do (which aspects she should improvecomparativelywithpreviousreports),butdidnotknow howtodoit,asdocumentedinthefollowing:

Anotherproblemwasthetitlesofthetables.Weneverknew whatweshouldwrite.(2ndInterview)

Whatwillbethenameofthistable?(3rdTask;3rdLesson) Anotherstrategyusedwasthecriticalinteractionwithherself. Saramadesuggestionstoimprovethecompletenessandaccuracy oftheassignmentsveryoften.Sheusedtoproposeseveralwaysto developtheworkortoexpressherideasuntilshegotonethatshe liked:

Whenweaddicetothelemonadeitwillgiveenergyintheform ofheatforastatechangetohappen,resultinginicemelting. Thisisallthechangeinenthalpy.No,onlyhere,statechange, occursicemelting.Hereweputenthalpy,changeofenthalpy. (3rdTask;2ndLesson)

However Sara rarely adopted a critical attitude about the teacher’ssuggestions,evenwhentheseopposedherstandardsof quality.Inthereportofthefourthtask,awrittencommentwas madesuggestingtheaggregationoftwotables:“Whydon’tyou includethesemeasures,which arealsodirectones,in thetable

above?(youcanimprovetheorganization)”(R4_V1).Inthesecond version,onlyonetablewaspresentedtorecordthedata,butSara consideredthatthisaggregationhadworsenedtheorganizationof thissection:

Sara—[afterreadingthewrittenfeedback]Itisperfectthewayit is!!!(4thTask;1stSession)

Sara—Wethoughtthatitwasbetterthisway,withtwotables. (...)

Researcher—So,youdidn’tagreewiththecomment,butyoudid itbecausetheteacherwantedit!

Sara—Yeah,becausetheteacherwantedit,exactly.Becausewe thoughtwe would be much... it would not be very well organized.Thethingswouldbeontopofeachotherandthenit wouldbemoredifficulttobuildthetable.It wouldn’tbeas perfectasitwaswithtwo[separatetables].(...)Inourviewit couldbeworse,butintheopinionoftheteacheritcouldbe better.(4thInterview)

Furthermore,Sarasoughttocreateafavorableenvironmentfor thedevelopmentofthereport,suggestingdoingsomesectionson anenvironmentthatallowedhertobemorefocused:

Weshouldwriteallthatwearesaying,whatweknow,butit’s complicated.Don’tyouwanttogoaheadtodosomethingeasier [insteadofdoingthetheoreticalframework]?Wearen’tvery concentratedrightnow.(3rdTask;2ndLesson)

4.2.ThecaseofGustavo

Gustavoisastudentwithanaverageperformanceinthesubject ofPhysicsandChemistry.Hisfinalgradewastwelveandelevenout of twenty in years 10 and 11 of the high school curriculum, respectively.Heisinvolvedinseveralextracurricularactivitiesand managestogetsatisfactoryresults.Heisconfidentinhisabilities andrevealscriticalsense.Atthebeginningofthestudy,hehada limitedexperienceincarryingoutinquirytasksanddevelopinglab reportsaswellasassessinghisownassignments:“Iamnotusedto dotheseexperiments(...)Now,Ihavelearnedtodevelopawell donereport,whichIdidn’tknowhowtodo”(1stInterview). 4.2.1.Understandingtheassessmentcriteria

Gustavo had considered the initial discussion about the assessmentcriteriabeneficial,becauseitwerethestudentswho talkedaboutwhatshouldbeaccomplishedinordertodevelopa highqualitywork:_“Ithinkitwasadvantageousbecausewetriedto presentallthestepsthatneededtobedoneso,thisway,itwould beeasiertomakethereportbecausewehadalreadystatedthe stepspreviously”(1stInterview).However,hepointedoutthatthe useofcertainwordsintherubric,suchas“adequate”maynotbe enlighteningif thestudent does not know its meaning in the contextof thetask.Forinstance,what is a suitablediagram to illustratetheprocedure usedtosolvetheproblem:“Onthelast levelitsays:Itincludesanadequatediagram,buthowdoweknow that ... ahh,whatisanadequatediagram?Idonotknow,maybe itcouldbealittlebitmoreexplicit”(3rdInterview).AsGustavodid notunderstandwhatwasexpected,hedidnotincludethedrawing oftheexperimentalsetupinthereportofthefirsttask.

Inthefirstversionofthereportofthesecondtask,itseemsthat hethoughtthatthediagramshouldillustratethecalculationsthey had done to solve the problem instead of representing the experimentalprocedure(Fig.2).

Theexemplarreportseemstohavepromotedthe understand-ingofwhatwasrequired,thushedevelopedasuitabledrawingof theexperimentalsetupinthereportofthefourthtask(Fig.3):“I havedifficultytodothediagram.Butafterseeingthediagramof

my colleagues[exemplar report] Ithought:Ah, that’sit!” (2nd Interview).

Regardingtheassessment criteriadefiningclearandsynthetic objectives,initiallyGustavodidnotunderstandthemeaningofthe wordsynthetic,butheonlyaskedforaclarificationafterthefirst task, during the interview. Meanwhile, he assumed that the objectives shouldbepresentedinaneasilyunderstandableway [sentence1],sothefirstonesincludeddefinitions[sentence2]. However, the clarificationprovided duringthe class discussion (whentheexemplarreportwasanalyzed)seemedtohavehelped himtounderstandwhatwasexpected,andinthefollowingtaskhe proposed to write a short objective (not mixed up with the definitionoftheconcepts)[sentence5]:

1. Whatdidtheteachermeanbysynthetic?Ithoughtthatitmeant clear,inawaythatitiswellunderstood.Synthetic,Ithoughtit was alsosomething related tothat as well: properly under-standable.(1stInterview).

2. Itriedtoputmyselfintheshoesofapersonwhodidn’tknow aboutthisandwhether,ifIreadtheobjectivestoapersonwho didnotknowabouttheAvogadro’slaw,wouldthispersonbe abletounderstandourobjectives.(2ndInterview)

3. StudentC—TheobjectivesIpresentareconsistentwiththetask and are clearand synthetic[reading the higher level of the rubric].

4. StudentD—No,wehavetosaytheapproach,inotherwords,itis moreorlesstheanswertothe_firstquestion.

5. Gustavo—No,whatisaskedhere,isonlytosaytheobjective. Thenhere[referringtotheconceptualframework]weshould saytherest.(3rdTask;1stSession)

Anotherassessmentcriterionthatwasnotinitiallyunderstood by Gustavo was the presentation of evidence to support the conclusions. He thought that the conclusions should be drawn througha plainanalysisof theresults[sentence1below].And, althoughtheexpectationsaboutthiscriterionhadbeendiscussed withhim,itseemsthattheteacherandGustavohadnotreacheda commonunderstandingbecauseinthefollowingreportshedid

Fig.2.Drawingoftheexperimentalsetup(R2_V1).

not make the results explicit on the conclusions; instead he consideredthatthis assessmentcriterion had beensuccessfully operationalized[sentences2and3]:

1.It means that it is about things that we can look and immediatelyrealize... wecanimmediatelydrawaconclusion (...). I thought that it was about being obvious. (2nd Interview)

2.TheconclusionsItakeareexplicitlybasedonthefindings:well,I thinkwedidthat.(3rdInterview)

3.Wetriedtomakeexplicittheresultsthatwehavefound.(4th Interview)

Asimilarsituationhappenedwiththeexplanationofhowthe identifiedsourcesoferroraffecttheresults.Gustavo’sanalysisabout theoperationalizationofthisassessmentcriterionappearedthat, atthe beginning, hehad not completelyunderstood what was expectedinordertoapplyitsuccessfully,butattheendheshowed thatheunderstoodit:

Researcher—Didyouexplaintheeffectofeachsourceoferrorin yourresults?

Gustavo—[afterreadinghisassignment](...)Ithinkwewrote whatthesourcesoferrordid.Ibelieveso.

Researcher—You wrote that the sources of error caused differences, but youdidn’t writethatbecausean amountof waterremainedinthebeaker,thefinaltemperaturewashigher orlowerthanthepredictedresult.

Gustavo—(...) We didn’t write how it did influenced... Exactly.Igetit.Iagree.

Researcher—Does itmean that youhadn’tpreviously under-stoodthispart,whatdoesthismean?

Gustavo—Well,I understood, but I didn’tremember... we didn’tremembertomentionthis,howithasinfluenced.Wejust wrotethatitinfluencedandwhatinfluenced,butnohow... Exactly,thisismissing.(3rdInterview)

4.2.2.Strategiesusedintheoperationalizationoftheassessment criteria

Gustavousuallystartedbyreadingthehigherlevelofqualityof therubricandthenheadoptedseveralstrategiestosupportthe operationalizationof theassessment criteria,suchas discussing ideaswithpeers[sentences1to5]andaskingforhelp[sentence6]to achieveaccuracyand completenessofsomepartsofthereport, especiallythoseinvolvingthedataanalysis.Hesoughtnotonlyto getacorrectanswer,butalsotounderstandthereasoningbehind it:

1.Gustavo—Howdoyoucalculatethemolarmass? (...)

2.StudentD—Nowyoudo:Mequalsfour 3.Gustavo—Wheredidyougetthefour?

4.StudentD—IsawitonthePeriodicTable.Timesone. 5.Gustavo—Howdoyouknowitisaone?(2ndTask;1stLesson) 6.Gustavo—Teacher,Gomesdidit[thepredictionoftheresults] this way. ButI don’t understandany of this. (3rd Task; 1st Lesson)

AnotherstrategyusedbyGustavowaslearningfromerrors.For instance,theintroductionofthereportsbecamemorecomplete over time (containing the definitions of the most important physicalandchemicalquantities):

Gustavo—Ithinkapersonalsolearnsfrommistakes,isn’tit? (...)

Researcher—Givemeanexample.

Gustavo—Anexample?Here,intheintroduction,forinstance, whenyouasktoexplainwhatdensityis.ThenexttimethatIuse

densityIwillhavetomentionitintheintroduction,givinga detailedexplanation.

Researcher—But,what ifthereportis notaboutthedensity, whatifitisaboutanothermatter?

Gustavo—IfitisaboutanothersubjectIwillalso(...)Igave theexampleofthedensity.(1stInterview)

Concerning the scientific rigor, it seems that Gustavo also learned from errors because some mistakes made in the first reports(towhichfeedbackhadbeenprovided)werenotrepeated insubsequentones.Forexample,inthefirstversionofthereportof thefirsttask,Gustavowrote:“Thenwedividethevalueofthemass by 10, in order to get the result in g/cm3” (R1_V1), without mentioning the physical quantity that he was calculating. Considering the written comment: “The result of what? What physicalquantityareyoucalculating?"(R1_V1),headded,inthe secondversion, thephysicalquantitythat wasbeingcalculated (thedensity).Furthermore,inbothversionsofthereportofthe secondtask,Gustavocontinuedtoindicatethephysicalquantity insteadof itsunit:’tocalculate themolarmass’,“calculate the molar volume” (R2_V1), “calculate the number of particles” (R2_V2).

Theorganizationofthetablesimprovedaswell.Afterawritten commentbeingmadeasfeedbacktothefirstversionofthereport ofthefirsttaskabouttheunitsthatwereregisteredineachrow (“The advantage of indicating the units here (in the column heading)isnothavingtorepeatit”),Gustavobegantoshowthe unitsonlyinthetitleofthecolumnsinthefollowingreports.

Gustavoalsoappealedtoacriticalinteractionwithhimselfwhile hewasdevelopingthereports.Occasionally,hemadesuggestions andchangesinordertoimprovethescientificrigor,thewritingand theanalyticalaccuracy:

Thisistheweightoftheballoon.Theweightoftheballoonis equalto96point753(...)Wait,whatweight?!Thisisthe mass,nottheweight.(2ndTask;1stLesson)

BecauseIstartedtodoallthemaths,butthenumbersofthe resultswereabitlarge.So,[Ithought]whatisgoingonhere?It isimpossibletohaveaslidemeasuring100m.IsawthatIhad forgottentoputin thesquareonthevelocity inthe kinetic energyexpression.(...)Iwaswrongmanytimes.(4thTask; 3rdLesson)

Gustavoalsousedhisreasoningtodefinesomewordsinthe conceptualframework(anddidnotsearchforinformation):“Iused myownwords_” (...)_“Basedonthename,Itriedto..._” (1st Interview).

Besides,inGustavoperformancesometimesprevailedthe self-imposedstandards,whichmeansthat,despiteunderstandingthe assessmentcriteria,Gustavodecidednottotakeintoaccountsome ofthembecauseitmademoresensetohimtodotheassignmentin a different way. This happened, for instance, regarding the presentation separately of the data collection and the data analysis,aswellas,thesynthesisoftheobjectives:

Ontheonehand,itiswelldone,butontheother,itisnotwell organized,perhaps because allthe collecteddatashould be together.Buthere,asweareshowingsomedataandthendoing thecalculation ... Ifwehadonlypresentedthecalculations [separately],peoplecouldthink:Okay,thisishere,butwhere didthisdata comefrom?(...).Ithink it’s betterthis way, becauseitissequentiallydone... butitisnotsoscientifically correct.(3rdInterview)

Researcher—Synthetic means that it is somethingshort and direct.

Gustavo—Exactly.

Researcher—For instance, regarding the horizontal displace-mentwouldn’tyouhavetheopportunitytoexplain,lateron,in

theconceptualframework,whatisitandhowyoucalculateit? Did the definition of horizontal displacement have to be includedinyourobjective?

Gustavo—Itdidn’thaveto,butwethoughtitwouldbebetterifit wastherebecause,forinstance,ifthepersonwhoisgoingto read,doesn’tknowwhatitis(...)he/shekeepsreadingbutis stillthinkingaboutwhatitis.However,ifweincludeitright away,thispersonwillknowimmediatelywhatitis.(...)Inmy opinion,itismorecorrecttowritethedefinitionimmediately aftertheword.(4thInterview)

Inadditiontothepreviousstrategies,Gustavotriedtocreatea favorable environment to the development of the report. He suggestedtodo or torevisesomeparts of it,theones thathe consideredmoredifficultorthoughtthatwerenotaswelldoneas hedesired,inaquieterenvironmentandbytakingmoretime:

Thestrategywehavechosenwastocalculatetheenergygiven bythelemonade,asthisisequaltotheenergyabsorbedbythe cold wateror the ice,isn’t it?I don’tthink this is properly explained.Ihavetoseethisathome.Youhavetosendmethis, Duarte.(...)NowIdon’tknowwhatelsetowrite.(3rdTask; 1stLesson)

5.Discussion

Inthetwocasestudy,studentsdidnotinitiallyunderstandthe totality of the assessment criteria, apparently because of the terminologythatwasusedintherubricand/oritshighcognitive demand. Although an initial discussion about the assessment criteriahappened intheclassroom,therubricdescriptorswere notwrittencollaborativelywiththestudents,andthe terminol-ogyusedwasnotalwaysunderstoodbythem(e.g.presentationof evidencetosupporttheconclusions).Theresultsshowthatboth studentswerenot sureaboutthemeaningofsomewords(e.g. synthetic), but they usually did not ask for clarifications. Moreover,inspiteofthefeedbackthathadbeengivenregarding some criteria, Sara did not succeed (although she had shown intentiontoconsiderit).Theabsenceofacommonunderstanding aboutwhatwasexpectedseemstobethereasonforthat.So,this study,likethatofClark(2012),suggeststhatoneofthebiggest difficulties in order to reach a common understanding is translating assessment criteria into a language that students canunderstand.

Forbothstudents,theanalysisoftheexemplarreportseemed to have a crucial role in the process of reaching a common understanding, as previously suggested by Andrade, Du, and

Mycek (2010), as well as Hendry et al. (2012). Seeing the

operationalization of those criteria seems to contribute to the studentsunderstandingoftheteacher’sexpectations.

Inordertooperationalizetheassessmentcriteria,studentsused some strategies:discussing with peers and searching for help, collecting information from available sources, learning from mistakes, critical interaction with themselves and creating a favorable environment, although some differences were found. Sara, whois insecure, was not usuallyeasilysatisfiedwithher suggestions, so she frequently revised them. Furthermore, she oftenaskedteachersandherpeersforhelpinordertovalidateher ideas,andshesearchedforinformation.Wantingtoachievehigh marks but seeming to be afraid of taking risks (to fail), Sara sometimestranscribedtheinformationwith“guaranteed”quality. Gustavo,whoismoreconfidentandnotascommittedtoachieve highmarksasSara,rarelysearchedforinformationand instead usedhisreasoningtoaccomplishtheassignments.

Another importantdifference isthat theprimaryconcernof Saraseemstobetoreachtheteacher’sexpectations,evenifshe

does not completely understand or agree with the teacher’s suggestions. However, Gustavo appears to be concerned with understanding the reasoning behind the criteria. His attitude showsthathisworkmustmakesensetohim.Thisseemstojustify, the tensions between the self-imposed standards and the assessment criteria that sometimes emerged throughout the study, as previously found by Andrade and Du (2007), Santos

and Gomes (2006), and Semana and Santos (2010). Gustavo,

seemedtoneedtoidentifya clearadvantagein developingthe assignments in a way that opposed his own quality criteria, otherwisehewouldconsiderhisself-imposedstandards(giving argumentsfor this procedure).However, Saraseemedtotryto understandthe “rulesof thegame” (Hadji,1989)and toact in accordancewiththem.Thisisanaspectthatseveralauthors(e.g.

Nicol & Macfarlane-Dick, 2006) have referred to—assessment criteria must be consideredimportant bystudents in order to prevent them from acting in a certain way just to please the teacher.So,thetensionbetweentheself-imposedstandardsand theassessmentcriteriaandthewaythistensiondecreasesreflects a complex process of internalization of assessment criteria (Andrade&Du,2007).

In the light of the above, it seems that the way students operationalize theassessment criteria is morerelated withthe students profile than with their average performance in the subject. Furthermore, there was a progress regarding the operationalization ofmost ofthe assessment criteria.However, somedifficultiesremainedaboutsomeofthem,whichsuggests that certainassessmentcriteriaaremore difficulttointernalize thanothers.

Regardingtheroleof thepedagogicalapproachusedforthe internalizationof theassessmentcriteria,theuseof therubric wasimportanttoguidethestudents,sincetheyusuallystarted by reading the highest-level descriptor of each section when theyweredoingthereport.Theuseofformerreports(withthe written feedback) wasconducive of learningfrom mistakes as previouslyreported(Santos&Gomes,2006).Thisstudy,aswell asthatofAndrade,Du,andMycek(2010),alsosuggeststhatthe analysis oftheexemplar report helpedstudents tounderstand what wasrequired from them.But one of the students (Sara) used the exemplar,as well as theformer reports, tocopy the layoutandterminology(Hendryetal.,2012).Thefact thatthis disadvantage was only evident in one of the case studies suggests that a mechanistic approach is more likely to arise whenstudentsareafraidofmakingmistakesandthereforelook for models to copy.

Theinteractionbetweenstudentshelpedmainlytheovercome of difficulties in operationalizing theassessment criteria. Feed-back,besidesthat,alsofavouredtheidentificationofmistakesand omissions and the production of an improved version of the originalassignment.

6.Concludingremarks

This study emphasises that understanding the assessment criteriaisacomplexprocessmainlybecausetheterminologyand thelogicbehindthesubjectandtheteacherareusuallydifferent from the students (Nunziati, 1990; Vial, 2012). In order to operationalize the assessment criteria, students used strategies stemming from the social context (teacher, peers and other didacticsources)aswellasfromtheirindividualexperience(errors made, engagementin critical thinking and creatinga favorable environment). Theways andfrequency of howthese strategies were used were variable and related to the students’ profile, evidencing the relation between different components of self-regulation—cognitive,affectiveandsocial.

Regarding the role of the pedagogical approach in the internalization of the assessment criteria, it seems that the exemplar report was crucial for the understanding of the assessmentcriteriathathadnotbeenpreviouslyunderstood(in theinitialdiscussion).Butallteachers’interventions(discussing the assessment criteria, using the rubric, giving feedback, discussing an exemplar report and promoting dialog between peers)seemtohaveplayedaroleintheoperationalizationofthe assessment criteria (guiding the students, helping them to overcome difficulties and to learn from mistakes). However, notwithstandingthepedagogicalapproachbeenusedtopromote theinternalizationoftheassessmentcriteria,thisprocessproved tobecomplex (Andrade&Du,2007), requiringcontinuityover time,evenforhighschoolstudents(withextensiveexperiencein theschool).

It should be highlighted that in this study researchers managed to guarantee the application of essential conditions topromotelearningthroughassessment:anenvironmentwhere students felt that mistakes are learning opportunities, using complexcognitivetasksandtransparencyofassessmentcriteria(

Segers, Dochy, & Cascallar, 2003). Researchers intentionally soughttopromotefavorableconditionsforthedevelopmentof self-regulation(Wiliam,2011),stimulatinghigher-levelcognitive learning.

One of the limitations of this study was that, despite the awarenessthattheparticipationonthisstudywouldnotinfluence theirgrades,studentsmayhavenotsharedalloftheirperceptions, emotionsandopinionsbecausetheresearcher(andinterviewer) hadbeentheirteacheronthefirstyearofcollectingdata.However, weconsiderthatthisdoublefunction(teachingandresearching) didnothaverepercussionsforthevalidityandreliabilityofthe findingsbecausetheinteractionsbetweentheteacher/researcher andstudentswerebasedonconfidence.Westronglybelievethat studentsfeltcomfortablesharingtheirfeelings.

Someoftheterminologyusedintheassessmentcriteriawas difficult to understand, and even though the teacher and the studentsseemedtohavereachedacommonunderstanding,the progressregardingtheoperationalizationofthosecriteriahadnot always been consistent over time. Therefore, we suggest that furtherresearchshouldinclude analysisof thepotentialitiesof writingtherubric’sdescriptorcollaborativelywiththestudentsor asking them to: i) re-write the descriptors (using a familiar terminology);ii)takesomenotesafterthediscussionaboutthem or; iii) underline the place where each assessment criteria is appliedintheexemplarreportandhighlightthecharacteristics thatshowthequalityofitsoperationalization.

Finally, as the methodology used in this study does not enablestatisticalgeneralization oftheresults, furtherresearch

should extendthisworktostudentsof other yearsand levels, with different characteristics, and include additional tools of learning/assessment, as well as additional subjects. Nonethe-less, the in-depth analysis of the two cases contributed to deepen the knowledge on students’ internalization of assess-ment criteria.

Acknowledgments

ThisresearchwassupportedbyagrantfromFCT,Portuguese FoundationforScienceandTechnology(SFRH/BD/36083/2007).

The authors are grateful to Elisabete Carmo-Silva, Juliana Carvalhoandtoanonymousreviewers,whocommentedonearlier draftsofthispaper.

AppendixA.—Students’GuidelinesfortheThirdTask

Summeriscomingandafreshlemonadewouldbegreat!Have you ever thought aboutwhat is more efficient for cooling the lemonade:liquidwaterat0_Coriceat0_C?

To get an answer to this question you have water at room temperatureinsteadoflemonadeandalsowaterandiceatabout 0C.

1.Which processes (adding water at 0C or ice at the same temperature) will be more efficient for cooling lemonade? Justifyyourprediction.

2.Plananexperimentalworkinordertoanswerthisproblem. 3.Predictwhatwillbetheequilibriumtemperatureofwaterafter

addingsomemassofwateratabout0_{C.Justifyyourprediction.}

4.Predictwhatwillbetheequilibriumtemperatureofwaterafter addingsomemassoficeatabout0C.Justifyyourprediction. 5.Performtheexperimentalworkthatwasplannedanddiscussed withtheteacher.Note:Youmustbreaktheiceandputitina glasswithwaterinorderforthetemperatureoftheinnericeto homogenizeat0C.

6.Comparetheresultswiththeinitialpredictionsanddiscussit. 7.Whatchangesdoyousuggesttocoolthewatermoreefficiently? 8.Howwouldyouanswernowtothequestionthatguidedyour

work?

AppendixB. SeeTableB1.

References

Allal,L.(2007).Régulationsdesapprentissages:Orientationsconceptuellespourla recherchéetlapratiqueenéduction.[Regulationoflearning:Conceptual orientationsforresearchandpracticesineducation].InL.Allal,&L.M.Lopez (Eds.),Re´gulationdesapprentissagesensituationscolaireetenformation [Regulationoflearninginschoolandtrainingsituation](pp.7–23).Brussells:De Boeck&Larciers.a..

Andrade,H.,&Du,Y.(2007).Studentresponsestocriteria-referenced self-assessment.Assessment&.EvaluationinHigherEducation,32(2),159–181. http://dx.doi.org/10.1080/02602930600801928.

Andrade,H.,Du,Y.,&Mycek,K.(2010).Rubric-referencedself-assessmentand middleschoolstudents’writing.AssessmentinEducation,17(2),199–214.http:// dx.doi.org/10.1080/09695941003696172.

Andrade,H.(2000).Usingrubricstopromotethinkingandlearning.Educational Leadership,57(5),13–18.

Andrade,H.(2001).Theeffectsofinstructionalrubricsonlearningtowrite.Current IssuesinEducation4(4).http://cie.ed.asu.edu/volume4/number4.

Bell,R.,Smetana,L.,&Binns,I.(2005).Simplifyinginquiryinstruction:Assessingthe inquirylevelofclassroomactivities.ScienceTeacher,72(7),30–33.

Black,P.,&Wiliam,D.(1998).Assessmentandclassroomlearning.Assessmentin Education:Principles,Policy&Practice,5(1),7–74.http://dx.doi.org/10.1080/ 0969595980050102.

TableB1

RubricforInquiryReports.

Section Level 1 2 3 4

Items

Introduction Objectives TheobjectivesIpresent areinconsistentwiththe task

TheobjectivesIpresentare consistentwiththetaskbut arenotclear

TheobjectivesIpresentare consistentwiththetaskandare clear

TheobjectivesIpresentareconsistent withthetaskandareclearandsynthetic Scientific

concepts relatedtothe experiment

Irefertothescientific conceptsrelatedtothe experiment,butdon’t explainthem

Irefertothescientific conceptsrelatedtothe experimentandexplain thempartiallyorgive shallowexplanations

Ideeplyexplainallthescientific conceptsrelatedtothe experiment

Ideeplyexplainallthescientificconcepts relatedtotheexperimentandjustifytheir importancetothetask

Planning Strategy ThestrategyIchooseis notadequatetothe objectives

ThestrategyIchooseis adequatetotheobjectives IpresentthestrategyI

choosewithfewdetails

IpresentthestrategyI choosewithadetailed description

IpresentthestrategyIchoose withadetaileddescriptionand explainmychoice.Igive justificationforeachstepofthe task.

IpresentthestrategyIchoosewitha detaileddescriptionandexplainmychoice andthereasonsIgiveupothers.Igive justificationforeachstepofthetask.

Expected results

Ioutlinetheexpected results,butdonotprovide logicalreasoningforit

Ioutlinetheexpectedresults andprovidelogical reasoningforit Procedure Task

development

Idescribeonlyafewsteps ofthetask

Idescribemostofthesteps ofthetask

Idescribeallthestepsofthetask Drawingof

the experimental setup

Iillustratetheprocedure butthediagramisnot completelyadequate

Iillustratetheprocedure withanadequatediagram

Iillustratetheprocedurewithan adequatediagram,correctly identifiedwithtitleandlabels

Results Data collection

Ipresentsomecollected data(measurements/ observations)

Ipresentmostofthe collecteddata (measurements/ observations)

Ipresentallthecollecteddata (measurements/observations)

Dataisreportedwith frequentinaccuracies

Dataisreportedwithfew inaccuracies

Dataisaccuratelyreported Dataanalysis Ianalyzesomeofthe

neededdata

Ianalyzemostoftheneeded data

Ianalyzealltheneededdata Dataanalysishasfrequent

inaccuracies

Dataanalysishasfew inaccuracies

Dataanalysisisaccurate Conclusions Conclusions TheconclusionsItakeare

notconsistentwiththe results

TheconclusionsItakeare consistentwiththeresults, buttherearenoreferencesto theappropriatefindings

TheconclusionsItakeare consistentwiththeresultsand theyareexplicitlybasedonthe findings

Limitations Icomparethefindings withtheexpectedresults andsuggestpossible errors,notindicatingtheir source

Icomparethefindingswith theexpectedresults, presentingerrorsand indicatingsomeoftheir probablesources

Icomparethefindingswiththe expectedresults,presenting errors,indicatingtheirprobable sourcesandtheireffectinthe results

Icomparethefindingswiththeexpected results,presentingerrors,indicatingtheir probablesourcesandtheireffectinthe results.Isuggestalternativeproceduresto minimizethem

Learning outcomes

Idescribeincompletely whatIhavelearned

IexplainsyntheticallywhatI havelearned

IexplainwhatIhavelearned withdetail

Eachsectionofthereport(introduction,planning,procedure,results,conclusions)shallbeanalyzedconcerningtheorganization,grammar,spellingandscientificstyle:

Category 1 2 3

Organization Ipresenttexts,calculations,tables,diagramsand graphicsinanunorganizedway

Ipresenttexts,calculations,tables,diagramsand graphicsinareasonablyorganizedway

Ipresenttexts,calculations,tables,diagrams andgraphicsinaveryorganizedway Grammarand

spelling

Therearefrequentgrammarandspellingmistakes Therearefewgrammarandspellingmistakes Grammarandspellingarecorrect Scientificstyle Scientificterminologyhassomeinaccuracies Scientificterminologyhasfewinaccuracies Scientificterminologyisaccurate

Itshallbedoneaholisticanalysisofthereportconcerningthestructureandthepresentation.

Category 1 2 3

Structureand presentation

Thereportdoesnotfollowthe proposedstructure

Thereportfollowstheproposedstructure,butdifferent sectionsarenoteasilydistinguished

Thereportfollowstheproposedstructureanddifferent sectionsareeasilydistinguished

Black,P.,Harrison,C.,Lee,C.,Marshall,B.,&Wiliam,D.(2002).Workinginsidethe blackbox:Assessmentforlearningintheclassroom,1sted.London:nferNelson.

Black,P.,Harrison,C.,Lee,C.,Marshall,B.,&Wiliam,D.(2003).Assessmentfor learning:Puttingitintopractice.London:OpenUniversityPress.

Boekaerts,M.,Maes,S.,&Karoly,P.(2005).Self-regulationacrossdomainsof appliedpsychology:Isthereanemergingconsensus?AppliedPsychology,54(2), 149–154.http://dx.doi.org/10.1111/j.1464-0597.2005.00201.x.

Brookhart,S.M.(2013).Howtocreateanduserubricsforformativeassessmentand grading.Alexandria,VA:AssociationforSupervisionandCurriculum Development(ASCD).

Burns,R.B.(2000).Introductiontoresearch:Methods,4thed.London:Sage Publications.

Clark,I.(2012).Formativeassessment:Asystematicandartisticprocessof instructionforsupportingschoolandlifelonglearning.CanadianJournalof Education,35(2),24–40.

Cohen,L.,Manion,L.,&Morrison,K.(2000).Researchmethodsineducation,5thed. LondonandNewYork:Routledge/Falmer.

Crisp,G.(2012).Integrativeassessment:Reframingassessmentpracticeforcurrent andfuturelearning.Assessment&EvaluationinHigherEducation,37(1),33–43. http://dx.doi.org/10.1080/02602938.2010.4942349.

Dann,R.(2002).Promotingassessmentaslearning:Improvingthelearningprocess,1st ed.NewYork:RoutledgeFalmer.

Gipps,C.(1999).Socio-culturalaspectsofassessment.ReviewofResearchin Education,24,355–392.http://dx.doi.org/10.3102/0091732X024001355.

Green,R.,&Bowser,M.(2006).Observationsfromthefield:Sharingaliterature reviewrubric.JournalofLibraryAdministration,45(1–2),185–202.http://dx.doi. org/10.1300/J111v45n01_10.

Griffin,M.(2009).Whatisarubric?AssessmentUpdate,21(6),4–13.

Hadji,C.(1989).L’evaluation,reglesdujeu:Desintentionsauxoutils.Paris:ESF.

Hafner,J.,&Hafner,P.(2003).Quantitativeanalysisoftherubricasanassessment tool:Anempiricalstudyofstudentpeer-grouprating.InternationalJournalof ScienceEducation,25(12),1509–1528.http://dx.doi.org/10.1080/

0950069022000038268.

Hendry,G.,Armstrong,S.,&Bromberger,N.(2012).Implementingstandards-based assessmenteffectively:incorporatingdiscussionofexemplarsintoclassroom teaching.Assessment&.EvaluationinHigherEducation,37(2),149–161.http:// dx.doi.org/10.1080/02602938.2010.515014.

Kirby,N.,&Downs,C.(2007).Self-assessmentandthedisadvantagedstudent: Potentialforencouragingself-regulatedlearning?Assessment&Evaluationin HigherEducation,32(4),475–494.http://dx.doi.org/10.1080/

02602930600896464.

Lichtman,M.(2006).Qualitativeresearchineducation:Auser’sguide.Thousand Oaks:SagePublications.

Merriam,S.B.(1988).Casestudyresearchineducation:Aqualitativeapproach,1sted. S.Francisco:JosseyBass.

NationalResearchCouncil(1996).Nationalscienceeducationstandards.Washington, D.C:NationalAcademyPress.

Nicol,D.J.,&Macfarlane-Dick,D.(2006).Formativeassessmentandself-regulated learning:Amodelandsevenprinciplesofgoodfeedbackpractice.Studiesin HigherEducation,31(2),199–218.http://dx.doi.org/10.1080/

03075070600572090.

Nicol,D.(2010).Frommonologuetodialogue:improvingwrittenfeedback processesinmasshighereducation.Assessment&EvaluationinHigher Education,35(5),501–517.http://dx.doi.org/10.1080/02602931003786559.

Norton,L.(2004).Usingassessmentcriteriaaslearningcriteria:Acasestudyin psychology.Assessment&EvaluationinHigherEducation,29(6),687–702.http:// dx.doi.org/10.1080/0260293042000227236.

Nunziati,G.(1990).Pourconstruireundispositifd’évaluationformatrice.Cahiers Pe´dagogiques,280,47–64.

Papaleoutiou-Louca,E.(2003).Theconceptandinstructionofmetacognition. TeacherDevelopment,7(1),9–30.http://dx.doi.org/10.1080/

13664530300200184.

Price,M.,Handley,K.,Millar,J.,&denOuter,B.(2007).Engagingstudentswith assessmentfeedback.ManuscriptoncasestudiesconductedfortheFDTL5project. Oxford:OxfordBrookesUniversity.

Rust,C.,Price,M.,&O’Donovan,B.(2003).Improvingstudents’learningby developingtheirunderstandingofassessmentcriteriaandprocess.Assessment &EvaluationinHigherEducation,28(2),147–164.http://dx.doi.org/10.1080/ 0260293032000045509.

Santos,L.,&Gomes,A.(2006).Self-assessmentandappropriationofassessment criteria.Proceedings30thpsychologymathematicseducation(PME),mathematics inthecenter(pp.49–56)..

Santos,L.,&Pinto,J.(2009).Lightsandshadowsoffeedbackinmathematicslearning. Thessaloniki,Greece,PME33,InternationalGroupforthePsychologyof MathematicsEducation,Vol.5,49–56.

Santos,L.,&Semana,S.(2015).Developingmathematicswrittencommunication throughexpositorywritingsupportedbyassessmentstrategies.Educational StudiesinMathematics,88(1),65–87. http://dx.doi.org/10.1007/s10649-014-9557-z.

Segers,M.,Dochy,F.,&Cascallar,E.(2003).Theeraofassessmentengineering.InM. Segers,F.Dochy,&E.Cascallar(Eds.),Optimisingnewmodesofassessment:In searchofqualitiesandstandards(pp.1–12).Dordrecht:KluwerAcademic Publishers.http://dx.doi.org/10.1007/0-306-48125-1_1.

Semana,S.,&Santos,L.(2010).Self-assessmentinwrittenmanuscripts.PME34, InternationalGroupforthePsychologyofMathematicsEducation,Vol.4,Brasil: BeloHorizonte169–176.

Strauss,A.,&Corbin,J.(1998).Basicofqualitativeresearch.Techniquesand proceduresfordevelopinggroundedtheory.ThousandOaks,CA:Sage Publications.

Swaffield,S.(2011).Gettingtotheheartofauthenticassessmentforlearning AssessmentinEducation:Principles.Policy&Practice,18(4),433–449.http://dx. doi.org/10.1080/0969594X.;1;2011.582838.

Taras,M.(2001).Theuseoftutorfeedbackandstudentself-assessmentin summativeassessmenttasks:Towardstransparencyforstudentsandfortutors. Assessment&EvaluationinHigherEducation,26(6),605–614.http://dx.doi.org/ 10.1080/02602930120093922.

Taras,M.(2003).Tofeedbackornottofeedbackinstudentself-assessment. Assessment&EvaluationinHigherEducation,28(5),549–565.http://dx.doi.org/ 10.1080/02602930301678.

Tierney,R.(2013).Fairnessinclassroomassessment.InJ.McMillan(Ed.),SAGE handbookofresearchonclassroomassessment(pp.125–144).London:SAGE Publications,Inc.

Tillema,H.(2014).Studentinvolvementinassessmentoftheirlearning.InC. Wyatt-Smith,V.Klenowski,&P.Colbert(Eds.),Designingassessmentforqualitylearning (pp.39–53).NewYork:Springer.http://dx.doi.org/10.1007/978-94-007-5902-2.

Vial,M.(2012).Serepe´rerdanslesmode`lesdel’e´valuation:histoire,me´thods,outils. Bruxelles:DeBoeck.

Vygotsky,L.S.(1934).Mindinsociety:Thedevelopmentofhigherpsychological processes.Cambridge,MA:HarvardUniversityPress[1934/1978].

Wharton,S.(2003).Definingappropriatecriteriafortheassessmentofmaster's levelTESOLassignments.Assessment&EvaluationinHigherEducation,28(6), 649–664.http://dx.doi.org/10.1080/0260293032000130261.

Wiliam,D.(2011).Embeddedformativeassessment.Bloomington,IN:SolutionTree Press.

Woolf,H.(2004).Assessmentcriteria:Reflectionsoncurrentpractices.Assessment &EvaluationinHigherEducation,29(4),479–493.http://dx.doi.org/10.1080/ 02602930310001689046.

Yin,R.(2002).Casestudyresearch:Designandmethods,3thed.NewburyPark:Sage Publications.

Yore,L.,Bisanz,G.,&Hand,B.(2003).Examiningtheliteracycomponentofscience literacy:25yearsoflanguageartsandscienceresearch.InternationalJournalof ScienceEducation,25(6),689–725.http://dx.doi.org/10.1080/

0950069032000076661.

Zimmerman,B.J.(2002).Becomingaself-regulatedlearner:Anoverview.Theory IntoPractice,41(2),64–70.http://dx.doi.org/10.1207/s15430421tip4102_2.