Unidade de Investigação para o Desenvolvimento do Interior (UDI) - PEst-OE/EGE/UI4056/2011 – Projeto financiado pela
Fundação para a Ciência e Tecnologia (FCT) Instituto Politécnico da Guarda (PORTUGAL)
mdc1792@gmail.com
Going back nearly two decades, this study traces the design and application of an EFL diagnostic test in its various roles and contexts, objectives and results. The contribution to instructional guidance and explicit lesson planning based on the test results is explored as well as the tenets behind effective implementation of initial EFL testing. Attention will be given to learner awareness and work designed for enhanced empowerment of the student role within the learning experience even at the diagnostic level. The results of this study are further compared with current national practices in higher education in Portugal.
Keywords: Testing, Higher Education, Polytechnic Institute of Guarda (IPG), CILC – Interactive Centre for Language and Cultures
* Please cite article as Arau Ribeiro, María del Carmen (2014). EFL Diagnostic Tests: A case study of design and application. In María del Carmen Arau Ribeiro and Isabel Chumbo (editors), Communication without Borders - Selected Papers of the International Conference Languages 2011: X Meeting of Aprolínguas and II Meeting of ReCLes.pt, pp. 56-59. Guarda: Instituto Politécnico da Guarda.
At the turn of the century, the Polytechnic Institute of Guarda (IPG – Instituto Politécnico da
Guarda) had larger classes than it does now, which implied more student anonymity, and
there was a strong need to satisfy the particular objective of improving teacher knowledge of student levels. This was the basic motivation for creating an EFL Diagnostic Test for use in existing EFL and ESP classes at the IPG.
In terms of background and design, it is a structurally-based test that focused more on grammar and lexis, typical of the late 60s and 70s, much like the tests used at ACHNA (Asociación Cultural Hispano Norte-Americana), in Madrid in the academic year 1991-92, and at ALI (the American Language Institute), in Oporto in 1992-92, based on John and Liz Soars’ Headway (Oxford), “probably the most popular course ever written” (Case 2005). The test itself, however, is designed with language knowledge of the Portuguese learner in mind.
In terms of format and scoring, the test uses multiple choice items and gap-filling. For the multiple-choice, the fourth option is always d) I don’t know. This discrete item recognition is complemented by a composition. The test is two pages (front and back), with 18 questions on each side with increasing language complexity. Extra space is left in the margins for scoring and comments, which are divided into two scores, one on each of the pages. Over a decade of use has proven that the test is quick to both administer and score.
Testing criteria involved extensive trials over the test development with students who could not be distributed in level courses. It has proven reasonable predictive validity, as witnessed informally by teacher and student concordance with the levels attributed.
Communication without Borders
Atas do Congresso Internacional Línguas 2011 – X Encontro da Aprolíngu@s e II Encontro da ReCLes.pt 57 Critical aspects of the test in question are i) it tests student knowledge of a range of individual items of structure and lexis in “everyday educated English” (Chaplen 1970: 174, in Morley 2000); ii) it covers a limited language domain; iii) it does not constitute a test of language production or even a test of language skills; iv) it is a finely tuned but narrow assessment of language competence in choice recognition for lexicon and grammar; and v) there is limited test exposure (it is not available to the public).
In the life of the test for students and teachers who are students in language training, it was also used in the 1st teacher development course in English in academic year 2003-04, when teachers at the IPG School of Technology & Management enrolled in the first year-long weekly extracurricular language course in English for teachers (now defined as CEFR level A2).
Contextually, the Language Lab(oratory) at the IPG School of Technology & Management was moved into new facilities within the school and was renovated as the new and improved – and significantly expanded – CILC (Centro Interativo de Línguas e Culturas or Interactive Centre for Languages and Cultures). Pronounced “silk”, the CILC rapidly took on a more prominent role supported by its more central location in the school; in effect, it had gone from dungeon (literally accessed via a back door in a dark area under the stairs!) to dragon, serving as a computer lab as well, combining a rich Multimedia Library with regular teacher availability in addition to a highly qualified full-time technician who was particularly talented at human relations. In the initial year, teacher office hours were even held at the CILC to create more traffic and better habits of actually using the CILC.
As if intentional, Bologna came along and there were suddenly many more English courses to be taught and students who needed support for learning languages. Both the Schools of Education (now ESECD, including Communication and Sports) and Technology & Management offered English assessment for the general community and teachers, students and staff for extracurricular A1 and A2 level trial courses in English. There were also more Language for Specific-Purposes (LSP) courses at all four school, including the School of Health and the School of Tourism & Hospitality Management. These additions were outshined by the proliferation of Professional courses (Levels III and IV) that required at least Technical English and others that further required Technical Spanish.
More recently, the IPG has promoted a linguistic policy making great strides in internationalization and encouraging mobility through the availability of in-house English language courses at three CEFR levels which are being offered for teachers and other staff. Three distinct levels based on number of errors on the test have been confirmed: A1; A2/B1; and B2/C1. Extracurricular courses are being created for specific technical and scientific areas, beginning with the current course of English for Computer Engineering students (2011).
Some of the incidental benefits of assessment are i) student response offers a wealth of information as pedagogical significance of common misunderstanding OR weakness in the question; ii) although discrete point testing is not task-based, the composition yields a more subjective account of language proficiency; and iii) to explore beyond the formal system (test) and reach other language dimensions – including the nature of meaning, the relationship between sentence and context, variation evident from user to user, and communicative and textual, illocutionary, and sociolinguistic competence (Alderson et al. 1995: 225). Furthermore, assessment reveals not just linguistic structures and lexis but also valuable information on learner use of functions/notions, strategies for learning and communication, interculturality and language awareness. For all of the above, assessment can serve as an indicator; it need not be the face of the institution.
Communication without Borders
Atas do Congresso Internacional Línguas 2011 – X Encontro da Aprolíngu@s e II Encontro da ReCLes.pt 58 In a study of eight ReCLes.pt member institutions, language centres report using diagnostic tests ranging from Macmillan online and the Oxford Placement Test (OPT) electronic & paper to DIALANG – hosted by Lancaster University, available for 14 languages and directed at higher education institutions (HEIs). Most also report effectively using interviews to carry out further assessment for language levels and placement.
By way of conclusion, experience has proven that this one simple testing tool with diverse objectives can have a long shelf life. In effect, selected-answer testing is practical and efficient. It can also be made more flexible, challenging and effective by using it to stimulate classroom activities and empower learners. By using the classroom as a laboratory for both teachers and students to actually learn, so much can be learned by students and teachers alike. Finally, test, test, test. Items for tests must be tested themselves yet it is a fact that “so many boards do not test their items” (see Alderson et al. 1995: 102).
This gem, entitled Necessity Is the Mother of Invention by Stephany Aulenback, literally illustrates through literature the truth of its title; that, when your school needs a tool, you are the right person to make the best tool possible given the parameters required by your local situation. Enjoy!
Necessity Is the Mother of Invention Stephany Aulenback
Necessity is the mother of Invention, and, as everybody knows, a skinny woman named Poverty is the mother of Necessity. Necessity is gritted teeth and a rusty helmet of hair. Invention is a real shock — she’s beautiful, sleek and lively, eyes the color of sky. Necessity can hardly believe this girl belongs to her. Sometimes it seems Invention doubts it, too. She’s forever looking beyond Necessity, over her shoulder, for someone else more plausible. Necessity doesn’t want her daughter to leave her, though she knows that’s what daughters often do.
Invention was planted in Necessity’s belly by Inspiration, one of a significant number of wandering men who, years ago, came through the town on the trains...
Whatever Necessity has to do, she does. So she married Duty, the son of the stationmaster, Stipulation. Eight months later, Invention was born with a caul covering the two tiny wings on the back of her neck. While the other children learned their ABCs, Invention was learning to fly. After work, in the evenings, people gathered in the field to watch her move across the sunset. No one was prouder than Duty.
Now and then Invention pretended to fall — she came spiraling down out of the sky like a twirling maple key — and Duty ran to catch her. She kissed the tip of his nose and everyone smiled.
Stipulation, who had a bad squint, remained suspicious… Stipulation told Necessity that her daughter was unseemly, that she put on airs and shouldn’t be permitted to show off. His opinions of Invention got the better of him once and he came at her with a pair of rusty scissors, threatening to cut off her wings.
But that’s all in the past, and now Invention is grown. Young men try to woo her. Her childhood pal Imagination claims to be in love. He’s attractive, he’s charming, and Invention can’t believe a word he says. Patent is the most persistent. Every time he sees her, he makes a grab for her hand, trying to push a promise ring onto her finger. Invention, thinking fast, introduces him to her cousin Novelty. Novelty is pretty and a little bit silly — just perfect for him. With Patent distracted, Invention walks out with Amusement, even though she knows better. And then there’s Commitment, waiting in the background. A pleasant, steady fellow, Invention thinks, though perhaps a bit too eager to please. She agrees to keep him in mind. But while she dithers, Creativity, the town’s other beauty, manages to seduce him. The two of them marry and have so many babies so fast, no one can keep track.
Communication without Borders
Atas do Congresso Internacional Línguas 2011 – X Encontro da Aprolíngu@s e II Encontro da ReCLes.pt 59 Invention seeks her grandmother’s advice. Poverty tells her granddaughter, “A thing cannot be both a door and a key. A thing cannot be both a pair of glasses and an eye.” But Invention believes a thing can…
Invention looks at pictures of city people in the newspaper. She reads their names and the stories about them: Ambition, Achievement, Triumph, and Success. She’ll leave town and soon. That must be what the wings are for. Yes, soon she’ll kiss Necessity good-bye and go.
REFERENCES
Alderson, J. C., Clapham, C. and Wall, D. (1995). Language Test Construction and Evaluation. Cambridge: Cambridge University Press.
Aulenback, S. Necessity Is the Mother of Invention.
http://www.mcsweeneys.net/articles/necessity-is-the-mother-of-invention, 25.10.11 Instituto Politécnico da Guarda Plano Estratégico 2010-2011. Guarda: Publicações IPG.
Morley, J. (2000). The Chaplen Test Revisited. Assessing English for Academic Purposes, G. Blue, J. Milton, and J. Saville, eds. Bern: Peter Lang European Academic Publishers, 49-62.
Case, A. (2005). Review: New Headway Advanced. tefl.net - Independent Online Resource Network for English Teachers. http://www.tefl.net/reviews/new-headway-advanced.htm, 24.10.11.
Wood, M. M. and Morley, J. (2008). Question Types in English Language Diagnostic Testing.
http://abc.cs.man.ac.uk/docs/General/papers/QTELD.pdf, 24.10.11.
Communication without Borders
Atas do Congresso Internacional Línguas 2011 – X Encontro da Aprolíngu@s e II Encontro da ReCLes.pt 60