Supondo que suas variáveis dependentes tivessem uma distribuição normal ou que sua amostra fosse suficientemente grande, deve verificar todas as possibilidades de análise: univariada , bivariada, múltipla e multivariada, se for o caso. A análise univariada é quando a variável é analisada per se, análise bivariada quando uma variável dependente é relacionada com uma única variável independente, análise múltipla quando se analisa uma variável dependente em função de várias variáveis independentes, e análise multivariada, quando se analisa várias variáveis dependentes contínuas em função de variáveis independentes categóricas ou quando se analisa a estrutura das variáveis, visando a redução do número de variáveis.
O quadro anterior não esgota as analises estatísticas, aliás existem outras tantas quantas as que apresentamos aqui, contudo mostra as mais utilizadas nas análises univariadas e bivariadas.
“As técnicas utilizadas na avaliação psicológica têm provocado questionamentos nas comunidades científicas e profissionais brasileiras, tanto no que se refere à qualidade dos instrumentos, de maneira geral, ao uso que os psicólogos fazem dos instrumentos, bem como em relação à validade geral dos resultados da avaliação psicológica. O presente trabalho teve como objetivo identificar as principais informações psicométricas a respeito dos instrumentos psicológicos presentes em seus manuais, tais como editora, data de publicação, variável medida, padronização, validade e precisão” (Noronha, Primi, & Alchieri, 2004, p. 1).
Existe, também, um conjunto de técnicas estatísticas comummente utilizadas na avaliação das qualidades psicometricas das escalas. No quadro 2, nas colunas da direita (tipo de estatísticas) podem observar-se as estatísticas mais utilizadas para verificar os fenómenos descritos na primeira coluna.
QUADRO 2: GRELHA DE DECISÃO DOS TESTES PSICOMÉTRICOS
Tipo de estatisticas
Descritivas Análises Bivariadas Outras análises mais robustas
Para verificar a consistência temporal entre dois itens
Número de casos válidos, média e desvios padrão no caso das variáveis quantitativas e frequências e percentagens no caso das qualitativas
Pearson, Kappa de Cohen ou Spearman dependendo do nível de mensuração
Fornecer modelos matemáticos para a explicação de
teorias cognitivas e comportamentais Número de casos válidos, média e desvios padrão Não se aplica
Matriz de correlação e respectivos niveis de significancia; Matriz determinante e inversa;
Matriz anti imagem;
Solução factorial inicial com as respectivas comunalidades, valores próprios e variancia explicada; Medidas de adequação da amostra de Kaiser-Meyer-Olkin e de esfericidade de Bartlett's test; Matriz factorial não rodada incluindo pesos factoriais, comunalidades e valores próprios; solução factorial rodada incluindo padrão de rotação e matriz de transformação. No caso das rotações oblíquas para além do padrão de rotação é necessária a matriz da estrutura, a matriz dos coeficientes factoriais e a matriz da coovariancia.
Para além das estatísticas referidas, são úteis os gráficos: scree plot dos valores próprios e loading plot dos primeiros factores.
Estudo da dimensionalidade e redução de variáveis Número de casos válidos, média e desvios padrão Não se aplica
Estudo da relação entre itens;
Identificar itens problemáticos (consistência interna; reprodutibiliade, etc)
Número de casos válidos, média e desvios padrão Coeficiente de correlação inter- item e intraclass
Hotelling's T-square para a igualdade das médias e Teste de aditividade; Tukey's;
Estimativas de fidedignidade
[Alpha de Cronbach (baseado na correlação média inter-item): metade-metade de spearman-brown (split-half);
Guttman (limite inferior de fidedignidade);
modelo paralelo para variâncias iguais e para a igualdade das medias;
ANOVA (no caso de dados ordinais Friedman’s chi-square e coeficiente de concordância de Kendall e no caso de dados dicotómicos a estatística Q de Cochran)]
Bibliografia
Aiken, L. R. (1982). Psychological testing and assessment (4.ª edição ed.). Boston: Allier & Bacon, Inc.
Alferes, V. R. (2008, 08 21). Psicometria: Análise da consistência interna de Escalas
Somativas Dicotómicas. Retrieved 02 06, 2009, from SPSS: Programas e
rotinas complementares (syntax files):
http://www.fpce.uc.pt/niips/spss_prc/psicom/kr_20_21/kr_20_21.htm
American Psychological Association. (1985). Standards for educational and
psychological testing. Washington: American Psychological Association, Inc.
Anastasi, A. (1977). Testes psicológicos ( 2ª edição ed.). São Paulo: Editora Pedagógica e Universitária Lda.
Anastasi, A. (1988). Psychological testing. New York: Macmillan Publishing Company.
Borsboom, D., Van-Heerden, J., & Mellenbergh, G. J. (2003). Validity and truth.
Internal Report.
Bryant, F. B. (2000). Assessing the validity of measurement. In L. G. (Orgs.),
Reading and understanding more multivariate statistics (pp. 99-146).
Washington: American Psychological Association.
Bryman, A., & Cramer, D. (2003). Análise de dados em ciências sociais. Introdução
às técnicas utilizando o SPSS para windows (3ª Edição ed.). Oeiras: Celta.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, pp. 81-105.
Cronbach, L. .., & Meehl, P. (1955). Construct validity in psychological tests.
Psychological Bulletin, 4, pp. 281-302.
Cronbach, L. J. (1949). Essentials of psychological testing.
Eco, U. (2002). Como se faz uma tese em ciências humanas (9 ed.). Lisboa: Editorial Presença.
Emory, C. (1985). Business research methods The Irwin Series in Information and
Faculdade de Medicina da Universidade do Porto. (2000). Medicina e Ciência - Do
Método Científico ao Método Clínico. (Serviço de Bioestatística e Informática
Médica) Retrieved Fevereiro 9, 2009, from MedStatWeb: um curso de
estatistica médica na Web:
http://stat2.med.up.pt/cursop/print_script.php3?capitulo=medicina_ciencia&nu mero=6&titulo=Medicina%20e%20Ci%C3%83%C2%AAncia
Freeman, F. (1980). Teoria e Prática dos Testes Psicológicos (2ª Edição ed.). Lisboa: Fundação Calouste Gulbenkian.
Golden, C. J., Sawicki, R. F., & Franzen, M. D. (1984). Test construction. In M. H. G. Goldstein, Handbook of psychological Assessment. New York: Pergaman Press.
Guilford, J. P. (1946). New standards for test evaluation. Educational and
Psychological Measurement,, pp. 427-439.
Gulliksen, H. (1950). Intrinsic validity. American Psychologist, pp. 511-517.
Haynes, S. N., Richard, D. C., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods.
Psychological Assessment, 3, pp. 238-247.
ITC-International Test Commission. (2008). International Test Commission
Guidelines Index. Retrieved Fevereiro 09, 2010, from International Test
Commission: http://www.intestcom.org/guidelines/index.php
Janis, I. L. (1965). The problem of validating content analysis. In N. L. H. D. Lasswell, Language of politics. Cambridge: MIT Press.
Kline, P. (1987). A handbook of test construction: Introduction to psychometric
design. London: Routledge Kegan & Paul.
LAP- Laboratório de Avaliação Psicológica. (2007). Introdução à Psicometria (curso
on-line). Retrieved Janeiro 5, 2010, from LAP- Laboratório de Avaliação
Psicológica: http://www.lap-am.org/ativ_concluida1.html
Marx, R., Bombardier, C., Hogg-Johnson, S., & Wright, J. (1999, Fevereiro). Clinimetric and Psychometric Strategies for Development of a Health Measurement Scale. Jourrnal of Clinical Epidemiology, 52 (2), pp. 105-111.
Matos, M. (1994, Janeiro). Normas para apresentação de dissertações: Bases
Essenciais. Retrieved Janeiro 2009, from paginas.fe.up.pt/~mam/normas.pdf
Messick, S. (1989). Validity. Em R. Linn (Org.), Educational measurement (3ª Edição ed.). New York: American Council on Education and Macmillan Publishing Company.
Mosier, C. (1947). A critical examination of the concepts of face validity. Educational
and Psychological Measurement, 7, pp. 191-205.
Mosier, C. (1951). Problems and designs of cross-validation. Educational and
Psychological Measurement, 11, pp. 5-12.
Noronha, A. P., Primi, R., & Alchieri, J. C. (2004, Dezembro). Psychometrics parameters: an analysis of the psychological tests commercialized in Brazil.
Psicologia: ciência e profissão, 24(4), pp. 88-89.
Nunnally, J. C. (1978). Tests and measurements. New York: McGraw-Hill.
Pasquali, L. (2007). Validade dos Testes Psicológicos: Será Possível Reencontrar o Caminho? Psicologia Teoria e Pesquisa, 23, pp. 099-107.
Passarelli, B. (1995). Teoria das Múltiplas Inteligências aliada à Multimídia na
Educação: Novos Rumos Para o Conhecimento. Retrieved 02 17, 2009, from
http://www.futuro.usp.br/producao_cientifica/artigos/multiplasintelig.pdf Pocinho, M. T. (2007). Factores socioculturais, depressão e suicidio no idoso
alentejano. Tese de Doutoramento, ICBAS, Ciências Biomédicas, Porto.
Prieto, G., & Muñiz, J. (2000, Novembro). Un modelo para evaluar la calidad de los
tests utilizados en España. Retrieved Fevereiro 02, 2009, from Papeles del
Psicólogo: http://www.cop.es/tests/modelo.htm.
Quivy, R., & Campenhoudt, L. V. ( 1992). Manual de Investigação em Ciências
Sociais. Lisboa: Gradiva.
Teixeira, J. M. (2001). Medicina baseada na evidência. Saúde Mental, III(Editorial), pp. 5-8.
Weber, R. P. (1990). Basic content analysis (2.ª Edição ed.). Newbury Park: Sage Publications.
White, E., Armstrong, B., & Saracci, R. (1992). Principles of Exposure Measurement
in Epidemiology. New York: Oxford University Press.
Wright, J. G., & Feinstein, A. (1992). A comparative contrast of clinimetric and psychometric methods for constructing indexes and rating scales. Journal
INTERNATIONAL TEST COMMISSION
INTERNATIONAL TEST COMMISSION
INTERNATIONAL GUIDELINES FOR TEST USE
The Council of the International Test Commission (ITC) formally adopted the Guidelines at its June 1999 meeting in Graz, Austria.
The European Federation of Professional Psychologists Associations’ Task Force on Tests and Testing also endorsed the Guidelines at its July 1999 meeting in Rome.
The Guidelines were officially published at the General Meeting of the ITC on 24 July, 2000 in Stockholm. The Guidelines are copyright of the ITC, 1999. The ITC is a non-stock corporation incorporated in the USA.
For further information on the ITC or for permission to quote from or reproduce the contents of this document, please contact the Secretary of the ITC:
Prof Jacques Grégoire, ITC Secretary, Université catholique de Louvain, Faculté de Psychologie,
Place du Cardinal Mercier, 10, 1348 Louvain-la-Neuve, Belgium.
Email: [email protected]
For further information regarding the Guidelines, contact: Prof Dave Bartram, ITC President,
SHL Group plc, The Pavilion, 1 Atwell Place, Thames Ditton, KT7 0NE, Surrey, England. Email: [email protected]
See also the ITC website (from which copies of the Guidelines can be obtained): http://www.intestcom.org
Contents
Acknowledgements 11
Introduction and background 13
The need for international Guidelines 13 Development of the Guidelines 15
The Guidelines 19 Key purpose 19
Scope of the Guidelines 19 Who the Guidelines are for 21 Contextual factors 22
Knowledge, Understanding, and Skill 23
1 Take responsibility for ethical test use 27 1.1 Act in a professional and ethical manner 27
1.2 Ensure they have the competence to use tests 27 1.3 Take responsibility for their use of tests 28
1.4 Ensure that test materials are kept securely 28
1.5 Ensure that test results are treated confidentially. 29 2 Follow good practice in the use of tests 30
2.1 Evaluate the potential utility of testing in an assessment situation 30 2.2 Choose technically sound tests appropriate for the situation 30 2.3 Give due consideration to issues of fairness in testing 31
2.4 Make necessary preparations for the testing session 33 2.5 Administer the tests properly 34
2.6 Score and analyse test results accurately 35 2.7 Interpret results appropriately 36
2.8 Communicate the results clearly and accurately to relevant others 37 2.9 Review the appropriateness of the test and its use 38
References 39
Appendix A: Guidelines for an outline policy on testing. 41
Appendix B: Guidelines for developing contracts between parties involved in the testing process. 43
Appendix C: Points to consider when making arrangements for testing people with disabilities or impairments 46
Appendix D. Conditions governing the translation of the ITC Guidelines on Test Use. 49
Acknowledgements
The Guidelines were prepared for the ITC Council by Professor Dave Bartram. The author is grateful for the assistance provided by Iain Coyne in the execution of this project and is grateful to the following individuals who took part in the 1997 Dublin workshop and who provided such valuable input to the development of the present Guidelines.
Ms Dusica Boben, Produktivnost, SLOVENIA;
Mr Eugene Burke, British Psychological Society, England; Dr Wayne Camara, The College Board, USA;
Mr Jean-Louis Chabot, ANOP, FRANCE; Mr Iain Coyne, University of Hull, England; Dr Riet Dekker, Swets and Zeitlinger, Netherlands;
Dr Lorraine Eyde, US Office of Personnel Management, USA; Prof Rocio Fernandez-Ballesteros, EAPA, SpAIN;
Mr Ian Florance, NFER-NELSON, England;
Prof Cheryl Foxcroft, Test Commission of South Africa, South Africa; Dr John Fremer, The College Board, USA;
Ms Kathia Glabeke, Commissie Psychodiagnostiek, BELGIUM; Prof Ron Hambleton, University of Massachusetts at Amherst, USA; Dr Karin Havenga, Test Commission of South Africa, South Africa; Dr Jurgen Hogrefe, Hogrefe & Huber Verlagsgruppe, Germany; Mr Ralf Horn, Swets and Zeitlinger, Germany;
Mr Leif Ter Laak, Saville and Holdsworth Ltd, England; Dr Pat Lindley, British Psychological Society, England;
Mr Reginald Lombard, Test Commission of South Africa, South Africa; Prof Jose Muniz, Spanish Psychological Association, Spain;
Ms Gill Nyfield, Saville & Holdsworth Ltd, England; Dr Torleiv Odland, Norsk Psykologforening, Norway;
Ms Berit Sander, Danish Psychologists’ Association,DENMARK; Prof Francois Stoll, Federation Suisse des Psychologues, Switzerland.
The author is also grateful to the many other individuals and organisations who provided feedback during the various stages of consultation and in conference presentations.
Introduction and background
The need for international Guidelines
The focus of the International Test Commission (ITC) project is on guidelines for good test use and for encouraging best practice in assessment. The work so far carried out by the ITC to promote good practice in test adaptations (Hambleton, 1994; Van de Vijver, F. & Hambleton, R., 1996) is an important step towards assuring uniformity in the quality of tests adapted for use across different cultures and languages. At its meeting in Athens in 1995, the ITC Council accepted a proposal to broaden this concern to include guidelines on the fair and ethical use of tests, from which standards for training and specifying the competence of test users could be derived.
There are a number of reasons why guidelines on test use are needed at an international level.
Countries differ greatly in the degree, if any, of statutory control they can exercise over the use of testing and its consequences for those tested. Some national professional societies have statutory registration of psychologists, others do not; some have mechanisms for the control of standards of test use by non-psychologists, others do not. The existence of a set of internationally-accepted guidelines would provide national psychological associations and other relevant professional bodies and organisations with a degree of support in the endeavours of such organisations to develop standards in countries where such standards are currently either lacking in some respect or non-existent.
Patterns of access, in terms of the rights to purchase or use test materials, vary greatly from country to country. In some countries, access is restricted to psychologists, in others to users registered with formally approved national test distributors, in yet others, test users may be free to obtain materials without restriction from suppliers in their country or directly from suppliers abroad.
A number of well-known instruments have appeared on the Internet in violation of copyright, without acknowledgement of the test authors or publishers, and without regard to issues of test security.
Within the occupational testing arena, the greater international mobility of labour has increased the demand for tests to be used on job applicants from a number of different countries - often with the tests being administered in one country on behalf of a potential employer in another country.
Development work is being carried out in the USA and in the UK on the use of Internet for distance- or remote-assessment in both occupational and educational settings. This raises a whole host of issues relating to standards of administration and control over the testing process, including test security.
Aim and objectives
The long-term aim of this project includes the production of a set of guidelines that relate to the competencies (knowledge, skills, abilities and other personal characteristics) needed by test users. These competencies are specified in terms of assessable performance criteria. These criteria provide the basis for developing specifications of the evidence of competence that would be expected from someone seeking qualification as a test user. Such competencies need to cover such issues as: professional and ethical standards in testing,
rights of the test taker and other parties involved in the testing process, choice and evaluation of alternative tests,
test administration, scoring and interpretation, report writing and feedback.
Insofar as they directly relate to test use, the Guidelines also have implications for: standards for test construction,
standards for regulating the supply and availability of tests and information about tests.
The present Guidelines represent the work of specialists in psychological and educational testing (i.e. psychologists, psychometricians, test publishers and test developers) drawn from a number of countries. The intention of this document is not to ‘invent’ new guidelines, but to draw together the common threads that run through existing guidelines, codes of practice, standards and other relevant documents, and to create a coherent structure within which they can be understood and used.
Development of the Guidelines
The Guidelines should be considered as benchmarks against which existing local standards can be compared for coverage and international consistency. By using the Guidelines as benchmarks or the basis from which to develop locally applicable documents (e.g. standards, codes of practice, statements on test taker rights), a high level of consistency across national boundaries will be promoted.
Work on the Guidelines began by drawing together materials concerned with test standards, codes of practice, test use, etc., from a number of countries3. While drawing on all of these sources, the present Guidelines have been particularly influenced by:
The Australian Psychological Society (APS) Supplement to guidelines on the use of Psychological Tests (Kendall et al., 1997).
The British Psychological Society (BPS) Level A and Level B standards for occupational test use (Bartram, 1995, 1996).
The American Educational Research Association (AERA) , American Psychological Association (APA), & National Council on Measurement in Education (NCME) (1985) Standards for educational and psychological testing.
American Association for Counseling and Development (AACD) Responsibilities of Users of Standardized Tests (Schafer, W.D, 1992).
The CPA (Canadian Psychological Association, 1987) Guidelines for Educational and Psychological Testing.
The APS document has been particularly valuable as it pulls together much of what is contained in the BPS and American publications as well as drawing on South African National Institute for Psychological Research (NIPR) publications and various publishers’ guidance for test users. It also embodies much of what has come out of Joint Committee on Testing Practices (JCTP) Test User Qualifications Working Group’s (TUQWG) seminal work on a data-based approach to promoting good test use (e.g., Eyde et al, 1988, 1993; Moreland et al., 1995), and the work of the JCTP on the Code of Fair Testing Practices in Education (JCTP, 1988; Fremer, Diamond, & Camara, 1989). Appendix B drew from the more recent work of the JCTP (JCTP, 2000) on test taker rights and responsibilities.
The content of the primary sources was analysed and statements categorised under 14 main headings. Where appropriate single statements were written to capture the common meaning of a number of statements from different sources. Statements were also modified in format such that they provided completion of a common stem (e.g.: “Competent test users will endeavour to....”, or “Competent test users can....”).
This initial structure of 14 main sections and their content was embodied in the first draft Framework Document. This formed the material for an international workshop held in Dublin in July 1997. The purpose of the ITC Workshop was to consider and critically evaluate all aspects of a framework document, with a view to producing a draft set of guidelines that would have international currency and acceptance. During the workshop, the Framework Document was examined in detail, with refinements being proposed in terms of form, structure and content. Following the workshop, the document was extensively revised (Version 2.0) and circulated to all those who attended for comment. A draft consultation document (Version 3.1) was prepared that incorporated all the comments and suggestions submitted regarding Version 2.0.
Copies of the Version 3.1 consultation document and a structured response form were widely circulated to key individuals and organisations for comment. A total of 200 were distributed. A total of 28 detailed responses were received including ‘corporate’ responses from the APA, the BPS and some other European professional associations. In the summer of 1998 the Guidelines were revised in the light of these comments, and 200 copies (Version 4.1) were sent out for further consultation. A total of 18 formal responses were received to this second round of consultation. In addition, supportive informal comments were provided by many recipients of the consultation documents by email or in meetings.
In producing the current version of the Guidelines (Version 2000), every effort has been made to take account of all these responses. Without exception, the responses were helpful and constructive.4
These Guidelines are to be seen as supportive rather than constraining. We need to ensure that the Guidelines embody universal key principles of good test use, without attempting to impose uniformity on legitimate differences in function and practice