Asian Association for Language Assessment

To  date, little remains unknown about an ‘optimal condition’ in which  appropriate and desirable strategy use results in significantly better  test performance. Language testing researchers and test developers do  not have a sufficient empirical understanding about test-takers’  metacognitive judgments about their current test performance and factors  affecting their judgment accuracy. This presentation reports on an  empirical study that investigates test-takers’ calibration and its  relationship to reported strategy use in an IELTS listening test.  Test-taker calibration denotes a perfect relationship between confidence  in performance success and actual performance outcome. In other words, a  study of calibration aims to evaluate an alignment between test-takers’  perceived confidence and their actual performance. Calibration or  miscalibration thus indicates the nature of test-takers’ metacognitive  judgment, monitoring accuracy and/or self-appraisal. 388 English as a  second language (ESL) test-takers in Australia took part in this study.  Before they took the listening test, they were asked to report on their  general strategy use in IELTS listening tests. While completing each of  the IELTS test questions, they were asked to report on the level of  their confidence in the correctness of their answer (e.g., 0% to 100%)  and at the end of the test, they were asked to report their cognitive  and metacognitive strategy use during this listening test. Their  calibration, confidence, and reported strategy use scores were analyzed  together using serval statistical tests and in particular a structural  equation modeling (SEM) approach. It was found that on average the test  takers were not well calibrated and had a tendency to be overconfident  across the listening test sections. Their calibration scores and  confidence ratings in performance were positively, yet marginally  related to their cognitive and metacognitive strategy use. The present  study helps us advance our knowledge of strategic processes including  calibration and strategy use that are part of or affect listening test  performance (IELTS). Implications of the study and recommendations for  future research will be articulated.