2017
Workshop
Alister Cumming
Verbal Reports on Writing Assessments
The purpose of this workshop is to introduce participants to, and to evaluate, methods of verbal reporting for illuminating, understanding, and validating the constructs of writing assessed in the contexts of classroom instruction as well as large-scale standardized tests. Approaches will be demonstrated and evaluated for their merits, limitations, and prevailing issues in (a) collecting data (participant orientation, concurrent think-alouds, summary recalls, stimulated recalls, interviews, focus groups, equivalency of conditions, natural vs. experimental contexts and motivations), (b) analyzing data (segmentation, coding, frequency counts, focused probes, group differences, triangulation with complementary data sources), (c) pedagogy (teacher demonstration, student orientation, task analysis, self- or peer-evaluation), and (d) interpretation (for exploratory or confirmatory purposes, reducing reactivity, acknowledging partial representations of cognitive and writing processes, member checks).
Alister Cumming is professor emeritus in the Centre for Educational Research on Language and Literacies (formerly the Modern Language Centre) at the Ontario Institute for Studies in Education, University of Toronto, where he has been employed since 1991 following briefer periods at the University of British Columbia, McGill University, Carleton University, and Concordia University. For 2014 to 2017 Alister is also a Changjiang Scholar in the National Research Centre for Foreign Language Education at Beijing Foreign Studies University. His research and teaching focus on writing in second languages, language assessment, language program evaluation and policies, and research methods