Asian Association for Language Assessment

Over the years several researchers (e.g., Bachman, 1990; Bachman & Palmer, 1996, 2010; Carroll, 1968; Davidson & Lynch, 2002; Lado, 1960) have proposed frameworks for the construction of language assessments. These frameworks have focused on the specification of task characteristics, which serve to control the elicitation of performance consistencies on language tests. Many of the frameworks also align with models L2 proficiency to specify which language use behaviors are likely to be elicited by any one task. In other words, the core focus of assessment is on the proficiency and elicitation (task) dimensions of assessment events. 

A further concern for testers, especially in the development and validation of performance assessments (McNamara, 1996), task-based assessments (Norris, 2017), or more recently, scenario-based assessments (Purpura, 2019), relates to the importance of aligning assessments with real world purposes, that is, ensuring the features of test tasks correspond to those of real world tasks, so that performance on test tasks can generalize to performance on tasks in the language use domain (Bachman & Palmer, 1996). In this respect, a third dimension of an assessment event, the contextual dimension, is an essential factor in assessment design frameworks.

 

While these frameworks are efficient in the design and measurement of relatively simple independent skills tasks situated within the confines of a hermetic task situation capable of targeting limited constructs of interest, and while well-established measurement models exist to evaluate evidence of performance consistencies across test facets, these frameworks are actually ill-equipped to provide a systematic means of designing or analyzing performance in more complex language use situations, where interpretations of success do not depend on the ability to succeed on a single decontextualized, independent skills task, but rather on the ability to succeed across a coherent sequence of interrelated tasks (e.g., collaborative problem-solving). This inadequacy is further highlighted in the assessment of language use that is situated within a sociocultural context, requires the integration of more than linguistic resources (e.g., topical, socio-cognitive, interactional, affective), and might require examinees not only to process but also remember test input, so that they can use this information to communicate in ways that reflect the accumulation of understandings acquired across the language use event.

One solution to addressing these inadequacies is through evidence-centered design (ECD) (Mislevy et al, 2002; Mislevy & Riconscente, 2005), which highlights not only the interplay between the proficiency dimension, elicitation (task) dimension, and contextual dimensions (domain analysis) of an assessment event, but also provides mechanisms for both understanding performance across a sequence of interrelated tasks in a language use situation, and relating design to analysis through evidential reasoning. 

It remains unclear, however, how these frameworks might handle the design and analysis of assessments contextualized within language use situations, where instruction, in the form of test input, assistance, or feedback play a role in the event, where learning is expected, and where the assessment event is mediated through social interaction, especially considering how the instructional, socio-cognitive, and social-interactional dimensions of the event can moderate performance. In these assessment contexts, a learning-oriented framework of assessment might help explain the different dimensions of the assessment event and highlight their interplay.

In this talk I will examine the strengths and weaknesses of several assessment frameworks, arguing that they all fail to account for critical dimensions of assessment event when language use is assessed in complex performance assessment situations. I will use an example of complex assessment event to illustrate and describe the different dimensions of the LOA framework along with their synergies. If time, I will also illustrate this with a spontaneous classroom assessment mediated by interaction.