Asian Association for Language Assessment

When students can ask ChatGPT to help them write an essay or practice a conversation, what does it really mean to be a competent language user? This is not merely an academic integrity issue. It challenges us to rethink what language ability actually entails (Xi, 2025). If AI can produce fluent text, then competence may lie less in the final product itself and more in learners’ capacity to engage meaningfully with language tasks and use feedback to improve. This plenary argues that learning-oriented assessment (LoA) offers a constructive response to this challenge because it shifts the focus from measuring linguistic output to facilitating deeper cognitive and metacognitive competencies (Turner & Purpura, 2016).

Drawing on two research projects, BalanceAI with school-age children and APLUS with university students, the talk examines how AI-supported LoA can provide timely feedback that strengthens learner agency and self-regulated learning through the metacognitive cycle of goal-setting, monitoring, and reflection that characterizes autonomous learners (Zimmerman, 2008). For teachers, such assessment systems can provide timely diagnostic information that supports responsive instruction. Yet, realizing this promise requires more than technical proficiency. Teachers need critical AI literacy to understand not only how to use these tools, but also how to question their limitations and blind spots. The reality is that AI systems often work less well for the students we serve. Research shows scoring differences for students from different linguistic backgrounds, and speech recognition systems may privilege native-speaker speech features (Hannah et al., 2022; Jang & Sawaki, 2025; Kim & Ockey, 2024; Wang & von Davier, 2014), Teachers therefore play a crucial role in interpreting results, challenging bias, and ensuring that AI serves all learners fairly.

The talk concludes by charting future directions for the language assessment community, advocating for culturally responsive AI design, human-in-the-loop approaches (Bolender et al., 2023), regional validation studies, and collaborative frameworks that welcome Asian voices in shaping how these technologies are developed and deployed.