Antony Kunnan
Antony John Kunnan is a language assessment specialist currently with Duolingo as a Principal Assessment Scientist and with Carnegie Mellon University as a Senior Research Fellow. His research interests are fairness of tests and testing practice, assessment literacy, research methods and statistics, ethics and standards, and language assessment policy. After completing his Ph.D. from UCLA, he has held academic positions in Ann Arbor, Los Angeles, Taichung, Hong Kong, Singapore and Macau. He has conducted 120 seminars, workshops, plenary and invited talks in 36 countries and published widely by authoring and editing books and writing journal articles and book chapters. His most recent book is Evaluating Language Assessments (2018); his most recent journal article is “Developing a Scenario-based English language assessment for an Asian University” in Language Assessment Quarterly (2022, with C. Qin and C. Zhao); and his most recent book chapter is “Revisiting language assessment for immigration and citizenship: The case of the U.S. Naturalization Test” in The 2nd Handbook for Language Testing edited by G. Fulcher and L. Harding (2022). He was founding editor of Issues in Applied Linguistics (1989-199, founding editor of Language Assessment Quarterly (2003-13) and Asian TEFL Journal (2017-). He was past president of the International Language Testing Association and founding president of the Asian Association for Language Assessment. More details about him are available at: www.antonykunnan.com
​
Title: Applying Fairness and Justice principles to English language assessments in southern India
​
Abstract: In Evaluative Language Assessments (Kunnan, 2018), I promoted two principles: the Principle of Fairness and the Principle of Justice. In Principle 1, I proposed that an assessment ought to be fair to all test takers; that is, there is a presumption of treating every test taker with equal respect with four sub-principles regarding assessments: (1) adequate opportunity to acquire the knowledge, abilities, or skills to be assessed; (2) consistent and meaningful in terms of its test score interpretation; (3) free of bias, in particular, by avoiding the assessment of construct-irrelevant matters; and (4) appropriate access, administration, and standard-setting procedures. In Principle 2, I proposed that an assessment institution ought to be just, bring about benefits in society, promote positive values, and advance justice through public reasoning with two sub-principles: (1) foster beneficial consequences to the test-taking community and (2) promote positive values and advance justice through public reasoning. These principles have been used to evaluate language assessments in different contexts (example, Hidetoshi et al., 2022).
In this presentation, I will present an overview of the first principle and focus on a few sub-principles related to Class 10 English language exams. In Karnataka, schools can be affiliated to one of many Boards of Education: the Karnataka State Educational Board (KSEB), the Central Board of Secondary Education, and the Indian Council of Secondary Education among others. All three Boards have English language exams for the equivalent of Grade 10 and the latter two Boards have them for Grade 12. These exams are part of the school leaving exams which include exams in languages (L1s, L2, etc.) and subject exams (such as History, Geography, Physics, etc.). The Class 10 or Secondary School Leaving Certificate (SSLC) exams can be seen as operationalized versions of the Boards syllabuses and curricula. A close examination of the KSEB SSLC Class 10 exam papers for English – Second Language from 2013 to 2022 was conducted (80 marks, 3 hours). The general findings are that there is a mixture of tasks related to various formal lexico-grammatical features, memory of studied literary works, reading comprehension and short writing tasks. The lexico-grammatical tasks are generally discrete-point test tasks that do not represent the language skill areas that might be useful for further study or employment. The studied literary works assess memorized responses and the single reading comprehension material and short writing tasks also do not represent any language skills that would be useful beyond high school. In addition, after examining all the web links on the KSEB SSLC websites, there was no information for public viewing that addresses other sub-principles: consistency in scoring, bias-free test texts and items, standard-setting across exams across administrations. Finally, given such lack of evidence in support of the SSLC exam, it is doubtful that the exam is providing beneficial consequences to test takers and stakeholders. In summary, this study shows that there is little evidence to support the sub-principles of Principle 1.
Rukmini Banerji
Rukmini Banerji has been with Pratham (www.pratham.org) since 1996. She has extensive field experience in program design and implementation and has led many of Pratham’s partnerships with state governments. Rukmini also led Pratham’s research activities, including the well-known Annual Status of Education Report (ASER) effort from 2005 to 2014 (www.asercentre.org). She became CEO of Pratham Education Foundation in July 2015. Initially trained as an economist in India, she did her B.A. at St. Stephen’s College and attended Delhi School of Economics. She was a Rhodes Scholar at Oxford University and completed her Ph.D. at the University of Chicago. In 2008, Rukmini was awarded the Maulana Abul Kalam Shiksha Puraskar by the Government of Bihar. She was the first recipient of this award. Also, she is the 2021 Yidan Prize awardee for education development. Originally from Bihar, she is now based between New Delhi and Pune. Rukmini writes frequently on education in both Hindi and English and enjoys creating stories for children.
​
Title: Assessment to Action: Opportunities for India
Abstract: India’s new education policy (NEP 2020) provides clear directions and goals for what we need to achieve as a county in the next few years. If we are successful in achieving foundational literacy and numeracy goals in primary school for most children, then we will be in a strong position to think about future progress. What do we know about foundational literacy in India so far – how do we measure it, how do we translate understanding of children’s realities into action. In order to build a strong foundational stage, assessment has to be linked to concrete action that is helpful to children and teachers to make progress together.
Constant Leung
Constant Leung is Professor of Educational Linguistics in the School of Education, Communication and Society, King’s College London. His research interests include academic literacies, additional/second language teaching and assessment, language policy, and teacher professional development. He is Editor of Research Issues of TESOL Quarterly, and serves on a number of editorial boards, including Australian Review of Applied Linguistics, Critical Inquiry in Language Studies, Language and Education, the Modern Language Journal and Research in the Teaching of English. His work in developing the English as an Additional Language Assessment Framework for Schools (funded by the Bell Foundation) was the recipient of the 2018 British Council ELTons International Award for Local Innovation. He is a Fellow of the Academy of Social Sciences.
​
Title: English Language Proficiency – A Changing Story
Abstract: The assessment of language proficiency, particularly in the form of standardized language testing, is often associated with the notions of conceptual stability and measurement trustwor¬thiness. In this talk my focus is on the shifting conceptualizations of English as an additional/second language proficiency in the past fifty years or so. I will argue that the notion of English proficiency, as it has been discussed in the world-wide English language Teaching (ELT) literature, is an artefact shaped by the ebbs and flows of intellectual movements and conceptual recontextualizations. The onset of the concept of communicative competence in the 1970s serves as the point of departure for this discussion. I will first give an account of the ways in which this concept has been filtered through a particular set of disciplinary and ideological perspectives that gave rise to a particular Anglocentric view of language communication and a universalist approach to curriculum development and assessment of proficiency. Following on from that, I will look at some of the recent research in relevant fields such as English as a Lingua Franca, flexible multilingualism and translanguaging that recognize more cross-lingual, dynamic and situated language use; these bodies of research have also signalled the need to embrace multilingualism in conceptualizing language proficiency. That said, there are complex implications of this more fluid view for language assessment. I will illustrate some of the challenges by analysing the conceptual and technical difficulties found in an international language curricu¬lum and assessment framework as it attempts to embody a more interactionally dynamic, linguistically fluid approach. In the final part of the discussion, I will explore the possible ways of expanding the notion of English language proficiency that can work with local language practices and assessments productively in different world locations.
James Enos Purpura
James Enos Purpura is Professor of Linguistics and Education in the Applied Linguistics and TESOL Program at Teachers College, Columbia University, where he teaches courses in SFL assessment. He is also Director of the Scenario-Based Language Assessment Lab. Jim has published several books and articles, and given scholarly talks on the following topics: cognition and language assessment, the assessment of grammar and pragmatics, integrated content and language instruction and assessment in ESP contexts, learning-oriented language assessment, and more recently scenario-based language assessment. Jim was President of ILTA, Editor of Language Assessment Quarterly, and a Fulbright Scholar in Italy.
​
Title: Using a learning-oriented approach to design scenario-based assessments of situated L2 proficiency in standardized and classroom contexts
Abstract: Over the years, the social, cultural, economic, and technological forces in our daily lives, in education, and at work have placed enormous demands on individuals to develop and deploy a range of competencies in order to be able to build and maintain relationships, access and manage goods and services, build and share knowledge, and work collaboratively for the social good (European Commission 2018; Gordon Commission 2013; Ontario Ministry of Education 2016: Organization for Economic Co-Operation and Development, 2003, 2019). As a result, the notion of “competency” has long served as an organizing principle for determining if educational efforts have succeeded in producing citizens with the skills needed to contribute meaningfully to society.
In the context of second and foreign language (S/FL) education, competencies have also been proposed in many contexts as a basis for second and foreign language (S/FL), where curricula and assessment might be organized around competencies such as the ability to summarize and synthesize information, or the capacity to solve problems collaboratively.
Critical to the development or successful deployment of these real-life competencies are obviously the fundamental linguistic resources of communication (Purpura & Dakin, 2019). However, successful communication in most language use situations involves much more than linguistic resources; it also requires the ability to marshal topical resources, contextual resources, socio-cognitive resources (e.g., learning, reasoning), affective resources (e.g., engagement, focus), social-interactional resources (e.g., collaborating), and in current times, technological resources (Purpura, 2018). Given this complex landscape, how might we reimagine the kinds of assessments we use in both standardized and classroom contexts to mirror the sociocultural and sociocognitive practices of situated S/FL use?
In order to shift perspectives on assessment, this talk will argue that traditional language-based, independent and integrated skill-based, and task-based approaches to S/FL proficiency assessment can be useful in certain assessment contexts; however, they are not actually engineered to measure S/FL proficiency in contexts of “situated” S/FL use – that is, where goal-oriented task accomplishment is located within a sociocultural context, and where the ability to accomplish simple and complex tasks is embedded within the mediated engagements and social practices of a particular community (Purpura, 2021).
To argue this, I will first recount a common, real-life collaborative problem-solving anecdote for consideration. This scenario is used to show how the various features of the event require examinees to engage different resources to perform successfully, thereby confirming the utility of “scenario” as a technique for assessing S/FL proficiency. I will then examine different approaches to assessing proficiency, highlighting the interpretative value of each. I will end the discussion by showing how a learning-oriented approach to assessment (LOA) (Purpura & Turner, 2018) can serve as a comprehensive conceptual assessment framework for converting “traditional” assessments into ones that are scenario-based and learning-oriented.
Jayakaran Mukundan
Jayakaran Mukundan, PhD, retired as Professor at UPM (Universiti Putra Malaysia), Malaysia, where he’s presently Honorary Professor. He was a secondary school teacher with the Ministry of Education for 11 years. He served on the National Panel for the Class Reader Program as well as on several Ministry of Education Assessment Panels. He also was for many years an evaluator for public exams. Then in 1990 he joined UPM as a Teacher Educator.
​
As a researcher he works mainly on ELT Materials investigations, the latest (2022) being a collaborative effort with professors at Imam Abdulrahman bin Faisal University, Dammam, Saudi Arabia. He has won several awards: in 2013, he won the highest award, the National Award for Academic Excellence in the Teaching Category. His research on Teaching Materials (Textbook Evaluation software) has won him Gold Medals at the British Invention Show, London (2009) and at IENA, Nuremberg, Germany, (2010).
​
Title: What shapes Teacher Fairness in Writing Assessment?
Abstract: Fairness in assessment has of late come under intense focus, as it is now viewed as just as important as validity. The aim of this study was to use two case studies, The Teacher and Adeline Ng Ai Choo, both short stories by Catherine Lim, set in writing classes in Singapore to investigate teacher fairness. Both stories are on writing teachers, whose decisions, which could be considered lacking in fairness, led to the humiliation of the protagonists, both of whom eventually committed suicide. Twenty six post graduate students, in an intact group enrolled on a Professional Development in TESL MA course took part in the study.
After reading both cases, they were asked to respond by way of a narrative on who was to blame for the deaths of the protagonists in both stories. After this the researcher conducted focus-group sessions and interviews so that data from these sources could be used for triangulation purposes. Qualitative interpretations followed. The findings revealed that teacher awareness on fairness in writing assessment was lacking as the subjects seemed to be more aware of what teachers had to do in the evaluation of writing rather than what human considerations and fairness should apply in differing contexts. The researcher believes that disciplinary knowledge deficiencies were the main reason why awareness of what constitutes fairness was lacking in these teachers.