A difficulty-informed approach to developing language assessment literacy for classroom purposes

Publications: Contribution to journalArticlePeer Reviewed


For pre-service teacher education, it would be helpful to know how difficult certain aspects of language testing and assessment are for students. Such information could serve different pedagogical purposes including program design and lesson sequencing. This paper, situated in the context of pre-service teacher education at the University of Vienna, Austria, presents an approach to developing language assessment literacy (LAL) which takes account of the difficulty of language assessment. To obtain empirical difficulty estimates, the items of a LAL test for pre-service teachers of English were converted into statements of ability, knowledge, and understanding and then calibrated by means of multi-facet Rasch analysis based on performance data from 420 students. Qualitative content analysis was used to identify clusters among the calibrations, thus characterizing the difficulty continuum. The findings show a clear progression which can offer a basis for the design of teacher education programs. I first report on the process of generating the difficulty estimates. Then I describe the resulting difficulty continuum with particular emphasis on classroom-based assessment. Finally, I suggest design principles for difficulty-informed assessment courses, illustrating how they could be implemented in the teacher education program at the University of Vienna.

Original languageEnglish
Pages (from-to)289-307
Number of pages19
JournalChinese Journal of Applied Linguistics
Issue number2
Publication statusPublished - 1 Jun 2023

Austrian Fields of Science 2012

  • 602007 Applied linguistics
  • 602008 English studies
  • 503007 Didactics


  • course design principles
  • difficulty of language assessment
  • language assessment literacy
  • multi-facet Rasch analysis
  • pre-service teacher education

Cite this