Education, Science, Technology, Innovation and Life
Open Access
Sign In
  1. Home
  2. Journals
  3. Journal of Language Testing & Assessment
  4. Vol 8, Issue 1, 2025

Argument-based validation frameworks in language testing: What are we expecting?

Download as PDF

DOI: 10.23977/langta.2025.080107 | Downloads: 3 | Views: 297

Author(s)

Don Yao 1

Affiliation(s)

1 College of Humanities and Foreign Languages, China Jiliang University, Hangzhou, Zhejiang, China

Corresponding Author

Don Yao

ABSTRACT

The emerging context of interest in language testing lies in the arena of argument-based approaches to validation especially over the last three decades. Among which, three argument-based validation frameworks are prevalent and have been adopted by numerous researchers in their scholarly research, i.e., (Kane, 1992; Kane,2006; Kane, 2012; Chapelle et al., 2008; Bachman & Palmer, 2010). The overarching purpose of the current paper is to critically review three contemporary validation frameworks and discuss the merits and demerits of each framework. Subsequently, classical argument-based validation research is reviewed to explore the relationship between validity evidence and test development and use and provide systematic and logical implications for further research. Results showed previous argument-based validation studies touched upon limited kinds of stakeholders, and only a partial validation framework was adopted to appraise certain inferences. Additionally, empirical research mainly focused on Kane's (1992, 2006, 2012) and Chapelle et al.'s (2008) frameworks. It is to be hoped that further research could take more kinds of stakeholders into consideration, and a more systematic and comprehensive validation study is suggested. Meantime, Bachman and Palmer's (2010) framework is also advocated because of its salient feasibility and practicability.

KEYWORDS

Argument-based approach, validation framework, critical review, merits and demerits, implications

CITE THIS PAPER

Don Yao, Argument-based validation frameworks in language testing: What are we expecting?. Journal of Language Testing & Assessment (2025) Vol. 1: 43-50. DOI: http://dx.doi.org/10.23977/langta.2025.080107.

REFERENCES

[1] Kane, M. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527-535. 
[2] Kane, M. (2006). Validation. In R. L. Brennan (4th Ed.), Educational measurement (pp.17-64). American Council on Education and Praeger.
[3] Kane, M. (2012). Articulating a validity argument.  In G. Fulcher, & F. Davison, The Routledge handbook of language testing. Routledge.
[4] Chapelle, C. A., Enright, M. K., & Jamieson, J. (2008). Building a validity argument for the test of English as a foreign language. Routledge.
[5] Bachman, L. F., & Palmer, A. S. (2010). Language assessment in the real world: Developing language assessments and justifying their use. Oxford University Press.
[6] Chapelle, C. A. (1999). Validity in language assessment. Annual Review of Applied Linguistics, 19, 254-272. 
[7] Chapelle, C. A., & Voss, E. (2014). Evaluation of language tests through validation research. In A. J. Kunnan (Ed.), The companion to language assessment (pp. 1079-1097). Wiley.
[8] Kunnan, A. J. (2017). Evaluating language assessments. Routledge.
[9] Messick, S. (1989). Validity. In R. L. Linn (3rd Ed.), Educational measurement (pp. 13-103). Macmillan.
[10] Knüsel, B., Baumberger, C., Zumwald, M., Bresch, D. N., & Knutti, R. (2020). Argument-based assessment of predictive uncertainty of data-driven environmental models, Environmental Modelling & Software, 134, 104754.
[11] Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (2nd Ed.), Educational measurement (pp. 443-507). American Council on Education.
[12] Cronbach, L. J. (1988). Five perspectives on the validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3–17). Lawrence Erlbaum Associates, Inc.
[13] House, E. R. (1980). Evaluating with validity. Sage Publications.
[14] Kane, M., Crooks, T., & Cohen, A. (1999). Validating measures of performance. Educational Measurement: Issues and Practice, 18(2), 5-17.
[15] Im, G. H., Shin, D., & Cheng, L. Y. (2019). Critical review of validation models and practices in language testing: Their limitations and future directions for validation research. Language Testing in Asia, 9(14), 1-26. 
[16] Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quart, 2(1), 1-34. 
[17] Toulmin, S. (2003). The uses of argument (2nd Ed.). Cambridge University Press.
[18] Bachman, L. F. (2004). Linking observations to interpretations and uses in TESOL research. TESOL Quarterly, 38(4), 723-728. 
[19] Bachman, L. F. (2006, April). Linking interpretation and use in educational assessments. National Council for Measurement in Education (NCME), San Francisco, U.S.
[20] Schmidgall, J. E. (2017). Articulating and evaluating validity arguments for the TOEIC tests. ETS Res Report. 
[21] Yao, D., & Wallace, M. P. (2021). Language assessment for immigration: A review of validation research over the last two decades. Frontiers in Psychology, 12, 773132.
[22] Kunnan, A. J. (2004). Test fairness. In M. Milanovic, & C. Weir (Eds.), Europe language testing in a global context: Selected papers from the ALTE conference in Barcelona (pp.27-48). Cambridge University Press. 
[23] Wallace, M. P. (2018). Fairness and justice in L2 classroom assessment: Perceptions from test takers. The Journal of Asia TEFL, 15(4), 1051-1064. 
[24] Wallace, M. P., & Qin, Y. (2021). Language classroom assessment fairness: Perceptions from students. LEARN Journal: Language Education and Acquisition Research Network, 14(1), 492-521. 
[25] Ho, A. O. K., Yao, D., & Kunnan, A. J. (2021). An analysis of Macau's Joint Admission Examination-English. Journal of Asia TEFL, 18(1), 208-222.
[26] American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. AERA.
[27] Kunnan, A. J. (2010). Test fairness and Toulmin's structure. Language Testing, 27(2), 183-189. 
[28] Yao, D., & Chen, K. (2020). Gender-related differential item functioning analysis on an ESL test. Journal of Language Testing & Assessment, 3, 5-19.
[29] Fan, J., & Yan, X. (2020). Assessing speaking proficiency: A narrative review of speaking assessment research within the argument-based validation framework. Front Psychol, 11(330), 1-14. 
[30] Chapelle, C. A., & Voss, E. (2021). Validity argument in language testing: Case studies of validation research. Cambridge University Press.
[31] Enright, M. K., & Quinlan, T. (2010). Complementing human judgment of essays written by English language learners with e-rater scoring. Language Testing, 27(3), 317-334.
[32] Bernstein, J., Van Moere, A., & Cheng, J. (2010). Validating automated speaking tests. Language Testing, 27(3), 355-377. 
[33] Pardo-Ballester, C. (2010). The validity argument of a web-based Spanish listening exam: Test usefulness evaluation. Language Assessment Quarterly, 7(2), 137-159.
[34] Koizumi, R., Saka, H., Ido, T., Ota, H., Hayama, M., Sato, M., & Nemoto, A. (2011). Development and validation of a diagnostic grammar test for Japanese learners of English. Language Assessment Quart, 8(1), 53-72.
[35] Lim, G. S. (2009). Prompt and rater effects in second language writing performance assessment. Deep Blue. 
[36] Chapelle, C. A., Chung, Y. R., Hegelheimer, V., Pendar, N., & Xu, J. (2010). Towards a computer-delivered test of productive grammatical ability. Language Testing, 27, 443-469. 
[37] Liu, M. (2013). Applying an assessment use argument to investigate a college-level English test in universities in Xi'an. PolyU Electronic Theses.
[38] Tominaga, W. (2014). Validating the score inference of the Japanese OPI ratings: The use of extended turns, connective expressions, and discourse organizations. Semantic Scholar. 
[39] Pochon-Berger, E., & Lenz, P. (2014). Language requirements and language testing for immigration and integration purposes. Report of the Research Center on Multilingualism, 2-40.
[40] Chapelle, C. A. (2020). Argument-based validation in testing and assessment. Sage Publications.

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.