Education, Science, Technology, Innovation and Life
Open Access
Sign In

Assessing Differential Item Functioning (DIF) For Pearson Test of English (PTE)-A study of Test Takers with Different Fields of Study

Download as PDF

DOI: 10.23977/appep.2022.030103 | Downloads: 14 | Views: 986

Author(s)

Hamed Ghaemi 1

Affiliation(s)

1 Bahar Institute of Higher Education, Mashhad, Iran

Corresponding Author

Hamed Ghaemi

ABSTRACT

Differential Item Functioning (DIF), which is a statistical feature of an item and provides a sign of unpredicted performance of items on a test, occurs once dissimilar groups of test takers with the same level of ability show different performance on a single test. The aim of this paper was to examine DIF on the Pearson Test of English (PTE) test items. To that end, 250 intermediate EFL learners with the age range of 26 - 36 in two different fields of study (125, Engineering, and 125 Sciences) were randomly chosen for the analysis. The Item Response Theory (IRT) Likelihood Ratio (LR) approach was utilized to find items showing DIF. The scored items of 250 PTE test takers were analyzed using the IRT three-parameter model utilizing item difficulty (b parameter), item discrimination (a parameter), and pseudo-guessing (c parameter). The results of the independent samples t-test for comparison of means in two groups depicted that Science participants performed better than the Engineering ones particularly in Speaking & Writing and Reading sections. It is evident that the PTE test was statistically easier for the Science students at 0.05 level. Linguistic analyses of Differential Item Functioning items also confirmed the findings of the quantitative part, indicating a far better performance on the part of Science students.

KEYWORDS

Differential Item Functioning (DIF); Item Response Theory (IRT); Likelihood Ratio Approach (LR); Fields of Study, Pearson Test of English

CITE THIS PAPER

Hamed Ghaemi, Assessing Differential Item Functioning (DIF) For Pearson Test of English (PTE)-A study of Test Takers with Different Fields of Study. Applied & Educational Psychology (2022) Vol. 3: 17-32. DOI: http://dx.doi.org/10.23977/appep.2022.030103.

REFERENCES

[1] Angoff, W. H. (1993). Perspectives on Differential Item Functioning methodology. NJ: Lawrence Erlbaum.
[2] Aryadoust, V., & Zhang, L. (2015). Fitting the mixed Rasch model to a reading comprehension test: Exploring individual difference profiles in L2 reading. Language Testing, 33(4), 529- 553. doi:10.1177/0265532215594640
[3] Baker, F. B. (1985). The basics of item response theory. NH: Heinemann.
[4] Carlton, S. T., & Harris, A.M. . (1992). Characteristics associated with Differential Item Functioning on the Scholastic Aptitude Test: gender and majority /minority group  comparisons. ETS Research Report, 92–64. 
[5] Chen, Y.-F., & Jiao, H. (2014). Exploring the utility of background and cognitive variables in explaining latent differential item functioning: An example of the PISA 2009 reading assessment. Educational Assessment, 19, 77-96. 
[6] Chen, Z., & Henning, G. . (1985). Linguistic and cultural bias in language proficiency tests. Language Testing, 2, 155–163. 
[7] Clauser, B. E., & Mazor, K.M. . (1990). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31–47. 
[8] De Beer, M. (2004). Use of differential item functioning (DIF) analysis for bias analysis in test construction. SA Journal of Industrial Psychology, 30(4), 52-58. 
[9] Ertuby, C., & Russel, R.J.H. . (1996). Dealing with comparability problem of cross-cultural data. Paper presented at the 26th International Congress of Psychology, Montreal.
[10] Federer, M. R., Nehm, R. H., & Pearl, D. K. (2016). Examining Gender Differences in Written Assessment Tasks in Biology: A Case Study of Evolutionary Explanations. CBE life Sciences education, 15(1), ar2-ar2. doi:10.1187/cbe.14-01-0018
[11] Grabe, W. (2009). Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press.
[12] Hambleton, R. K. (1996). Guidelines for Adapting Educational and Psychological Tests European Journal of Psychological Assessment. 
[13] Hambleton, R. K., Swaminathan, H., & Rogers, H.J. . (1991). Fundamentals of item response theory. CA: Sage Publications.
[14] Hatch, E. M., & Farhady, H. . (1982). Research design and statistics for applied linguistics.Tehran Rahnama Publications.
[15] Hong, S., & Min, S.-Y. (2007). Mixed Rasch Modeling of the Self-Rating Depression ScaleIncorporating Latent Class and Rasch Rating Scale Models. Educational and Psychological Measurement - EDUC PSYCHOL MEAS, 67, 280-299. doi:10.1177/0013164406292072
[16] Koda, K. (2005). Insights into second language reading: A cross-linguistic approach. New York: Cambridge University Press.
[17] Koo, J., Becker, B. J., & Kim, Y. S. (2014). Koo, J., Becker, B. J., & Kim, Y. S. (2014). Examining differential item functioning trends for English language learners in a reading test: A meta-analytical approach. Language Testing, 31(1), 89-109. 
[18] Lawrence, I. M., & Curley, W.E. (1989). (1989). Differential Item Functioning for males and females on SAT-Verbal Reading subscore items: follow-up study. Educational Testing Service Research Report, 89–22. 
[19] Lawrence, I. M., Curley, W. E., & McHale, F. J. (1988). Differential item functioning for males and females on SAT verbal reading subscore items. New York: College Entrance Examination Board.
[20] Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. NJ.: Lawrence Erlbaum Assoc.
[21] Mantel, N., & Haenszel, M.W. (1959). Statistical aspects of thee analysis of data from retrospective studies of disease. J Nat Cancer Inst., 22, 719-748. 
[22] McBride, J. R. (1997). Technical Perspective. American Psychological Association, 29-44. 
[23] Osterlind, S. J. (1983). Test item bias. Beverly Hills: Sage.
[24] Owen, K. (1998). The Role of Psychological Tests in Education in South Africa [microform]: Issues, Controversies and Benefits / K. Owen. [Washington D.C.]: Distributed by ERIC Clearinghouse.
[25] Ownby, R. L., & Waldrop-Valverde, D. (2013). Differential item functioning related to age in the reading subtest of the test of functional health literacy in adults. Journal of aging research, 2013, 654589-654589. doi:10.1155/2013/654589
[26] Raju, N. S. (1990). Determining the Significance of Estimated Signed and Unsigned Areas Between Two Item Response Functions. Applied Psychological Measurement, 14(2), 197-207. doi:10.1177/014662169001400208
[27] Ryan, K. E., & Bachman, L. F. (1992). Differential item functioning on two tests of EFL proficiency. Language Testing, 9(1), 12-29. doi:10.1177/026553229200900103
[28] Schmitt, A., & Dorans, N. . (1990). Differential item functioning for minority examinees on the SAT. Journal of Educational Measurement, 27, 67–81. 
[29] Smith, M. U., Snyder, S. W., & Devereaux, R. S. . (2016). The GAENE—Generalized Acceptance of EvolutioN Evaluation: development of a new measure of evolution acceptance. Journal of Research in Science Teaching, 53, 1289–1315. 
[30] Thissen, D. (1991). MULTILOG (Version 6.30) [Computer Software]. Chicago, IL: Scientific Software.
[31] Thissen, D., Steinberg, L., & Wainer, H. (1994). Detection of differential item functioning using the parameters of item response models. NJ: Lawrence Erlbaum.
[32] Thissen, D., Steinberg, L., & Wainer, H. . (1988). Use of item response theory in the study of group differences in trace lines. NJ: Erlbaum.
[33] Tittle, C. K. (1982). Use of judgmental methods in item bias studies. MD: Johns Hopkins University Press.
[34] Van de Vijver, F. (1998). Multicultural assessment: How suitable are Western tests? European Journal of Psychological Assessment, 14(1), 61. 
[35] Zumbo, B. D. (2007). Three Generations of DIF Analyses: Considering Where It Has Been, Where It Is Now, and Where It Is Going. Language Assessment Quarterly, 4(2), 223-233. doi:10.1080/15434300701375832.

All published work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2016 - 2031 Clausius Scientific Press Inc. All Rights Reserved.