The Factor Structure of a Written English Proficiency Test: A Structural Equation Modeling Approach

Document Type : Research Paper

Authors

University of Tehran

Abstract

The present study examined the factor structure of the University of Tehran English Proficiency Test (UTEPT) that aims to examine test takers’ knowledge of grammar, vocabulary, and reading comprehension. A Structural Equation Modelling (SEM) approach was used to analyse the responses of participants (N= 850) to a 2010 version of the test.  A higher-order model was postulated to test if the underlying factor structure, obtained in a data-driven manner, corresponds with the proposed structure of the test. The results revealed an appropriate model fit with the data, pointing to the fact that the three sections of UTEPT, i.e., structure, vocabulary, and reading, and their sub-components, except for the restatement section of reading, are good indicators of written language proficiency as assessed by the UTEPT. It was also found that the three sections assess distinctive constructs. The findings suggest that UTEPT is a valid measure of the written language proficiency of Ph.D. applicants to University of Tehran.

Keywords

References

Alderson, J. C. (1981). Report of the discussion on general language proficiency, In J. C. Alderson & A. Hughes (Eds.), Issues in language testing, ELT Documents 111. London: The British Council.
Alderson, J. C. (1986). Innovations in language testing. In M. Portal (Ed.), Innovations in Language Testing. Windsor: NFRE-Nelson.
Alderson, J. C.  (1991). Language testing in the 1990s: How far have we come? How much further have we to go?. In J.­ C. Anivan (Ed.), Current developments in language testing (pp. 1-19). Singapore: SEAMEO Regional Language Center.
Alderson, J. C. (2000). Assessing reading. Cambridge, UK: Cambridge University Press.
Anivan, S. (Ed.). (1991).Current developments in language testing (pp. 1-19). Singapore: SEAMEO Regional Language Center.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bachman, L. F., Davidson, F., Ryan, K., & Choi, I. C. (1995). An investigation into the comparability of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. Cambridge: Cambridge University Press.
Bachman, L. F., & Palmer, A. (1981). The construct validation of the FSI oral interview. Language Learning, 31, 67–86.
Bachman, L. F., & Palmer, A. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16, 449-465.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.
Bae, J., & Bachman, L. F. (1998). A latent variable approach to listening and reading: Testing factorial across two groups of children in the Korean/English two-way immersion program. Language Testing, 15(3), 380-414.
Bentler, P. M., & Yuan, K. H. (1999). Structural equation modelling with small samples: Test statistics, Multivariate Behavioral Research, 34, 181–197.
Byrne, B. M. (2010). Structural equation modelling with AMOS: Basic concepts, applications, and programming (2nd ed.). Routledge: Taylor & Francis Group.
Canale, M., & Swain, M. (1980).Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1-47.
Celce-Murcia, M., Dornyei, M. Z., & Thurrell, S. (1995). ­Communicative competence: A pedagogical motivated model with content specifications. Issues in Applied Linguistics, 6(2), 5-35.
Carroll, J. B. (1993). Human cognitive ability: A survey of factor analytic studies. Cambridge University Press.
Chapelle, C., Enright, M. K., & Jamieson, J. (Eds.). (2008). Building a validity argument for the test of English as a foreign language. New York: Routledge.
Cummins, J. (1984). Wanted: A theoretical framework for relating language proficiency to academic achievement among bilingual students. In C. Rivera (Ed.), Language proficiency and academic achievement. Avon, England: Multilingual Matters Ltd.
Eckes, T., & Grotjahn, R. (2006).A closer look at the construct validity of C-tests. Language Testing, 23 (3), 290-325.
ETS. (2002). LanguEdge courseware score interpretation guide. Princeton, NJ:
Farhady, H. (1983). On the plausibility of the unitary language proficiency factor. In J. W. Oller (Ed.), Issues in language testing research (pp.11-29). Rowley, Mass: NewburyHouse
Farhady, H. (2005). Language assessment: A Linguametric perspective. System, 2(2), 147-164.
Farhady, H., & Abbassian, G. R. (2000). Test method, level of language proficiency and the underlying structure of language ability.  Alzahra Journal, 9(29), 27-32.
Hoe, S. L. (2008). Issues and procedures in adopting structural equation modeling technique. Journal of Applied Quantitative Methods, 3(1), 76-83.
Hughes, A., & Porter, D. (1983). Current developments in language testing. UK: University of Reading, 3-37.
In’nami, Y., & Koizumi, R. (2011). Factor structure of the revised TOEIC test: A multiple-sample analysis. Language Testing, 29(1), 131–152.
Kunnan, A. J. (1995).Test taker characteristics and test performance: A structural modeling approach. Cambridge: University of Cambridge Local Examinations Syndicate and Cambridge University Press.
Milanovic, M. (1988). The construct validation of a performance based battery of English language progress test. Unpublished PhD dissertation, University of London.
Oller, J. W., Jr. (1978). The language factor in the evaluation of bilingual education. In J. Alatis (Ed.), International dimension of bilingual education. Washington, D. C.: Georgetown University Press.
Oller, J. W., Jr. (1983). A consensus for the eighties?. In J. W. Oller, Jr. (Ed.), Issues in language testing research (pp. 351–356). Rowley, MA: Newbury House.
Oller, J. W. Jr., & Damico, J. S. (1991). Theoretical considerations in the assessment of  LEP students. In E. Hamayan & J. S. Damico (Eds.), Limiting bias in the assessment of bilingual students. Austin: Pro-ed publications.
Oller, J. W., Jr., & Hinofotis, F. A. (1980). Two mutually exclusive hypotheses about second language ability: Factor analytic studies of a variety of language subtests. In J. W. Oller, Jr., & K. Perkins (Eds.), Research in language testing (pp. 13–23). Rowley, MA: Newbury House.
Pasternak, M., & Bailey, K. M. (2004). Preparing nonnative and native english-speaking teachers: Issues of professionalism and proficiency. In L. D. Kamhi-Stein (Ed.), Learning and teaching from experience: Perspectives on nonnative English-speaking professionals (pp. 1 55-175). Ann Arbor: The University of Michigan Press.
Sasaki, M. (1996). Second language proficiency, foreign language aptitude, and intelligence: Quantitative and qualitative analyses. New York, NY: Peter LangPublishing, Inc.
Sawaki, Y., Stricker, L. J., & Oranje, A. H. (2009).  Factor structure of the TOEFL Internet-Based test. Language Testing, 26(1), 005- 030.
Schroeders, U., Wilhelm, O., & Bucholtz, N. (2010). Reading, listening, and viewing comprehension in English as a foreign language: One or more constructs? Intelligence, 38, 562–573.
Shin, S. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22(1), 31-57.
Song, M. Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modelling approach. Language Testing, 25(1), 435-464.
Spolsky, B. (1973). What does it mean to know a language, or how do you get someone to perform his competence? In J. W. Oller & J.C. Richards (Eds.), Focus on the learner: Pragmatic perspectives for the language teacher, Rowley, Mass.: Newbury House.
Stricker, L. J., Rock, D. A., & Lee, Y. (2005). Factor structure of the languedge™ test across language groups, ETS, Princeton, NJ.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed). Pearson: Allyn and Bacon.
Vecchio, A. D., & Guerrero, M. (1995). Handbook of English language proficiency
tests. EAC-West, New Mexico Highlands University, Albuquerque.
Vollmer, H. J. (1983). The structure of foreign language competence. In A. Hughes & D. Porter (Eds.), Current developments in language testing (pp.3-30). UK: University of Reading.
Vollmer, H. J., & Sang, F. (1983). Competing hypotheses about second language ability: A plea for caution. In Oller, J. W., Jr. (Ed.), Issues in language testing research (pp. 29–79). Rowley, MA: Newbury House.
Zhang, B. (2010). Assessing the accuracy and consistency of language proficiency classification under competing measurement models. Language Testing, 27(1), 119 - 140.
 

Files

Share

How to cite

Statistics