English and Persian Undergraduate Students’ Perceptions of the Construct-(ir)Relevance of Language Proficiency in the Assessment of Literary Competence

Document Type : Research Paper

Authors

Shahid Chamran University of Ahvaz

Abstract

Of the many dilemmas facing the assessment of literary competence, one is the extent to which language should constitute part of the target construct intended to be measured. Some argue for the construct-irrelevance of language and hence recommend that it be eliminated or minimized in favor of an exclusive focus on literary competence. In practice, this does not seem to be the case, as language proficiency considerations seem to creep into assessment, clouding assessment outcomes. The current study sought to examine students' perceptions of the degree to which knowledge of language constitutes part of the construct of literary competence in two departments of English and Persian literature. To this end, a total of seventy students in two poetry courses, one in the English department and the other in the Persian, responded to a questionnaire designed to gauge their perceptions of the extent to which language competence constitutes a component of the literary competence. Data were analyzed through one sample and independent samples t-tests. It was found that language competence is somehow construct-irrelevant in testing literary competence. Interestingly, measurement-invariance was observed regarding Persian and EFL students’ stance on the construct-irrelevance of language in tests of literary achievement and competence.   

Keywords

References

Alderson, J. C., & Wall, D. (1993). Does washback exist?. Applied Linguistics, 14(2), 115-129.
Bachman, L. F. (2004). Statistical analysis for language assessment. Oxford: Oxford University Press.
Beach, R.  (2014). Assessing responses to literature. In A. J. Kunnan (Ed).  The Companion to Language Assessment (pp. 85-101). John Wiley & Sons, Inc. DOI: 10.1002/9781118411360.wbcla017
Boone, H. N., & Boone, D. A. (2012). Analyzing likert data. Journal of Extension, 50(2), 1-5.
Brunfaut, T. (2014). Language for specific purposes: Current and future issues. Language Assessment Quarterly, 11(2), 216-225.
Brown, J. D. (2014). Classical theory reliability. In A. Kunnan (Ed). The companion to language testing (pp. 1149-1166). New York: Routledge.
Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. TESOL Quarterly, 32(4), 653-675.
Cooper, R. C. (1971). Measuring appreciation of literature: A review of attempts. Research in the Teaching of English, 5(1), 5-23.
Douglas, D. (2010). Understanding language testing. London: Hodder Education.
Douglas, D. (2001). Three problems in testing language for specific purposes: Authenticity, specificity and inseparability. In C. Elder, A. Brown, E. Grove, K. Hill, N. Iwashita, T. Lumley, . . . & K. O’Loughlin (Eds.), Experimenting with uncertainty: Essays in honour of Alan Davies (pp. 45–52). Cambridge, England: Cambridge University Press.
Freeman, D., & Johnson, K. E. (1998). Reconceptualizing the knowledge‐base of language teacher education. TESOL Quarterly, 32(3), 397-417.
Fulcher, G., & Davidson, F. (2008). Language testing and assessment: An advanced resource book. New York, NY: Routledge.
Gaston, P. L. (1991). Measuring the Marigolds: Literary studies and the opportunity of outcomes assessment. The Journal of the Midwest Modern Language Association, 24(2), 11-20.
Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM). 2nd Edition. Thousand Oaks: Sage. 
Hatch, E. M., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. New York, NY: Newbury House Publishers.
Kadhim, H. M. (2015). Testing English literature at the university level in Iraq. Unpublished M.A thesis. University of Thi-Qar.
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1–55.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241-256.
Miall, S. D., &Kuiken, D. (1995). Aspects of response to literature: A new questionnaire. Research in the Teaching of English, 29(1), 37-58.
Paran, A. (2010). Between Scylla and Charybdis: The dilemmas of testing language and literature. In A. Paran, & L. Sercu (Eds). Testing the Untestable in Language Education (pp.143-165). Bristol: Multilingual Matters.
Poehner, M. E., & Lantolf, J. P. (2004). Dynamic assessment in language classroom. Language Teaching Research, 9(3), 233-265.
Purves, C. A. (1986). ERIC/CRC report: Testing in literature. Language Arts, 63(3), 320-323.
Purves, C. A. (1979). Evaluation of learning in literature. Evaluation in Education, 3(2), 93-172.
Purves, A. (1967). Literary criticism, testing, and the English teacher. College English, 28(4), 310-313.
Shohamy, E. (2014). The power of tests: A critical perspective on the uses of language tests. Routledge: New York.
Shohamy, E. (2001). Democratic assessment as an alternative. Language Testing, 18(4), 373-391.
Tseng, W.-T., Dörnyei, Z., & Schmitt, N. (2006). A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition. Applied Linguistics, 27(1), 78-102.
Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and from innovation theory. Language Testing, 13(3), 334-354.
Xie, Q. (2011). Is test takers’ perception of assessment demand related to construct validity? International Journal of Testing, 11(4), 324-348.
Xie, Q. & Andrews, S. (2013). Do test design and uses influence test preparation? Testing a model of washback with structural equation modeling. Language Testing, 30(1), 49-70.
Iranian Journal of Applied Language Studies
Volume 9, Issue 2
November 2017
Pages 83-110

Files

Share

How to cite

Statistics