ILETS Test Evaluation

ILETS Test Evaluation IELTS The International English Language Testing System The purpose of any test is to diagnose what the learners know or dont know, in other words to check learning. The International English Language Testing System or IELTS test is a language proficiency test which covers all four language skills reading, writing, listening and speaking and is the most widely used and accepted test for English language. Each paper holds a percentage of the final score each equally weighted at 25%. The IELTS test is unique in that you cant fail it (Takeielts.britishcouncil.org, 2017). Although we live in a very more modern, electronic world, the IELTS test is still a pen and paper-based test (Takeielts.britishcouncil.org, 2017). The papers/tests covering the four skills are: Reading a 60 minute test which includes three texts to be read Writing a 60 minute test showing the test takers ability to summarise, explain or describe Listening this test is a 30 minute test Speaking this is the shortest of all four tests at 11-14 minutes. My essay will be focusing on the evaluation and the validity or fitness for purpose of the speaking test within the IELTS test. Michael Halliday (1968 cited in Levine and Spencer, 1996, p. 46) quotes; Language is a form of culturally determined behaviour and this behaviour includes the ability to take on a range of linguistically defined roles in speech situations. Unless the child grows up in an environment in which all these speech situation roles are open to him, he will fail to master important areas in the grammar of his language. With this is mind we must understand that all types of tasks have both advantages and limitations (or disadvantages) so it is necessary to ascertain the Fitness for purpose of a particular task and to use a different range of task types within the framework. We must provide different speaking tasks which activate our different speaking processes. The tasks should elicit behaviour which truly represents the candidates ability and which can be scored validly and reliably (Hughes, 1991, pg. 101). Although speaking can seem both simple and natural it is, in fact, a very complex skill with several stages to follow. From the conceptualisation of an idea we must then convert that idea into language. This is done through both lexical and grammatical knowledge. Through this knowledge we are able to form sentences and phrases in order to vocalise our idea. Once we have the essential words, sentences and phrases these are then changed into sound or articulated. This is how a non-verbal idea becomes an actual verbal expression. As you will know, when we speak we are constantly monitoring ourselves in order to ensure the flow of speech continues smoothly, whether this be through our internal access to vocabulary, grammar or intonation, etc. When we s peak all of these stages are completed automatically and naturally this is why speaking can be so challenging for learners. If a learner lacks both lexical and analytical knowledge their retrieval of that vocabulary or grammar may well take longer causing a less natural or automatic response to a task or question through longer pauses and hesitations. We must then ensure that testing incorporates all manner of validity and reliability in order to produce a rounded and more thorough outcome. The IELTS speaking test is a face-to-face, one-on-one interview between the test taker and the IELTS trained examiner. The interview only lasts between 11 to 15 minutes and is recorded. The speaking test can be taken before the other three sections listening, reading, writing of the test and the speaking comprises of three separate sections. The first section or part 1 is known as the introduction and interview segment. Here the interviewer introduces him or herself to the test taker. This is then continued as if it were an interview albeit a little formal whereby the test taker shares information with the examiner. The examiner will have a script of questions to ask the test taker which cover every day familiar topics e.g. work, study, where you live, food, holidays, friends, goig out, festivals, sports, school and public transport (TeachingEnglish | British Council | BBC, 2017). The test taker provides the examiner with information about themselves, their family, work and interests through a series of open-ended questions like those in appendix 1 e.g. Whats the most interesting part of your village/town? Would you say its a good place to live? Why? (Ieltsessentials.com, 2017) and answers them to the best of their ability. This section usually takes around 4-5 minutes. The second section or part 2 is known as the individual long turn. In this segment the candidate or test taker is given a topic card (see appendix 2). The topic card has a subject on it which the test taker must talk about for two minutes. Before speaking, the candidate is allowed one minute in order to prepare notes on their given topic. These topic tasks are generally about a personal experience, for example, a memorable day or a person of great significance to them and they then have to produce a monologue on this topic. Once the candidate has completed their monologue the examiner will then ask them some follow up questions which then leads them into part 3 of the test. The third section or part 3 is known as the two-way discussion. This two-way discussion or dialogue is between the test taker and the examiner and is generally thematically linked (Karim and Haq, 2014) to part 2. If you look at appendices 2 and 3 you will see that the candidate in part 2 spoke about a piano he once owned and in part 3 the examiner then asks to have a discussion regarding things we own. Interviewing is a tried and tested way of assessing a language learners speaking capabilities, however, it can seem intensely formal to the interviewee and cause the interaction to be dominated by the examiner (Karim and Haq, 2014) in their role as interviewer in the IELTS test. In the test then, this tends, to lend itself toward a situation in which the speaking is less natural in form than it could be if it were in a different format e.g. an informal discussion over coffee. Hughes (1991) states the most obvious format for the testing of oral interaction is the interview, however, it has at least one potentially serious drawback. When the test taker feels they are in an interview situation they will speak to the examiner as if speaking to a superior (Hughes, 1991, pg.104). Therefore, the speaking confidence of the test taker may be put to a disadvantage. This limitation may well be avoided should the candidate feel they are able to ask questions as well as the examiner. Through a re laxed dialogue rather than an interview the conversation could well take a more natural path providing more confidence for the test taker. Also, interviews can cause unnecessary anxiety and nervousness to the test taker which could limit their interaction. We could also argue that natural conversation is built up with more informal language, whereby a conversation is not scripted but flows along an unseen path to an unseen destination. Conversation does not follow a strict or distinct route but is nudged along in many different directions. The question we are asking is Is the test successful in achieving its objectives? and for this we need to look at what is known as the validity and reliability of a test. Validity has several forms and I have looked at content, face and criterion-related validity. Content validity is a representative sample of the skills and abilities which have been taught. The structure and content of the IELTS speaking test is the same at all levels. This means that an elementary level candidate will be answering the same questions of those at advanced levels. This does not provide a concrete level of testing as the structure and content do not provide a more challenging course for those with more advanced skills. Face validity looks at the measurement of skill the test should be assessing. If we look at the format of the IELTS test we can see that it is clearly established. It is universal in its approach and anywhere you look for information on the IELTS test it always provides the same information. In contrast both IELTS academic and General English tests are no different with regards to what is tested for in the speaking section. If we look at the band descriptor (appendix 4) we can see that there is no distinction for specific English. This brings into question the validity of criterion-based validity. The reliability of the IELTS test must also be looked at. A single examiner is responsible for rating the candidate, therefore, the inter-rater reliability has to be questioned due to interpretation but a single person of the test taker. The scoring is split between four categories (appendix 4) within these four categories there is no in-depth breakdown so scoring is at the discretion of the examiner. The effectiveness of the IELTS speaking test has some limitations. Overall it is a meaningful test which shows validity and reliability in some areas but there are areas in which both validity and reliability fall short as has been shown. The following suggestions could make the IELTS speaking test somewhat more valid and reliable provide more time for the candidate to talk freely in the first section. In this way we could elicit more authentic data from the candidate and be provided with a more in-depth analysis of the actual ability the candidate has. More tasks would elicit a better performance from the candidate discussion about a picture or tasks which represent different communicative processes. The grading scale should be open to more variables to provide a broader perspective and finally, more than one examiner would ensure a more reliable assessment. APPENDICES APPENDIX 1 Speaking sample task Part 1 https://www.ieltsessentials.com/global/-/media/IELTS/Global/Files/MP3s/speakingsamplepart1recording.ashx?la=en Part 1 Introduction and interview [This part of the test begins with the examiner introducing himself or herself and checking the candidates identification. It then continues as an interview.] Lets talk about your home town or village. What kind of place is it? Whats the most interesting part of your town/village? What kind of jobs do the people in your town/village do? Would you say its a good place to live? (Why?) Lets move on to talk about accommodation. Tell me about the kind of accommodation you live in? How long have you lived there? What do you like about living there? What sort of accommodation would you most like to live in? Speaking sample task Part 1 transcript Part 1 Introduction and interview [This part of the test begins with the examiner introducing himself or herself and checking the candidates identification. It then continues as an interview.] Transcript Examiner: Now, in this first part, Id like to ask you some more questions about yourself, OK? Lets talk about your home town or village. What kind of place is it? Candidate: Its quite a small village, about 20km from Zurich. And its very quiet. And we have only little two little shops because most of the people work in Zurich or are orientated to the city. Examiner: Whats the most interesting part of this place village? Candidate: On the top of a hill we have a little castle which is very old and quite well known in Switzerland. Examiner: What kind of jobs do people in the village do? Candidate: We have some farmers in the village as well as people who work in Zurich as bankers or journalists or there are also teachers and some doctors, some medicines. Examiner: Would you say its a good place to live? Candidate: Yes. Although it is very quiet, it is à ¢Ã¢â€š ¬Ã‚ ¦ people are friendly and I would say it is a good place to live there, yes. Examiner: Lets move on to talk about accommodation. Tell me about the kind of accommodation you live in APPENDIX 2 Speaking sample task Part 2 https://www.ieltsessentials.com/global/-/media/IELTS/Global/Files/MP3s/speakingsamplepart2recording.ashx?la=en Part 2 Individual long turn Candidate Task Card Describe something you own which is very important to you. You should say: where you got it from how long you have had it what you use it for and explain why it is important to you. You will have to talk about the topic for 1 to 2 minutes. You have one minute to think about what youre going to say. You can make some notes to help you if you wish. Rounding off questions Is it valuable in terms of money? Would it be easy to replace? Speaking sample task Part 2 transcript Part 2 Individual long turn Transcript Examiner: Alright? Remember you have one to two minutes for this, so dont worry if I stop you. Ill tell you when the time is up. Candidate: OK Examiner: Can you start speaking now, please? Candidate: Yes. One of the most important things I have is my piano because I like playing the piano. I got it from my parents to my twelve birthday, so I have it for about nine years, and the reason why it is so important for me is that I can go into another world when Im playing piano. I can forget whats around me and what I can forget my problems and this is sometimes quite good for a few minutes. Or I can play to relax or just, yes to à ¢Ã¢â€š ¬Ã‚ ¦ to relax and to think of something completely different. Examiner: Thank you. Would it be easy to replace this, this piano? Candidate: Yes, I think it wouldnt be that big problem but I like my piano as it is because I have it from my parents, its some kind unique for me. APPENDIX 3 Speaking sample task Part 3 https://www.ieltsessentials.com/global/-/media/IELTS/Global/Files/MP3s/speakingsamplepart3recording.ashx?la=en Part 3 Two-way discussion Lets consider first of all how peoples values have changed. What kind of things give status to people in your country? Have things changed since your parents time? Finally, lets talk about the role of advertising. Do you think advertising influences what people buy? Speaking sample task Part 3 transcript Part 3 Two-way discussion Transcript Examiner: Weve been talking about things we own. Id like to discuss with you one or two more general questions relating to this topic. First, lets consider values and the way they can change. In Switzerland, what kind of possessions do you think give status to people? Candidate: The first thing which comes in my mind is the car. Yes, because lots of people like to have posh cars or expensive cars to show their status, their place in the society. Examiner: Is that a new development? Candidate: No, I think it isnt. Examiner: People have thought like that for quite a long time? Candidate: Yes. Another thing is probably the clothing. It starts already when you are young. When the children go to school they want to have posh labels on their jumpers or good shoes. Examiner: What do you think of this way of thinking that I need to have a car or certain clothes to show my status? Candidate: Probably its sometimes a replacement for something you dont have, so if your wife has left you or your girlfriend, you just buy some new, I dont know, new watches or new clothes to make you satisfied again. Examiner: You dont think of it as a healthy way of thinking? Candidate: Its probably not honest to yourself. You can understand what I mean? Examiner: Yes. And do you think this will change? In the future, will cars and designer clothes be status symbols in the same way? Candidate: Im sure that clothes will be that the thing with the clothes will be the same. Im not so sure about the cars because cars cause lots of environmental problems and probably in some years, a few years, this will change because its not reasonable to drive a car anymore. Examiner: Can you tell me a little bit more about that? APPENDIX 4 IELTS Speaking Band Descriptors Band Fluency and coherence Lexical resource Grammatical range and accuracy Pronunciation 9 speaks fluently with only rare repetition or self-correction; any hesitation is content-related rather than to find words or grammar speaks coherently with fully appropriate cohesive features develops topics fully and appropriately uses vocabulary with full flexibility and precision in all topics uses idiomatic language naturally and accurately uses a full range of structures naturally and appropriately produces consistently accurate structures apart from slips characteristic of native speaker speech uses a full range of pronunciation features with precision and subtlety sustains flexible use of features throughout is effortless to understand 8 speaks fluently with only occasional repetition or self- correction; hesitation is usually content-related and only rarely to search for language develops topics coherently and appropriately uses a wide vocabulary resource readily and flexibly to convey precise meaning uses less common and idiomatic vocabulary skilfully, with occasional inaccuracies uses paraphrase effectively as required uses a wide range of structures flexibly produces a majority of error-free sentences with only very occasional inappropriacies or basic/non-systematic errors uses a wide range of pronunciation features sustains flexible use of features, with only occasional lapses is easy to understand throughout; L1 accent has minimal effect on intelligibility 7 speaks at length without noticeable effort or loss of coherence may demonstrate language-related hesitation at times, or some repetition and/or self-correction uses a range of connectives and discourse markers with some flexibility uses vocabulary resource flexibly to discuss a variety of topics uses some less common and idiomatic vocabulary and shows some awareness of style and collocation, with some inappropriate choices uses paraphrase effectively uses a range of complex structures with some flexibility frequently produces error-free sentences, though some grammatical mistakes persist shows all the positive features of Band 6 and some, but not all, of the positive features of Band 8 6 is willing to speak at length, though may lose coherence at times due to occasional repetition, self-correction or hesitation uses a range of connectives and discourse markers but not always appropriately has a wide enough vocabulary to discuss topics at length and make meaning clear in spite of inappropriacies generally paraphrases successfully uses a mix of simple and complex structures, but with limited flexibility may make frequent mistakes with complex structures though these rarely cause comprehension problems uses a range of pronunciation features with mixed control shows some effective use of features but this is not sustained can generally be understood throughout, though mispronunciation of individual words or sounds reduces clarity at times 5 usually maintains flow of speech but uses repetition, self correction and/or slow speech to keep going may over-use certain connectives and discourse markers produces simple speech fluently, but more complex communication causes fluency problems manages to talk about familiar and unfamiliar topics but uses vocabulary with limited flexibility attempts to use paraphrase but with mixed success produces basic sentence forms with reasonable accuracy uses a limited range of more complex structures, but these usually contain errors and may cause some comprehension problems shows all the positive features of Band 4 and some, but not all, of the positive features of Band 6 4 cannot respond without noticeable pauses and may speak slowly, with frequent repetition and self-correction links basic sentences but with repetitious use of simple connectives and some breakdowns in coherence is able to talk about familiar topics but can only convey basic meaning on unfamiliar topics and makes frequent errors in word choice rarely attempts paraphrase produces basic sentence forms and some correct simple sentences but subordinate structures are rare errors are frequent and may lead to misunderstanding uses a limited range of pronunciation features attempts to control features but lapses are frequent mispronunciations are frequent and cause some difficulty for the listener 3 speaks with long pauses has limited ability to link simple sentences gives only simple responses and is frequently unable to convey basic message uses simple vocabulary to convey personal information has insufficient vocabulary for less familiar topics attempts basic sentence forms but with limited success, or relies on apparently memorised utterances makes numerous errors except in memorised expressions shows some of the features of Band 2 and some, but not all, of the positive features of Band 4 2 pauses lengthily before most words little communication possible only produces isolated words or memorised utterances cannot produce basic sentence forms Speech is often unintelligble 1 no communication possible no rateable language 0 does not attend REFERENCES Hughes, A. (1991). Testing for language teachers. 1st ed. Cambridge [England]: Cambridge University Press. Ieltsessentials.com. (2017). Speaking Practice Tests. [online] Available at: https://www.ieltsessentials.com/global/prepare/freepracticetests/speakingpracticetests [Accessed 01 Mar. 2017]. Karim, S. and Haq, N. (2014). An Assessment of IELTS Speaking Test. International Journal of Evaluation and Research in Education (IJERE), 3(3). Levine, J. and Spencer, M. (1996). Developing pedagogies in the multilingual classroom. 1st ed. Stoke-on-Trent: Trentham Books. Takeielts.britishcouncil.org. (2017). IELTS teachers questions answered | Take IELTS. [online] Available at: http://takeielts.britishcouncil.org/teach-ielts [Accessed 7 Feb. 2017]. TeachingEnglish | British Council | BBC. (2017). Evaluating speaking the IELTS speaking test. [online] Available at: https://www.teachingenglish.org.uk/article/evaluating-speaking-ielts-speaking-test [Accessed 07 Mar. 2017].