ILETS Test Evaluation
IELTS – The International English Language Testing System
The purpose of any test is to diagnose what the learners know or don’t know, in other words to check learning. The International English Language Testing System or IELTS test is a language proficiency test which covers all four language skills – reading, writing, listening and speaking and is the most widely used and accepted test for English language. Each paper holds a percentage of the final score – each equally weighted at 25%. The IELTS test is unique in that you can’t fail it (Takeielts.britishcouncil.org, 2017). Although we live in a very more modern, electronic world, the IELTS test is still a “pen and paper-based” test (Takeielts.britishcouncil.org, 2017).
The papers/tests covering the four skills are:
Reading – a 60 minute test which includes three texts to be read
Writing – a 60 minute test showing the test takers ability to summarise, explain or describe
Listening – this test is a 30 minute test
Speaking – this is the shortest of all four tests at 11-14 minutes.
My essay will be focusing on the evaluation and the validity or fitness for purpose of the speaking test within the IELTS test.
Michael Halliday (1968 cited in Levine and Spencer, 1996, p. 46) quotes;
Language is a form of culturally determined behaviour and this behaviour includes the ability to take on a range of linguistically defined roles in speech situations. Unless the child grows up in an environment in which all these speech situation roles are open to him, he will fail to master important areas in the grammar of his language.
With this is mind we must understand that all types of tasks have both advantages and limitations (or disadvantages) so it is necessary to ascertain the “Fitness for purpose” of a particular task and to use a different range of task types within the framework.
We must provide different speaking tasks which activate our different speaking processes. The tasks should elicit behaviour which truly represents the candidates’ ability and which can be scored validly and reliably (Hughes, 1991, pg. 101). Although speaking can seem both simple and natural it is, in fact, a very complex skill with several stages to follow. From the conceptualisation of an idea we must then convert that idea into language. This is done through both lexical and grammatical knowledge. Through this knowledge we are able to form sentences and phrases in order to vocalise our idea. Once we have the essential words, sentences and phrases these are then changed into sound or articulated. This is how a non-verbal idea becomes an actual verbal expression. As you will know, when we speak we are constantly monitoring ourselves in order to ensure the flow of speech continues smoothly, whether this be through our internal access to vocabulary, grammar or intonation, etc. When we speak all of these stages are completed automatically and naturally this is why speaking can be so challenging for learners. If a learner lacks both lexical and analytical knowledge their retrieval of that vocabulary or grammar may well take longer causing a less natural or automatic response to a task or question through longer pauses and hesitations.
We must then ensure that testing incorporates all manner of validity and reliability in order to produce a rounded and more thorough outcome.
The IELTS speaking test is a face-to-face, one-on-one interview between the test taker and the IELTS trained examiner. The interview only lasts between 11 to 15 minutes and is recorded. The speaking test can be taken before the other three sections – listening, reading, writing – of the test and the speaking comprises of three separate sections.
The first section or part 1 is known as the introduction and interview segment. Here the interviewer introduces him or herself to the test taker. This is then continued as if it were an interview – albeit a little formal – whereby the test taker shares information with the examiner. The examiner will have a script of questions to ask the test taker which cover every day familiar topics e.g. work, study, where you live, food, holidays, friends, goig out, festivals, sports, school and public transport (TeachingEnglish | British Council | BBC, 2017).
The test taker provides the examiner with information about themselves, their family, work and interests through a series of open-ended questions like those in appendix 1 e.g. What’s the most interesting part of your village/town? Would you say it’s a good place to live? Why? (Ieltsessentials.com, 2017) and answers them to the best of their ability. This section usually takes around 4-5 minutes.
The second section or part 2 is known as the individual long turn. In this segment the candidate or test taker is given a topic card (see appendix 2). The topic card has a subject on it which the test taker must talk about for two minutes. Before speaking, the candidate is allowed one minute in order to prepare notes on their given topic. These topic tasks are generally about a personal experience, for example, a memorable day or a person of great significance to them and they then have to produce a monologue on this topic. Once the candidate has completed their monologue the examiner will then ask them some follow up questions which then leads them into part 3 of the test.
The third section or part 3 is known as the two-way discussion. This two-way discussion or dialogue is between the test taker and the examiner and is generally thematically linked (Karim and Haq, 2014) to part 2. If you look at appendices 2 and 3 you will see that the candidate in part 2 spoke about a piano he once owned and in part 3 the examiner then asks to have a discussion regarding things “we own”.
Interviewing is a tried and tested way of assessing a language learners speaking capabilities, however, it can seem intensely formal to the interviewee and cause the interaction to be dominated by the examiner (Karim and Haq, 2014) in their role as interviewer in the IELTS test. In the test then, this tends, to lend itself toward a situation in which the speaking is less natural in form than it could be if it were in a different format e.g. an informal discussion over coffee. Hughes (1991) states the most obvious format for the testing of oral interaction is the interview, however, it has at least one potentially serious drawback. When the test taker feels they are in an interview situation they will speak to the examiner as if speaking “to a superior” (Hughes, 1991, pg.104). Therefore, the speaking confidence of the test taker may be put to a disadvantage. This limitation may well be avoided should the candidate feel they are able to ask questions as well as the examiner. Through a relaxed dialogue rather than an interview the conversation could well take a more natural path providing more confidence for the test taker. Also, interviews can cause unnecessary anxiety and nervousness to the test taker which could limit their interaction.
We could also argue that natural conversation is built up with more informal language, whereby a conversation is not scripted but flows along an unseen path to an unseen destination. Conversation does not follow a strict or distinct route but is nudged along in many different directions.
The question we are asking is “Is the test successful in achieving its objectives?” and for this we need to look at what is known as the validity and reliability of a test. Validity has several forms and I have looked at content, face and criterion-related validity. Content validity is a representative sample of the skills and abilities which have been taught. The structure and content of the IELTS speaking test is the same at all levels. This means that an elementary level candidate will be answering the same questions of those at advanced levels. This does not provide a concrete level of testing as the structure and content do not provide a more challenging course for those with more advanced skills.
Face validity looks at the measurement of skill the test should be assessing. If we look at the format of the IELTS test we can see that it is clearly established. It is universal in its approach and anywhere you look for information on the IELTS test it always provides the same information.
In contrast both IELTS academic and General English tests are no different with regards to what is tested for in the speaking section. If we look at the band descriptor (appendix 4) we can see that there is no distinction for specific English. This brings into question the validity of criterion-based validity.
The reliability of the IELTS test must also be looked at. A single examiner is responsible for rating the candidate, therefore, the inter-rater reliability has to be questioned due to interpretation but a single person of the test taker. The scoring is split between four categories (appendix 4) – within these four categories there is no in-depth breakdown so scoring is at the discretion of the examiner.
The effectiveness of the IELTS speaking test has some limitations. Overall it is a meaningful test which shows validity and reliability in some areas but there are areas in which both validity and reliability fall short as has been shown. The following suggestions could make the IELTS speaking test somewhat more valid and reliable – provide more time for the candidate to talk freely in the first section. In this way we could elicit more authentic data from the candidate and be provided with a more in-depth analysis of the actual ability the candidate has. More tasks would elicit a better performance from the candidate – discussion about a picture or tasks which represent different communicative processes. The grading scale should be open to more variables to provide a broader perspective and finally, more than one examiner would ensure a more reliable assessment.
APPENDICES
APPENDIX 1
Speaking sample task – Part 1
Part 1 Introduction and interview
[This part of the test begins with the examiner introducing himself or herself and checking the candidate’s identification. It then continues as an interview.]
Let’s talk about your home town or village.
What kind of place is it?
- What’s the most interesting part of your town/village?
- What kind of jobs do the people in your town/village do?
- Would you say it’s a good place to live? (Why?)
Let’s move on to talk about accommodation.
- Tell me about the kind of accommodation you live in?
- How long have you lived there?
- What do you like about living there?
- What sort of accommodation would you most like to live in?
Speaking sample task – Part 1 transcript
Part 1 Introduction and interview
[This part of the test begins with the examiner introducing himself or herself and checking the candidate’s identification. It then continues as an interview.]
Transcript
Examiner: Now, in this first part, I’d like to ask you some more questions about yourself, OK? Let’s talk about your home town or village. What kind of place is it?
Candidate: It’s quite a small village, about 20km from Zurich. And it’s very quiet. And we have only little … two little shops because most of the people work in Zurich or are orientated to the city.
Examiner: What’s the most interesting part of this place … village?
Candidate: On the top of a hill we have a little castle which is very old and quite well known in Switzerland.
Examiner: What kind of jobs do people in the village do?
Candidate: We have some farmers in the village as well as people who work in Zurich as bankers or journalists or there are also teachers and some doctors, some medicines.
Examiner: Would you say it’s a good place to live?
Candidate: Yes. Although it is very quiet, it is … people are friendly and I would say it is a good place to live there, yes.
Examiner: Let’s move on to talk about accommodation. Tell me about the kind of accommodation you live in …
APPENDIX 2
Speaking sample task – Part 2
Part 2 – Individual long turn
Candidate Task Card
Describe something you own which is very important to you.
You should say:
- where you got it from
- how long you have had it
- what you use it for
- and explain why it is important to you.
You will have to talk about the topic for 1 to 2 minutes.
You have one minute to think about what you’re going to say.
You can make some notes to help you if you wish.
Rounding off questions
• Is it valuable in terms of money?
• Would it be easy to replace?
Speaking sample task – Part 2 transcript
Part 2 – Individual long turn
Transcript
Examiner: Alright? Remember you have one to two minutes for this, so don’t worry if I stop you. I’ll tell you when the time is up.
Candidate: OK
Examiner: Can you start speaking now, please?
Candidate: Yes. One of the most important things I have is my piano because I like playing the piano. I got it from my parents to my twelve birthday, so I have it for about nine years, and the reason why it is so important for me is that I can go into another world when I’m playing piano. I can forget what’s around me and what … I can forget my problems and this is sometimes quite good for a few minutes. Or I can play to relax or just, yes to … to relax and to think of something completely different.
Examiner: Thank you. Would it be easy to replace this, this piano?
Candidate: Yes, I think it wouldn’t be that big problem but I like my piano as it is because I have it from my parents, it’s some kind unique for me.
APPENDIX 3
Speaking sample task – Part 3
Part 3 – Two-way discussion
Let’s consider first of all how people’s values have changed.
- What kind of things give status to people in your country?
- Have things changed since your parents’ time?
Finally, let’s talk about the role of advertising.
- Do you think advertising influences what people buy?
Speaking sample task – Part 3 transcript
Part 3 – Two-way discussion
Transcript
Examiner: We’ve been talking about things we own. I’d like to discuss with you one or two more general questions relating to this topic. First, let’s consider values and the way they can change. In Switzerland, what kind of possessions do you think give status to people?
Candidate: The first thing which comes in my mind is the car. Yes, because lots of people like to have posh cars or expensive cars to show their status, their place in the society.
Examiner: Is that a new development?
Candidate: No, I think it isn’t.
Examiner: People have thought like that for quite a long time?
Candidate: Yes. Another thing is probably the clothing. It starts already when you are young. When the children go to school they want to have posh labels on their jumpers or good shoes.
Examiner: What do you think of this way of thinking that I need to have a car or certain clothes to show my status?
Candidate: Probably it’s sometimes a replacement for something you don’t have, so if your wife has left you or your girlfriend, you just buy some new, I don’t know, new watches or new clothes to make you satisfied again.
Examiner: You don’t think of it as a healthy way of thinking?
Candidate: It’s probably not honest to yourself. You can understand what I mean?
Examiner: Yes. And do you think this will change? In the future, will cars and designer clothes be status symbols in the same way?
Candidate: I’m sure that clothes will be … that the thing with the clothes will be the same. I’m not so sure about the cars because cars cause lots of environmental problems and probably in some years, a few years, this will change because it’s not reasonable to drive a car anymore.
Examiner: Can you tell me a little bit more about that? …
APPENDIX 4Â IELTS Speaking Band Descriptors
|
Band Fluency and coherence |
Lexical resource |
Grammatical range and accuracy |
Pronunciation |
|
|
9 |
or grammar •
|
uses vocabulary with full flexibility and precision in all topics uses idiomatic language naturally and accurately |
characteristic of native speaker speech • • |
uses a full range of pronunciation features with precision and subtlety sustains flexible use of features throughout is effortless to understand |
|
8 |
rarely to search for language •
• |
uses a wide vocabulary resource readily and flexibly to convey precise meaning uses less common and idiomatic vocabulary skilfully, with occasional inaccuracies uses paraphrase effectively as required |
• |
uses a wide range of pronunciation features sustains flexible use of features, with only occasional lapses is easy to understand throughout; L1 accent has minimal effect on intelligibility |
|
7 |
some flexibility • |
uses vocabulary resource flexibly to discuss a variety of topics uses some less common and idiomatic vocabulary and shows some awareness of style and collocation, with some inappropriate choices uses paraphrase effectively |
|
shows all the positive features of Band 6 and some, but not all, of the positive features of Band 8 |
|
6 |
hesitation •
|
has a wide enough vocabulary to discuss topics at length and make meaning clear in spite of inappropriacies generally paraphrases successfully |
|
uses a range of pronunciation features with mixed control shows some effective use of features but this is not sustained can generally be understood throughout, though mispronunciation of individual words or sounds reduces clarity at times |
|
5 |
|
manages to talk about familiar and unfamiliar topics but uses vocabulary with limited flexibility attempts to use paraphrase but with mixed success |
|
shows all the positive features of Band 4 and some, but not all, of the positive features of Band 6 |
|
4 |
connectives and some breakdowns in coherence • |
is able to talk about familiar topics but can only convey basic meaning on unfamiliar topics and makes frequent errors in word choice rarely attempts paraphrase |
|
uses a limited range of pronunciation features attempts to control features but lapses are frequent mispronunciations are frequent and cause some difficulty for the listener |
|
3 |
|
uses simple vocabulary to convey personal information has insufficient vocabulary for less familiar topics |
|
shows some of the features of Band 2 and some, but not all, of the positive features of Band 4 |
|
2 |
|
only produces isolated words or memorised utterances |
• cannot produce basic sentence forms • |
Speech is often unintelligble |
|
1 |
|
|||
|
0 • does not attend |
||||
REFERENCES
Hughes, A. (1991). Testing for language teachers. 1st ed. Cambridge [England]: Cambridge University Press.
Ieltsessentials.com. (2017). Speaking Practice Tests. [online] Available at: https://www.ieltsessentials.com/global/prepare/freepracticetests/speakingpracticetests [Accessed 01 Mar. 2017].
Karim, S. and Haq, N. (2014). An Assessment of IELTS Speaking Test. International Journal of Evaluation and Research in Education (IJERE), 3(3).
Levine, J. and Spencer, M. (1996). Developing pedagogies in the multilingual classroom. 1st ed. Stoke-on-Trent: Trentham Books.
Takeielts.britishcouncil.org. (2017). IELTS teachers’ questions answered | Take IELTS. [online] Available at: http://takeielts.britishcouncil.org/teach-ielts [Accessed 7 Feb. 2017].
TeachingEnglish | British Council | BBC. (2017). Evaluating speaking – the IELTS speaking test. [online] Available at: https://www.teachingenglish.org.uk/article/evaluating-speaking-ielts-speaking-test [Accessed 07 Mar. 2017].