Evaluating Speaking I. What to evaluate A. ______【T1】______ targets: — the individual sounds — stressed and weak sounds in words

In this section you will hear a mini-lecture. You will hear the mini-lecture ONCE ONLY. While listening to the mini-lecture, please complete the gap-filling task on ANSWER SHEET ONE and write NO MORE THAN THREE WORDS for each gap. Make sure the word(s)you fill in is(are)both grammatically and semantically acceptable. You may use the blank sheet for note-taking.
    You have THIRTY seconds to preview the gap-filling task.
     (a thirty-second interval)
    Now listen to the mini-lecture. When it is over, you will be given THREE minutes to check your work.
                            Evaluating Speaking
    Good evening. Last time, we looked at what a speaker needs to be able to do to use spoken English as an effective form of communication. In today’s lecture, we’ll look at whether these different elements can be evaluated formally, and what ways there are to do this. Now let us start by considering which elements of speech can be evaluated, and which cannot — or should not.
    So my first question is: what can we evaluate?
    First, phonological features. Speakers need to be able to produce the phonological features of speech well enough to be understood, so it is fundamental that these are included in evaluation in some way. Things such as the individual sounds, stressed and weak sounds in words and speech, rhythm and intonation patterns are easy to elicit and identify. We can then measure them against a standard based on whether we can understand them or not, or perhaps more accurately, whether a typical listener could.
    Second, rules of language. Speakers need to be able to understand and obey the rules of structure, lexis, and discourse when they speak. Again, this is easy to evaluate through observation, although we need to start thinking now about providing the speaker with suitable tasks and a suitable context, e. g. in interaction with other speakers. Observing the speaker, we can ask questions such as:
    Is the speaker choosing the right vocabulary? Does it make sense? Is it formal enough? Does it collocate with other words correctly?
    Is the speaker following rules about grammatical structure?
    Is it coherent and cohesive?
    Third, paralinguistic devices. It is clear that speakers need to be able to understand and use paralinguistic devices, as they are an essential part of communication, and competence includes the ability to manage these devices. We can convey an enormous amount with use of eye contact and facial expression, for example, and gestures work in the same way as linguistic communication. However, there are problems. For example, it is difficult to evaluate many of these features explicitly. The speaker’s use of gestures, expressions and verbal tools such as noises can be observed, but can we establish a standard of correct use? Eye contact, movement of the body and head, and posture all send powerful messages, but how do we describe them in a framework for evaluation? How do we elicit them in a controlled form? In addition to this, we may feel that it is not appropriate to evaluate this area at all as part of spoken language testing, and that the best way to address these may be under another heading, such as intercultural communicative competence. This then means evaluating them separately, using very different techniques.
    Fourth, Communicative functions. Speakers need to be able to recognise, understand and use the communicative functions of speech. This means that speakers actually communicate with their choices of vocabulary and grammar, intonation and stress, changes in volume and tone, etc. These features can be evaluated through observation of the speaker’s performance and comparison against a standard. As we are evaluating communicative functions, it is relevant to evaluate a speaker from this perspective, for example by asking:
    Does the speaker use intonation and stress effectively to support their message?
    Does the speaker manage volume and tone appropriately to support communication?
    Does the speaker use pauses, repetition and noises appropriately to support communication?
    Fifth, social meaning. Speakers need to be able to understand and use the social meaning of speech and many aspects of this can be evaluated formally. We can for example assess the ability to use formal and informal language, and the degree of directness, by using suitable tasks which recreate social factors such as status and age. The speaker’s understanding of conversational principles and rules can be observed in interaction with others, e. g. in a group task. Connotations of language can be included as part of evaluation of use of vocabulary.
    Then, my second question is: how do we evaluate?
    As we can see, most aspects of spoken language can be evaluated formally. The challenge is to find a form for testing which enables us to do so. An effective format for evaluation should enable us to isolate and analyse various elements, possibly under a series of general headings, but also use tasks which allow us to measure the speaker’s communicative competence in general. Clearly if we want to measure a speaker’s language we want them to perform to their best ability, so we also need to consider the best ways to reduce the impact of emotional factors such as stress and nerves. Finally, we need to think about practical concerns around available resources, such as time, examiners, equipment if we are going to record the speaker, and space.
    There are a wide range of test types and elicitation available. If we limit them to formal evaluation by an examiner, as opposed for example to self-assessment or informal evaluation during classes, the most common ways for a teacher or examiner to evaluate speaking formally include: natural conversation, joint tasks, presentations and descriptions, role-plays, interviews, using visual prompts, retelling a story, reading aloud, etc. For most of the time, they will be used in a mixed way.
    Today, we have reviewed which elements of spoken language can be evaluated and what kinds of questions we can ask when we consider each. These questions measure communicative competence, in other words, with the underlying message of " How good is the speaker at communicating?" We have also considered some of the many types of tests available. These need to be matched to testing aims, types of candidates and the resources available. Next time, we will look at three formal speaking tests, identify how they work, and consider their effectiveness. Thank you for your attention.