Evaluation Of CALL Website English Language Essay

Introduction

The methods, in which teachers make advantage of the computer, including multimedia material or internet, have been increasingly introduced into the language teaching and learning. Such type of language learning is called Computer-Assisted Language Learning (henceforth, CALL). This trend in language learning is a product of significant improvement in the individual processing capability for a PC and reliability improvement of network systems, not to mention the advancement of computer hardware itself. Furthermore, an objective and quantified collection of data such as learner’s corpus has been organized, and research in this field contributes to the content of CALL material. Additionally, recent improvement in learners’ computer literacy can also be considered to have played a significant part in the introduction of CALL into class.

It is considered that a CALL practitioner has to concern him/herself to the issues about equipment and facilities such as the introduction of the sufficient number of PCs, the maintenance of the computer room, or development of materials aimed for language learning. In other words, these are inevitable issues in that CALL is not to be implemented without computers. On the other hand, with the relative enrichment of equipment or facilities, there is the other kind of issues which needs to be addressed, namely the matter of how CALL is integrated in teaching methodology and syllabus in class. Although the introduction of CALL may have resulted in raising learners’ motivation in class, it has been reported that it does not directly lead to improvement of learners’ performance (Stenson et al. 1992, Schcolniketal. 1995 cited in Warschauer and Healey 1998). In other words, it is not only necessary to introduce a CALL material into class: without considering how and how often it is used according to class, its effect cannot be expected. With regard to this point, Levy and Stockwell (2006: 199) states that “effective use of CALL in the classroom requires practitioners to be aware of the constraints that the choice of certain technologies brings, and how to deal with these constraints.” This leads to the idea that, without fitting into the actual situation where a material is introduced, it does not produce the desired result no matter how good material is. Therefore, it can be said that one important factor for the successful use of CALL is adoptability of material to the actual teaching and learning situations, and, in order to investigate it, practitioners need to evaluate material based on various aspects.

The aim of this paper is to evaluate a certain CALL website, Using English for Academic Purpose (henceforth, UEfAP). [] At first, three main approaches to CALL evaluation are briefly overviewed and the detail of the framework applied in this paper is introduced in section 2. Subsequently, in section 3, UEfAP is evaluated based on the framework mentioned in the previous section. Finally, concluding remarks are given in the last section.

An Approach to Evaluation of CALL Material

As mentioned above, it is considered that one of the measures for effective introduction of CALL material into class is appropriate evaluation of the object material in advance. In other words, evaluating CALL material according to whether it suits the aim of class or whether its level is appropriate for learners can be the first step to incorporate it into class. Although this is also the case for evaluation of a traditional textbook used in class, more extensive check is required for evaluating CALL material in that there are more considerations such as audio, speech recognition, or student record keeping, all of which are not included in textbooks as paper-based media.

2.1 Evaluation Criteria for CALL

Approaches used in evaluating CALL material can be roughly classified into three: checklist, methodological approach, and SLA-based approach (Hubbard 2006).

Checklist has been proposed in the early stage of research on evaluation and has still been widely used. According to Hubbard (2006), checklists have been criticized in that, for example, some put much focus on technology than teaching methodology and others are not exhaustive; Susser (2001), however, argues that the problem does not lie in the concept itself but in its instantiations. In response to this point, Susser and Robb (2004: 288-290) propose the following successive processes particularly with regard to evaluation of web-based material: (a) “the screening process” where the premise for evaluation based on the learning environment and the aim of learning are considered, (b) “finding potential sites” where sites, which are searched in terms of several key words, are addressed by means of screening criteria established in (a), and (c) “creating a checklist” in which the framework is used to create a checklist by selecting principles relevant to the situation. In other words, checklists are not applied as they are but are adapted for a given situation and accordingly updated. As for the advantage of a checklist, it seems to make teachers recognize various items relating to a software application. In particular, it is considered to be practical for teachers who make use of CALL material in their lessons for the first time but have little knowledge about evaluation.

Although a methodological approach overlaps with a checklist approach in some points, they are different in the following two points (Hubbard 2006): methodological frameworks are aimed at describing rather than judging in their form; they are designed to associate with the aspect which is outside of technology, that is, considerations for language teaching and learning. With regard to the first point, it can be considered that methodological frameworks enable teachers to make in-depth evaluation in that they may be required not to give probably binary and ready judgement on existing items but to make a detailed analysis of it through description based on the situation. Hubbard (1988 cited in Hubbard 2006) presents the main criteria for CALL evaluation, developing three language teaching methods proposed by Richards and Rodgers (1982, 2001), namely approach, design, and methods, into “teacher fit,” “learner fit,” and “operational description” respectively, and adding “technical preview” to these three criteria. The detail of these criteria is overviewed in the next section.

On the other hand, SLA-based approach is particularly based on SLA theory and it exploits findings from non-CALL research to adapt them to CALL. Chapelle (2001) focuses not only on CALL software but also on activities and learners’ performance. She proposes the principles for evaluating CALL material emphasising on such elements as a situation-specific argument, SLA theory, the purpose of the task, judgemental analysis of software and tasks, empirical analysis of learners’ performance, and most importantly, language learning potential. On the basis of them, Chapelle (2001: 55) suggests the following six criteria: (a) language learning potential, (b) learner fit, (c) meaning focus, (d) authenticity, (e) positive impact, and (f) practicality. Given the space limitation, this paper cannot take these in detail, but they are compatible with criteria both in checklist approaches and methodological frameworks in some respects; criteria (b), for example, seems to be consistent with the concept of “learner fit” in Hubbard’s framework.

This paper adopt Hubbard’s (1992, 1996, 2006) framework to evaluate CALL material since the methodological frameworks can be considered to be more neutral than SLA-based approach which has its basis on the Task-Based Approach as exemplified in, for instance, (c) meaning forcus or (d) authenticity in the above Chapelle’s criteria, while all three approaches stated above have their advantages as Hubbard (2006) mentions. The following section refers this framework in detail.

2.2 Methodological framework

Hubbard (2006) defines evaluation as the process of selection, implementation, and assessment: selection refers to the phase in which CALL material is judged its adoptability for a given learning situation; implementation, on the other hand, refers to the phase of certifying the effective way to implement a given material in that setting; finally, assessment is the phase where practitioner estimates its effect and judges whether it is continued to use or is adjusted to future use. Although, as Hubbard (2006) says, there is no difference with the degree of importance among these three stages, this paper is concerned in the first stage of evaluation.

According to Hubbard (1992, 1996, 2006), the selection stage in evaluation involves four main components: (a) technical review, (b) operational description, (c) teacher fit, and (d) learner fit. Each component is briefly described as follow:

(a) Technical review

The most fundamental concern here is whether a given material will run adequately on the equipment used by teachers and learners. More specifically, this assessment involves various perspectives such as simplicity of installation and un-installation, speed of program operation, reliability of operation, platform compatibility, screen design and management, user interface, exploitation of computer potential, and the rest. Furthermore, issues of bandwidth and server access are also the technical matter.

(b) Operational description (procedure)

Assessment here is always required to be made with regard to such activity types as instructional (e.g. tutorial or text reconstruction), collaborative (e.g. game or simulation), or facilitative (e.g. exploration). In other words, an evaluator is required to look into what types of activities the software offer, how well they are designed, and what learners do using them.

(c) Teacher fit (Approach)

In this assessment, an evaluator tries to look into the underlying theoretical framework of material such as a structural, functional, interactional approach to language, and to judge whether it is compatible with theories of cognitive development, SLA, and classroom methodology.

(d) Learner fit (Design)

The concern here is how well the contents, skills and linguistic level of materials meet the learners’ needs, styles and interests of intended learners especially with regards to the objectives in the course syllabus. In detail, material can be evaluated on the basis of learner variables as follows: individual learner differences (e.g. age, sex, L1, introverted vs. extroverted learners), learning styles (e.g. recognition, recall, comprehension, and experiential learning), learning strategies (e.g. field-dependent/-independent reasoning, deductive/inductive reasoning, visual-graphic/visual-textual learning), learner control, design flexibility/ modifiability by teachers and/or learners.

Evaluation is made linearly based on these four components, and, in particular, the assessment in learner fit and teacher fit feeds into (d) appropriateness judgements and (e) implementation schemes which are more likely to relate with the implementation stage. In assessment of appropriateness judgements, on the basis of the judgement following teacher fit and learner fit, an evaluator makes a decision to teach learners with a given material or not, taking the costs and benefits of implementation into consideration as well. On the other hand, in assessment of implementation schemes, an evaluator needs to reflect on how materials might be fit into the course: in particular, these includes accessibility, preparatory and follow-up activities, content preparation, and a number of teacher control such as the teaching approach, classroom management, site monitoring, student records, and teacher authoring possibilities. Although implementation is the matter which occurs after selection of material in evaluation, considering the way of implementation seems useful for determining how to use it effectively and may influence whether it is used in class or not. Therefore, although this paper especially deals with the four main components from (a) to (d), it seems important to keep considerations of implementation in mind during evaluation.

In the following section, a particular web-based material is evaluated on the basis on Hubbard’s criteria.

Evaluation of UEfAP

To start with, this section mentions the context to which CALL material would be introduced, and subsequently, the site, UEfAP, is evaluated with regard to Hubbard’s criteria in turn.

3.1 Teaching Situation

This paper posits a 10-weeks class in pre-sessional EAP (English for academic purpose) course where the students come from various countries, have different cultural backgrounds, speak in different L1, vary with their age usually from 22 to 35, and intend to attend a graduate programme after finishing the course. In the introduction of web-based material, the students are expected to improve their academic use of English, which is previously taught in class, mainly by self-study outside of class.

3.2 General Description

UEfAP is a free website which consists of materials and exercises for learning academic English. This website started for people who want to use English for academic purposes on 1st May 1999, and last updated on 28th March 2011. The author of the website is Dr. Andy Gillet who runs the English language and educational consulting company involving a range of projects including material design, course planning, supporting EAP teachers’ development. The site covers all skills needed for Academic English, and there are explanations and a good deal of exercises for each skill. In detail, it is consists of 11 sections: “About,” “Accuracy,” “Assessment,” “Background,” “Links,” “Listening,” “Materials,” “Reading,” “Speaking,” “Vocabulary,” and “Writing.” Each section is divided into subsections based on the language component or the skill.

3.3 Technical Preview

It can be assumed that learners can easily use UEfAP on all platforms. Firstly, because it is the web-based material, learners need not to install/uninstall any software. Secondly, the “About” section includes the site map and the video clip instruction for the whole programme in which users can know how to use the site. Thirdly, although it contains much information, it should not take long to load pages since the site consists of many pages so that information could be subdivided. Moreover, external links or long texts in exercises are set to be opened in a new window; so users can see further information while preserving the current page.

Furthermore, navigation seems consistent and quite clear. The screen is divided into two parts: the left part, which is a fixed space having menu buttons, and the right part, where information appears. The left edge of the right space is colored according to the color of menu button on the left part so that users can easily know their current location. In addition, it can be said that the site is also self-explanatory in that each section has its introduction in which learners can know the goal in a given section.

The site exploits computer potential such as www connectivity and sound as well as video clip instruction as stated above. It contains a wide range of external links; for example, many outside resources or language courses are introduced in the “Links” section where users can go onto other websites in order to take relevant courses or to do further exercises according to their interests and needs. Users can also easily go back to the main menu by clicking the logo mark on every page, and refer other pages in the same site relating a given topic by following hyperlinks. Additionally, it makes use of sound recorded in RealAudio format, Windows Media format and/or MP3 format. On the exercise page in the Listening section, users are required to download one of RealPlayer, Windows Media Player, Quicktime Player or Flash Player by clicking a link button on the page if their PCs are not equipped with any of them. Unfortunately, users can choose sound players only in some exercises, so they have to download, for example, Real Player if a given exercise provides sound only in the RealAudio format.

3.3 Operational Description (Procedure)

As mentioned above, the website covers all language skills including reading, speaking, listening, vocabulary, and writing while the contents related to them are specified into Academic English. Linguistic focus is well balanced in that, for example, the Speaking section focuses on pronunciation, the Writing section focuses on syntax and spelling, and the Vocabulary section puts its focus on lexis and morphology.

It mainly offers a range of instructional activities such as tutorials, text reconstruction and drills, and especially for the word lists of Academic English in the Vocabulary section, there is the facilitative activity in that learners can look in the on-line dictionary (Cambridge Advanced Learner’s Dictionary) to see the definition of words. Each activity is presented variously.

Take, the text reconstruction activity for vocabulary building, for example: it is presented by means of multiple-choice exercises, matching exercises, or gap-fills. In these exercises, learners receive information by means of text and are required to click the possible answer, to drag a word to the appropriate position, or to directly type the correct word into blanks. With regard to gap-fills, the site does not anticipate learners’ responses by offering information on misspellings or other possible answers, and it does not accept misspelled answers. On the other hand, nearly half of all exercises provide feedback for learners’ responses. In several exercises, the incorrect part of a response is highlighted so that learners could retry a given exercise based on it. With regard to this, although learners can repeat a given exercise to correct their mistakes, the total score will become lower if they do so. This is also the case with using hints to solve a question in some exercises.

As for other types of exercises, which deal with a longer text (e.g. the exercise in which learners listen to a lecture or a presentation and write down them in academic written words or exercise where learners are asked to paraphrase or summarize the original text), a suggested answer is given because there is no single answer for a question. Furthermore, some exercises have incomplete answer keys. For instance, “listening test” part of the Listening section have no answer, or some of the tests for “reporting” part of the Writing section just say “Show your answers to someone. If you are in one of my classes, e-mail the paraphrase/summary/synthesis to me.”

It is noteworthy that, in the “Material” section, there are checklists named “EAP Needs” for each skill so that learners can use them to identify what their Academic English language needs are and their ability in each area. This allows learners and/or teachers to customize lessons to the syllabus.

As a whole, the website is complementary to things being taught in class in that it not only offers a wide range of instructional exercises associated with the contents in class but also provides information by which learners can review information given in class. Furthermore, it can be supplementary to class as well. The detailed process of vocabulary learning, for example, may not be taught in class because of the limited time of the EAP course, so information on the site can be useful for learners to obtain additional information.

3.4 Teacher Fit (Approach)

The author of the site states that EAP is a branch of ESP (English for Specific Purpose), citing Robbinson’s (1991) criteria for ESP courses; for example, it should be based on a needs analysis to be geared to learners’ needs and that a very high level of proficiency and accuracy is not required as long as learners can succeed in getting good marks for assignments (Gillet 1996). Therefore, it can be assumed that the author has committed to the Content-Based approach to language learning. However, the website itself can be considered to an example of tutorial CALL and the author bases this program on a structural and behaviouristic approach to language in that the site likely focuses on each linguistic level and expects learners strengthen skills for these levels through a range of instructional activities such as drills.

Although the site may be designed for self-access use, it can be integrated to a classroom curriculum as a supplementary and complementary to class. One reason for this is that the author has carefully built the site grouping the contents according to skills and sub-skills and offering more than two different types of exercises for each skill; hence teachers could selectively utilize a certain part or a certain exercise on the basis of the point to which learners need to address or of learners’ style. Additionally, as mentioned above, the purpose of learning a particular skill or sub-skill is clearly stated in the introduction of each section, so that learners may understand the reasons why particular skills are important for their learning and how they improve these skills.

3.5 Learner Fit (Design)

The pre-sessional EAP course consists of students who vary in age, countries, cultural backgrounds, interests, L1, or learning style as stated above. However, all students are generally of similar ability (i.e. vantage or intermediate) for English which is tested before attending the course, and they equally want to improve their skills for the academic use of English in advance of the start of their graduate programme. Therefore, the syllabus of exercises in the website addresses the specific needs of learners. They are grouped according to functional purposes in that they are arranged not on a progressing scale of difficulty, but on skills or sub-skills.

Furthermore, the exercises focus on different learning styles such as recognition, recall, and comprehension. Take, the exercises in the Writing section, for example. The exercise for rewriting the text which has many spelling mistakes enable learners to recognize common spelling mistakes and the importance of proofreading, and the multiple-choice exercises and the mixed-up sentence exercises for writing a reference list after being taught how to write it in class help learners enhance recall. Moreover, the exercises for identifying the topic of question or for reordering words to build the paragraph promote learners’ comprehension of the types of research questions or the pattern of establishing paragraphs respectively.

Moreover, although almost exercises are intended for individual work, learners are required to work on by a pair or a group in some exercises in the Speaking section. In these exercises, one learner look at a picture A and another learner look at a picture B which is slightly different from A; then learners are required to describe own pictures and ask questions about other’s picture in order to find the differences between them without looking at each other’s. This is also the case with a group work except that three pictures (A, B and C) are prepared for it.

Concerning the recording function for learners’ responses, the website has no option to keep the scores, to record the number of attempts and incorrect answers, and to keep track of total time spent on an exercise; so learners have to keep their own scores by themselves to make them available to the teacher. In addition, learners cannot save completed exercises after quitting the program.

Lastly, it should be noticed that some exercises seems too lengthy and demanding even for the motivated learners. In some exercises for the “General Listening” test in the Listening section, for instance, learners are required to fill more than 150 gaps with listening to the speech extract, or, in some exercises for the “Pronunciation” part of the Speaking section, learners are expected to transcribe the text, which includes total 245 words, using the IPA symbols. These exercises may tire learners.

Conclusion

UEfAP is a useful free resource of materials and exercises not only for self-directed learner but also for teachers who intend to introduce tutorial CALL to their class. On the other hand, it has some shortcomings as follows: some exercises have incomplete answer keys or feedback; some may be fatigue because of their length; the site has no options to save learners’ responses or time spent on an exercise, which means that it does not fully exploit the computer potential to provide opportunities for learner-computer interaction. Therefore, when teachers decide to implement the website in their class, they need to selectively utilize it. In particular, in the stage of implementation schemes after evaluation, teachers need to carefully consider how it might be fit into the course.

Although CALL has gained attention in various educational environments and there is significant improvement in equipment and facilities, it has been pointed out the difficulty of integrating CALL material in class because the introduction alone cannot lead educational effectiveness. In this situation, making careful and in-depth evaluation can be the most effective indication of deciding whether it is appropriate for one of the course materials, and how it is implemented into class. Furthermore, noticing various possible problems can be also contributory factor to successful introduction of CALL; that is, evaluation enables a practitioner to appropriately and immediately deal with such problems when they occur in actual use.