Content Analysis of Education Module

CONTENT ANALYSIS ON ENGLISH MODULE”BAHASA INGGRIS”

Dwi Aryani

Abstract

This research is aimed to evaluate the quality of the English module used by students of SMA N 6 Yogyakarta. There were many problems such as mistype, inappropriate content or material, and so on which were found inside the book. Then, the module used by the students should be evaluated on the content to know whether the book met the criteria of good module or not. The research was about case study using content analysis as the method. It used textbook evaluation criteria from Cunningsworth which contained ten aspects of evaluation and compared with the criteria of evaluation from Hutchinson and Waters. The evaluation concerned on the content such as language content and skill, the organization and appearance of the book. The result from Cunningsworth checklist showed that the book got 74. 25% which considered as good book. While the result from Hutchinson and Waters showed 78. 57%. The aspect which got low percentage was vocabulary and reading material, while the rest were considered as good. From evaluation using two concepts from Cunningsworth which compared with the evaluation criteria by Hutchinson and Waters showed a little bit difference. The result showed differences in range 4. 32%.

Keywords: module, content analysis, evaluation, and evaluation checklist

INTRODUCTION

Teaching resources are all materials that are used to facilitate in teaching and learning process. They can be divided into three forms which each has different functions and advantages. The first form of material is printed materials such as books, module or workbooks, worksheets, help the beginner teachers to cover and design each lesson carefully in detail. They also give all the plans and lessons that teachers need to present the topic in some details. Moreover, printed material can be used in any location or bring everywhere. Nonprinted materials like audio and video materials have their own advantages such as; inexpensive materials, since nonprinted materials have already prepared by the teachers and students should not buy it. There are materials that comprise both print and nonprint such as materials from internet which is also have some advantages. Internet provides many materials in a form of audio, text, picture, graphic and video. Students only need to access on the internet pages suggested by the teachers and follow the instruction. It was quite easy for school which provides internet access (Richards-2002).

Material has important roles in language teaching such as presentation resources, practice and communication interaction, language competence references, classroom activities sources, syllabus, and help less experience teachers (Cunningsworth-2005). Those are the role of materials in language teaching for both students and teachers.  Module in several schools used as the primary sources in conducting teaching and learning activity. Module is categorized as commercial books which contain some potential negative effect as cover inauthentic language, distort content, not reflect students’ needs, deskill teachers, and expensive (Richards-2002). Related to the negative potential effect, it means that the materials covered by module are needed to be evaluated.

In order to judge the quality of the book, the book should cover some criteria of good teaching resources. The criteria of good teaching resources on this context are criteria for printed materials. Teaching resources take a part in the teaching and learning process. The criteria are; 1) arouse the learners’ interest, 2) remind them earlier learning, 3) tell them what they will be learning next, 4) explain new learning content to them, 5) relate these ideas to learners’ previous learning, 6) get learners to think about new content, 7) help them get feedback on their learning, 8) encourage them to practice, 9) make sure they know what they are supposed to doing, 10) enable them to check their progress, and 11) help them to do better.

Evaluation is measuring of strengths and weakness (Davies-2002). It can be conclude that the materials which are covered by module can be evaluated to know the strength and weakness. The evaluation can be done in three ways; pre-use evaluation, in-use evaluation, and post-use evaluation (Cunningsworth-2005). The process of evaluation depends on the one who desires to evaluate the materials. Moreover, in order to know the strength and weakness of materials (book or module), it can be done through the evaluation checklist from the expert on the field (the expert on this context is expert in teaching). There are many experts who provide the checklist to help in evaluating book.

METHOD

This research aimed to evaluate the content of module for senior high school. The type evaluation on this research is post-use evaluation since the evaluation conduct after the book was used by the students. The research was focused on the content of the book in order to meet the criteria of good teaching resources by measuring the strength and weakness. Therefore, it is classified as case study research category. Case study is a research method which is done towards a system; it can be program, activity, and event. Case study is aimed to get a data, take a conclusion, and get an understanding from the data itself (Sukmadinata-2012). A case study may be understood as the intensive study of a single case where the purpose of that study is to shed light on a larger class of classes or a population (Gerring-2006). Moreover, it was not aimed to get the conclusion from the population. The conclusion from case study is simply from the case that the researcher taken to heart. There are some technique in collecting the data on case study research such as interview, observation, and documentary study. Those are the technique to collect the data and to get the conclusion.

The data source was the materials covered by “Bahasa Inggris”, a module used by students in SMA N 6 Yogyakarta grade XII on the first semester. The book was written and published by CV Viva Pakarindo which was consisted of eighty pages and divided into five chapters. The book was applying Kurikulum Tingkat Satuan Pendidikan or abbreviated as KTSP as the curriculum.

Instruments were needed to collect the data. This research used content analysis as an instrument to collect the data from students’ module. There were two instruments to support in conducting the research. First instrument used in this research was documentary study. Sukmadinata (2012) explained that documentary study is data collecting technique which is accumulate and analyze the document, whether documentary, picture, or electronic. The document which is accumulated is already chosen with purpose and focus on the problem.

Moreover, the other instrument on this research was content analysis where the study is focus on deeply into the content of written information or mass media. Moreover, content analysis is defined as a research method for the subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns. Wilkinson (2003) mentioned “content analysis has been successfully used to analyze text and solve issues of disputed authorship of academic paper….” and “…examples of texts suitable for content analysis includes essays, journal articles, books and chapter in books…” From the explanation which stated in the Wilkinson give a clear statement that module or workbook which categorized as book can be analyzed. In this case, module included as one of academic paper such as mentioned before. Then, content analysis focused on the characteristics of language as communication with attention to the content or contextual meaning of the text.

Since the subject on this research was the module, so the researcher prepared some checklists to evaluate the book. The researcher provides two evaluation checklists to evaluate the book. The first evaluation checklist is taken from Cunningsworth which covers ten descriptions such as; design and organization, language content, vocabulary, grammar, language skill, listening, speaking, reading, writing, and practical consideration. The other evaluation checklist is taken from Hutchinson and Waters which cover main aspect such as audience, aims, contents, methodology, and others. Every aspect on evaluation checklist covers some questions or statements to be answered in evaluation process.

There were some procedures for conducting this research. The procedures will be explained as follow; a) Preparation, on this step the researcher took a part to choose the book which was needed to be evaluated. Choosing the book which was needed to be evaluated from the external and internal factor; b) Analysis, took a part on analyzing the book using criteria designed by the researcher based on the criteria or concepts from the expert. It means that the evaluation criteria were not pure from the researcher, but adapting from one concept to evaluate the book; c) Report, this step was the last step on this procedure. After the researcher pass the preparation and analysis step, means that the researcher should make the report of the result after conducting the evaluation.

After conducting the evaluation towards the book, however, scoring is needed to get the result of the evaluation. Book evaluation using Cunningsworth’s checklist given score one until five. The indicator is modified using Likert Scale which indicates negative until positive (Sugiyono-2012). Moreover, evaluation checklist suggested by Hutchinson and Waters has the own rate or score by using zero, one, and two. These score and indicators are originally designed by Hutchinson and Waters. The result of the evaluation is calculated using mean deviation. Therefore, the result of calculation will be described in a form of paragraph by using descriptive qualitative.

Figure 1. A pattern to calculate the score of each evaluation.

Note :

Æ©X = the total of criteria which is met the criteria

N   = the total criteria points in each aspect

Table 1. The level of Suitability

80% – 100%	Very Good
60% – 79%	Good
50% – 59%	Fair
Under 50%	Poor

FINDINGS AND DISCUSSION

The evaluation taken from two sources; Cunningsworth checklist which is used as the evaluation focus, while the result of evaluation from Hutchinson and Waters is used to compare on this findings. Based on the evaluation, Hutchinson and Waters provides twenty one questions and divided into five main aspects including with the additional information. The overall evaluation from Hutchinson and Waters got result 78, 57% in percentage. It will be compare with the result of evaluation from Cunningsworth evaluation checklist.

The findings of this research presented ten specific aspects where each of them are analyzed based on Cunningsworth’s evaluation checklist.

Design and Organization[P1]

This aspect covers five criteria of evaluation with the total score 20 out of 25. The percentage is 80% and it is categorized as very good. The result of design and organization evaluation is presented on the table as follow.

Table 2. Design and Organization Checklist

No	Design and Organization Criteria	Score
1	The content is organized	5
2	The content is sequenced	5
3	Provide adequate recycling and revision
4	Provide reference sections for grammar and vocabulary	5
5	The materials suitable for individual study	5
Total	20
Percentage	80%

The first criteria is about the content which is organized. It means the contents of the workbook such as topic, structure, skills, and function are organized. All units provide complete topic, all four skills in every unit, language functions and language features. The evaluation shows that the first criteria got perfect score. Such as the organization, the second criteria is about the sequential of the content. It discusses whether the workbook has difficult level for the students and also the function of the workbook for individual study. From all units, the materials can be used for the students to study by themselves. The book cannot fulfill the third criteria by presenting recycle and revision section. The fourth criteria can be fulfilled by providing reference sections for grammar and vocabulary. The materials also suitable for individual study. It can be seen from the contents of the workbook which provide material on the first page in every chapter and continued by the exercise. There are many exercises on the workbook such as exercises for all skills, language contents (grammar and vocabulary), and also exercise that cover all materials. It is presented in the last part in every chapter. The exercise also in different types such as multiple choice and essay. From all those exercises, the workbook is easy for the students even for individual study.

Language Content

Language content is the second aspect of evaluation with the achievement 60% from the total score 9 out of 15. The table below presenting the result of language content evaluation.

Table 3. Language Content Checklist

No	Language Content Criteria	Score
1	The book include materials for pronunciation work
2	The book deal with the structuring and conventions of language use	5
3	Language style is matched to social situation	4
Total	9
Percentage	60%

From the table 3 we can conclude that the first criteria cannot fulfill by the book by providing materials for pronunciation.  The second criteria got perfect score related to the structuring and conventions of the language use. It deals with how to take a part in conversation, how to structure a piece of extended writing, and how to identify the main point in reading passage. Language style is the choice of words used by a specific group of people when they speak. For example there are bureaucratese, the words, jargons, and abbreviation. It should be matched with the social situation of the students. There is one part of the book the researcher found that inappropriate language style with the social situation of the students. It was because the students accusing her friend took her money which actually she does not.

Grammar

The achievement on grammar aspect is on 88% with the total score 22 out of 25. The suitability of grammar materials categorized into very good (see Table 1). The result of the evaluation on grammar aspect is presented on the table below.

Table 4. Grammar checklist

No	Grammar Criteria	Score
1	The book covers grammar items for the level	5
2	Grammar presented in small pieces units	2
3	Grammar emphasis on language form	5
4	Grammar emphasis on language use (meaning)	5
5	Language use and form are in balance presentation	5
Total	22
Percentage	88%

Vocabulary [P2]

Table below is the result of evaluation of vocabulary section where covers five criteria of evaluation. The aspect got low percentage with 44% where there was one criteria cannot fulfill by the book even the worst is contain two score zero.

Table 5. Vocabulary checklist

No	Vocabulary Criteria	Score
1	Materials for vocabulary teaching adequate in terms of quantity and range of vocabulary
2	Vocabulary materials are strategies for individual learning
3	The materials emphasis placed on vocabulary development	5
4	Vocabulary presented in a structured and purposeful way	5
5	Provide distinction between active and passive vocabulary	1
Total	11
Percentage	44%

Language Skill

Language skill gets 95% on the result of evaluation where it is considered as very good. Most of criteria are fulfilled by the book with achievement 5 in average.

Table 6. Language Skill Checklist

No	Language Skill Criteria	Score
1	All four skills were covered	5
2	All four skills are presented balance	4
3	The materials are integrated skills work	5
4	Reading passages and associated activities are suitable with the students’ levels	4
5	Provide sufficient reading material	5
6	Materials for spoken English are well designed	5
7	Writing activities are suitable in terms of amount of guidance/control	5
8	Writing material is using appropriate style	5
Total	38
Percentage	95%

should be well design. Related with the materials and exercises for each skill, writing section should provide guidance or control in writing activities. The style used for writing skill also should be presented in appropriate style. From all criteria, the book gets high score in these eight criteria of evaluation.

Listening

Listening aspect covers four criteria where there is one criteria cannot fulfill by the book. The evaluation shows that the book got 75% from the total score 15 out of 20.

Table 7. Listening Checklist

No	Listening Criteria	Score
1	Listening form part of dialogue or conversation work	5
2	Provide specific listening passages	5
3	Listening material set in a meaningful context	5
4	Provide pre-listening task
Total	15
Percentage	75%

context. However, the book does not provide pre-listening tasks before they focus on the listening section.

Speaking

Speaking got perfect achievement with the percentage 100%. It indicates that the book provide all contents that the criteria mention.

[P3]

Table 8. Speaking Checklist [P4]

No	Speaking Criteria	Score
1	Provide spoken English materials	5
2	The material contains oral presentation and practice of language items	5
3	The material contains dialogues	5
4	The material contains roleplay	5
Total	20
Percentage	100%

The materials for speaking are presenting inside the book where they contain oral presentation and practice for language items. In order to help the students in practicing their speaking skill, the book provides materials such as dialogue and role-play.

Reading

From the evaluation, reading got low percentage with the achievement is 32.5%. The total score of fulfillment is 13 where actually the total score is 40. The book provides reading text where it is used for introducing new

language items such as grammar and vocabulary. Therefore, the materials are not focus on the development of reading skill and strategies for students. Reading materials, however, provide integrated materials with other skills and work. On the contrary, the book does not provide setting scene and background information on reading passage. Then, the book also does not provide pre-reading activities. Thus, reading materials similarly appears on the middle of units where it is commonly appears on the early unit.

Table 9. Reading Checklist

No	Reading Criteria	Score
1	Reading text used for introducing new language items (grammar and vocabulary)	3
2	The material focus on the development of reading skill and strategies
3	The material linked to other skills and work	5
4	Reading text appears on the early unit
5	The subject matter is appropriate	5
6	The text provide setting scene
7	The text provide background information
8	Give pre-reading question
Total	13
Percentage	32.5%

Writing

The evaluation shows that writing covers five criteria where there is one criteria cannot

fulfill by the book. The score of evaluation is 19 out of 25 and it is presented on the percentage with the achievement 76%.

Table 10. Writing Checklist[P5]

No	Writing Criteria	Score
1	The material handle by guided writing	5
2	The material handle by controlled writing	5
3	Provide variety task	5
4	Provide language resources specific to the written form such as punctuation, spelling, and layout	4
5	Learners encourage to review and edit their written work
Total	19
Percentage	76%

Practical Consideration

The last evaluation is on practical consideration which covers five criteria. The result shows that the book got 92% from the total score 23 out of 25. The evaluation presents that the book represents good value for money. It means that the book neither too expensive nor too cheap. Even though the price is quite cheap, but the quality of the book is strong and long-lasting because the materials which is used to print the book. Related to the price of the book, it is easy to be obtained by the students; the school has already stock the book. Therefore, the students may ask to the school or their teachers. On the contrary, the appearance of the book does not attractive as they should.  The cover of the book is colored by burgundy where it looks like dark and gloomy. The detailed achievement from practical consideration is presented in the following table.

No	Practical Consideration Criteria	Score
1	The book represent good value for money	5
2	The book is strong and long-lasting	5
3	The book is attractive in appearance	3
4	The book is easy to obtain	5
5	The parts of package is require particular equipment	5
Total	23
Percentage	92%

The findings based on the Hutchinson and Waters showed that the book got 78.57%. Moreover, the result of evaluation from Cunningsworth got 74.25%.It can be assumed that the book was good even the evaluation taken from different evaluation checklist from different expert. The The detail achievement from ten aspects of evaluation are; design and organization got 80%, language content got 60%, grammar got 88%, vocabulary got 44%, language skill got 95%, listening skill got 75%, speaking skill got 100%, reading skill got 32.5%, writing skill got 76%, and practical consideration got 92%.

There were five aspects which were considered as very good such as design and organization, grammar, language skill, speaking, and practical consideration. These five aspects got high score in evaluation with the range 80%-100%. These five aspects met all criteria provided in the evaluation. Design and organization and practical consideration were two aspects which evaluated outside the material with the fulfillment was very good. While the other aspects which was got very good fulfillment were the contents of the book such as grammar, language skill, and speaking.

However, the other evaluation whose result was fair contained three aspects such as language content, listening, and writing. The range for fair percentage was 60%-79%. The evaluation on language content was the evaluation from grammar and vocabulary aspects. Moreover, listening and writing were two aspects which evaluated in deeper evaluation which gets fair result. While the rest aspects such as vocabulary and reading were considered as poor fulfillment. The range for two aspects which considered as poor fulfillment was under 50% of achievements.

REFERENCES

Cunningsworth, A. (2005). Choosing Your Coursebook. Oxford: Heinemann.

Davies, P., & Pearce, E. (2002). Success in English Teaching. New York: Oxford University Press.

Gerring, J. (2007). Case Study Research:Principles and Practices. Cambridge: Cambridge University Press.

Hutchinson, T., & Waters, A. (2003). English for Specificc Purposes: A Learning-centred Approach. Cambridge: Cambridge University Press.

Richards, J.C. (2001). Curriculum Development in Language Teaching. Cambridge: Cambridge University Press.

Sugiyono. (2012). Metode Penelitian Pendidikan. Bandung: Alfabeta.

Sukmadinata, N. S. (2012). Metode Penelitian Pendidikan. Bandung: Remaja Rosdakarya.

Wilkinson, D., & Birmingham, P. (2003). Using Research Instruments: A Guide for Researchers. London: RoutledgeFalmer.

[P1]Jeda paragraf terakhir dg batas bawah halaman terlalu lebar mas

Bisa diambilkan dr paragraf selanjutnya utk disambungkan supaya tabel tidak terpotong dua halaman

[P2]Dibetulkan setiap ada item2nya itu ada jeda 1 baris

[P3]Ini dibetulkan juga

[P4]Utk tabel 1 spasi saja mas, yang ini tadi sudah saya betulkan

[P5]Mas tolong semua tabel dibuat seperti ini jarak dari paragraf yang menghimpitnya