School Inspections Improve School Quality Education Essay
Introduction
This critical appraisal of a research article entitled “Do school inspections improve school quality? Ofsted inspections and school examination results in the UK” looks at the research methodology, style and approach adopted by the writer, Leslie Rosenthal. The article has been chosen as it discusses whether the inspection process, that has been designed to improve standards in schools, actually make exam results worse, in particularly in the year during inspection. If the researcher’s theory is found to be correct, this would lead to questions about the effectiveness, and value for money of the inspection process.
The objectives for carrying out a critical appraisal of this piece of research are to identify
the research approach that has been adopted
the research style that has been used
the intended research population
how information will be gained from the research population
the research instruments used
any other considerations such as validity, reliability and credibility
Each of the objectives will be discussed in turn, making explicit reference to the research article and current research methods literature. A summary of findings will be presented at the end of the assignment.
Background
Leslie Rosenthal (2004) states that “Improving the quality of state-funded secondary school education remains a major explicit policy aim in the UK and is the central means by which state-provided education in the UK is now regulated, though a system of inspections carried out by the Office of Standards in Education (Ofsted), set up in 1992”. Research has been conducted which looks at the exam results of state-funded ,non-selective English secondary schools that had been inspected during the year of the exams.
Rosenthal, L. (2004) describes the research as ’empirical research’ several times throughout the text. The data used for the research has come from publicly available material based on the exam success of 15-16 year old students in state-funded school. This measure of performance, from secondary school performance tables, is used to test the proposition that in the year of an Ofsted inspection, the measure of the quality of the school changes.
Empirical research is described as research of any phenomenon that results in outwardly measurable criteria, or can be verified by observation or used to derive a particular interpretation, (Opie, C. 2006). Empirical research lends itself to quantitative approaches in educational research in order to lead to generalisability. This means that the research findings can be applied to other settings than where they where originally founded in order to show that the statement is true (CEM, 2009).This could be important to empirical or experimental research where the research design may not be ethical or practical in the required setting, but the results could be compared from another setting. Empirical research is described by Verma and Mallick (2004) as the starting point to a research problem, with a view to explaining important factors or what happened after the research has been carried out. This implies that the research has been carried out with an open mind and results should not be corrupt or biased.
Rosenthal carried out empirical research using data from the year previous to, and post Ofsted inspection to confirm the validity and generalisability of the results obtained (these terms will be discussed in more detail under objective 6). The purpose of this is to show that there exists a small negative effect on exam results during the year that Ofsted inspects a school, even though the general philosophy of the agency’s approach is described by its own motto as “Improvement through Inspection”, and one of their four objectives is to raise standards of achievement by students in exams (Ofsted, 1995).
Objective 1 – Identify the research approach that has been adopted
The overall approach to the research that was carried out by Rosenthal was a quantitative approach. Bryman, (2008) describes quantitative research as deductive and testing theory rather than being inductive or generating theory as with qualitative analysis. The theoretical and conceptual structure of the research is determined before testing it through empirical observation. Quantitative research is not merely number crunching but is the use of theory to deduce a hypothesis. Primary or secondary information is then gathered to test the hypothesis. Primary information research involves gathering new data by means of written questionnaire, survey, or face to face and telephone interviews.
However, Rosenthal used secondary information, which is research or information that has already been collected by somebody else, in this case, Government officials collecting data about schools to measure attainment and performance.
To carry out quantitative research, any information, whether from primary or secondary sources, needs to be processed in some way in order to quantify it. Rosenthal has used statistical measures to analyse the secondary data and show the effects of Ofsted inspections in school quality.
Opie, (2006) describes quantitative research as being objective and positivistic as opposed to subjective and anti-positivistic. Objectivity can be defined as unbiased and open-minded, and that only one interpretation or meaning can be drawn from the data, where as a subjective point of view is regarded as belonging to an individual as a result of their own consciousness or thought. Subjectivity is not based on facts that can be demonstrated or observed, but could in fact be interpreted in different ways. This piece of research has been based on facts from secondary information, and independent variables that could have a positive or negative effect on results have been tested to ensure that the results are unbiased and can not be interpreted in different ways therefore demonstrating the quantitative nature of the research.
Objective 2 – Identify the research style that has been used
The style of research that has been adopted by the researcher is ‘Evaluation Research’. According to Burton & Bartlett (2005), evaluation research is used to assess the effectiveness of specific projects to see if the original aims have been met. Rosenthal states that the objective of the research is to investigate the direct influence of Ofsted inspections on exam performance of inspected schools for the year of the inspection, as one of the four objectives for inspection is to raise the standards of achievement by students (in exams). In other words, the research has been carried out to evaluate how effective the inspection process has been at raising student achievement in exams at GCSE level.
Evaluation research is often funded by government in order to evaluate the effectiveness of policies or procedures, and can be broken down into formative and summative evaluation.
According to Trochim and Donnelly (2007) formative evaluation research is carried out during a program or activity in order to form the direction or strengthen the outcomes of the activity. The formative research is used as a method of improving the quality of an activity by looking at the needs, conceptualisation, implementation and processes of the activity. Summative evaluation research looks to summarise the activity in terms of meeting objectives, cost-analysis and impact. The research may be conducted to improve performance or processes if the activity is to be repeated, or to carry out secondary analysis, which re-examines existing data to answer new questions that may have arisen from the activity. Rosenthal’s research is summative as it is not intended to improve the processes of inspection, but to summarise whether or not the inspection has met its own objectives of raising standards of achievement in exams.
Objective 3 – Identify the intended research population
The indented research population is clearly stated by Rosenthal, in section 4 of the report as ‘state-funded, non-selective English secondary schools that serve students up to 16 years. State-funded schools that are selective on the basis of religion (voluntary aided, voluntary assisted and special agreement schools) and/or on the basis of academic ability (grammar and secondary modern, and selective grant maintained schools) have been excluded from the study. Only those schools that were inspected under Section 10 of the 1992 Education (Schools) Act were included in the study and not those classed as “failing” or having “serious weaknesses”. Data was gathered from approximately 2,300 schools inspected during the years 1993/4 to 1997/8 with some 2,700 Ofsted inspections taking place. Under Section 10 of the 1992 Educations (Schools) Act, schools were to be inspected at least once in every four year cycle; therefore around one quarter of the 2,300 schools would be inspected each year, during the four year research period. The exact figures have been shown in table form in section 4 (Data and method) by Rosenthal.
Objective 4 – Identify how information will be gained from the research population
The information needed to carry out the empirical research was, and still is, freely available in the public domain for use by Rosenthal. Rosenthal references the web address for Secondary School Performance Tables (or Achievement and Attainment Tables as they are now called) and these tables are published annually on the Department for Children, Schools and Families website (http://www.dcsf.gov.uk/performancetables/) and in national newspapers. Information contained in these performance tables includes the official measures of exam success of 15-16 year old students at GCSE (General Certificate of Secondary Education) level or equivalent. The data collected focuses specifically on the number of GCSE passes at grades A* – C in each school year of the research period 1993/4 to 1997/8.
Objective 5 – Identify the research instruments used
Secondary data analysis has been carried out by Rosenthal. As discussed in objective one, secondary data is information that has been gathered previously by another person for some purpose, rather than information which is being gathered for the first time as with primary data. The data that Rosenthal used was gathered by the Department for Education and Skills (DfES), and published in the annual league tables that shows the performance of a school in terms of attainment and achievement, pupil attendance and proportion of students with Special Education Needs (SEN) [DCSF accessed on 4th March 10].