Validity Reliability Quantitative

Use a variety of sources to answer the questions, 2) not copy directly out of any book, 3) include sources to responses and cite them at the end of his writing, and 4) provide more details about how you applied his knowledge of research to his independent study.  His goal is to demonstrate through writing his research knowledge and that he applied that knowledge to his independent study.

Question # 5

Validity and generalizability aspects of quantitative, qualitative and mixed methods research:

Quantitative research

Validity and reliability are two important aspects in order to approve and validate the quantitative research. Moskal & Leydens (2002) defined the validity as “the degree to which the evidence supports that the interpretations of the data are correct and the manner in which interpretations used are appropriate”. In other words, Joppe (2000) stated that validity is the one if the obtained results are truthful and believable. He also adds that to determine the validity, the researchers generally poses a series of questions, and will often look for the answers in the research of others to know whether the measurements are accurate or not (p. 1). Joppe (2000) also defined reliability as how consistent the results are when the experiment is repeated a number of times under same methodological conditions, then the instrument is said to be reliable. Kirk and Miller (1986) identified three types of reliability referred to in quantitative research, which relate to: (1) producing the same results under same measurement conditions (2) the stability of a measurement with respect to time; and (3) the similarity of measurements in a given time period (p. 41-42). Charles (1995) argued that reliability is the consistency with which an individual’s scores remain relatively the same and can be determined through the test-retest method at two different times. HE refers to this type of instrument as stable instrument. He also stated that a high degree of stability indicates a high degree of reliability, which means the results are repeatable.

According to Creswell (2003) there are several threats of validity that proves or raises issues about the accuracy of the data or results or application of statistical tests to conclude the effects of an outcome. They are internal threats, external threats, statistical conclusion threats, and construct validity threats. Campbell and Stanley (1963) states that the basic requirement to interpret an experiment is to clearly define internal validity. Internal validity threats are experimental procedures, treatments, or experiences of the participants that threaten the researcher’s ability to draw correct inferences from the data in an experiment. These are raised due to usage of inadequate procedures like changing the instrument or a tool during an experiment, changing the control group participants under study etc. Because of these inadequate procedures, the experimenter should find whether the experiment make a difference in this instance or not.

External validity threats arise when the researcher concludes incorrect inferences from the sample data to other persons. It addresses the question of generalizability that to whom can we generalize the obtained results. A statistical conclusion validity threat arises when experimenters draw inaccurate inferences from the data because of the violation of the assumptions of the statistical test being used for the collected data. Construct validity threat arises when investigators use inadequate definitions and measure variables based on those inadequate definitions.

In general, methods of establishing validity in quantitative research are:

• Experiment review

• Data triangulation

• Participant feedback

• Regression analysis

• Statistical analysis

In my study, I clearly defined and took care of the internal validity threats that can arise from the data collected and the tools used for collecting the data. In order to avoid this threat in Microsoft Excel, as the random data generated keeps on changing, I copied the data on multiple sheets so as to make sure the data was not lost for further analysis. I took care of the external validity by inferring the data to their respective parts, products and machines correctly and obtained results after performing a careful analysis. I applied descriptive statistics analysis and drew line graphs between the numbers of products, delay time and interpreted the results clearly.

Qualitative research

Golafshani (2003) argues that the concepts of validity and reliability related to the quantitative research may not be applicable or support for qualitative research. Winter (2000) points out that the necessary tools such as precision; credibility and transferability are considered to validate the qualitative research. Transferability refers to the extent in which the obtained results can be used by other researchers. Credibility is nothing but the data, data collection and the results obtained are trustworthy and believable. All the technical issues such as validity, reliability, triangulation and generalizability show the effect of credibility. In terms of the quantitative research, reliability means, the result is replicable. But, in terms of the qualitative research, reliability is testing the information for high quality which otherwise looks confusing making the reader change the track of the study. Stenbacka (2001) relates that in the quantitative study, quality or reliability refers to evaluating the purpose of explaining, while in qualitative study, it serves the purpose of generating understanding of the information. Creswell & Miller (2000) stated that the strength of the qualitative study depends on the accuracy of the findings in view of the researcher, participant or the readers. Moskal & Leydens (2002) defined the validity as “the degree to which the interpretations of the data accurately describe the phenomenon under investigation”. I qualitative research, internal validity refers to the match between the researchers observations, interpretations, categories and reality. In addition to validity, Moskal & Olds (2002) used trustworthiness as an alternate method to validate the study which states that the made interpretations are accurate. Patton (2001) states that validity and reliability are two factors qualitative researchers should be aware of in designing, analyzing, interpreting, and judging the quality of study. According to Seale (1999), to ensure reliability in qualitative research, the research needs to examine the trustworthiness. He adds that trustworthiness of the research report becomes important if the validity and reliability of the study is discussed.

Testing or maximizing the validity of the qualitative research: Stenbacka (2003) suggested that the structure of documenting and doing a high quality research work leads to a generalizability if the validity and trustworthiness are maximized. To do this, a data triangulation method has to be implemented to control bias and to establish valid propositions. Creswell & Miller (2000) defined data triangulation as the formation of themes or categories using a validity procedure for convergence among multiple and different sources of information. It involves comparison of results obtained from different data methods such as surveys, interviews, observations etc.

So, to establish multiple ways of truth for a qualitative research, researcher needs to redefine, implement and test the validity, reliability, generalizability and data triangulation concepts thoroughly.

Methods for establishing the validity and to enhance the credibility of qualitative research are:

• Negative case analysis – Presenting the negative information that runs counter to the themes

• Audit trail – Clarifying the bias that the researcher brings to the study

• Prolonged field experience – Spending prolonged time in the field to develop an in-depth understanding of the phenomenon under study

• Data triangulation – triangulate different data sources of information by examining evidence from the sources and using it to build a coherent justification

• Member checking – to determine the accuracy of the qualitative findings

Mixed methods research:
Designs combining both qualitative and quantitative research methods to collecting, analyzing, interpreting and reporting data are called as mixed methods research. In mixed methods strategy, the researcher should consider both quantitative and qualitative validity strategies and mix both of these in a way that best works to build credibility and trustworthiness of the data, data collection, and out comes of the study.

Question # 2

Framework Elements of Research:
For every research proposal, a definite framework exists to follow a certain pattern. According to Creswell (2003), there are three different research approaches: qualitative, quantitative, and mixed methods approach. To understand these approaches fully, every researcher needs to consider three framework elements: knowledge claims, strategies of inquiry, data collection & statistical analysis called as methods.

My study falls under quantitative approach. The three framework elements that are related to quantitative approach are:

Knowledge claims:

Creswell (2003) defines knowledge claim as initiation of the project with certain assumptions about how the study will be learned and what the outcomes of the study are during the inquiry. Hunston (1993) suggests a researcher, to treat knowledge claim as an item in the research article. He also adds that the chosen item should be agreed by the community of that discipline. Dahl (2007) argues that the constructions of such claims are complex with the text sequences because; the concept of knowledge claim is not easily understandable. It should be originated from the critical thinking of the discipline that is embodied in the positional form. He also investigated on how the new knowledge claims are constructed for the present-day research article introductions in economics and linguistics. According to Creswell (2003), the different types of knowledge claims are post positive knowledge claims, socially constructed knowledge claims, advocacy or participatory knowledge claims and finally pragmatic knowledge claims.

When the research includes determining or reducing or an empirical observation and measurement or theory verification, then the type of knowledge claim is post-positivism. Popper (Wikipedia) supports that post positivists believe human knowledge based on conjectural thinking. Nicholas & Philips (2000) believed that thinking is not based on solid foundations rather they think as a prospective outcome believing that there is something real which we should find out. Creswell (2003) states that “Post positivism refers the thinking and after positivism; challenging the absolute truth and recognizing that we can not be “positive” about claims of knowledge when studying the behaviors and action of human”. Post positivism reflects in determining the effects or outcomes, examining the causes that reflect the outcomes by doing experiments, reducing the ideas into a small, set of ideas to test such as variables that constitute hypothesis and research questions, developing numeric measures of observations and studying the behavior of individuals. The problem studied by post positivist refers that there is a need to examine, and analyze the causes so as to interpret the outcomes. Post positivist can also be a reductionism where the variables of hypothesis and research questions are selected and carefully analyzed.

Knowledge claims that arise for my independent study are: Post positivism, believing that the delay time can be minimized in realty with the help of an algorithm. It is also believed that any production work order can be optimized with the help of developed algorithm. The knowledge claims that refer to my study are Post-positivism which includes combination of determination, observation and measurement of the delay time. The study involves recognizing the possible causes of a factor (work order) and determining the effects (delay time) of a factor by a careful observation and finally minimizing the effect or improving the performance (Productivity) of a system. The study involves development of a random data for careful observation of the delay time and analyzing the generated data in a real-time analysis for finding out the causes such as delay time and its effect o the productivity of the system. Then an algorithm is developed to test the behavior of the data so as to improve the performance of the system.

Strategies of inquiry:

A stage of inquiry in quantitative research includes numerical summaries, generalizations across populations and comparisons between populations. The design followed is experimental design in which the researcher examines how the phenomenon changes as a result of developing or implementing a method. Strategies of inquiry provide specific designs for procedures in the research design. Though strategy of inquiry is using from decades, has become more important with the increase in the computer technology, and the ability to analyze complex models. Strategies associated with quantitative research were those that invoked the post positivist perspectives. These include true experiments and less vigorous experiments called quasi-experiments and correlation studies, and specific single-subject experiments (Campbell & Stanley, 1963). He also states that, in this computer world, quantitative research strategies are involved with complex experiments that can analyze many variables and treatments witht eh help of factorial designs and repeated measure designs. Strategies associated with quantitative approach are:

A strategy of inquiry that is employed in my study is experiments. The experiments that I conducted for my study include visiting four manufacturing industries, observing the flow of products on different manufacturing production lines, and finally analyzing the type of model or an algorithm to be developed. Based on my observation, I see that the flow of parts on every assembly line is being obstructed by high process time’s parts causing a waiting of other products which resulting in high delay time for preceding products. This causes the performance of the system and ultimately affecting the productivity of the manufacturing production line. The observation is done in four manufacturing industries, each involving many numbers of assembly lines. Based on the observations data, I concluded that the part times and the product assembly times are different for different assembly lines and for different manufacturing industries, so I realized to work on optimizing the random part, product times which should be suitable to any production line. My study has comparison of the data sets with each data set consisting of 500 products of initial generated delay time with final delay time after incorporating the developed algorithm for the delay time. It is an experiment model which involves a creation of random data with Microsoft Excel and analyzing the various machine parameters with the help of Microsoft tools.

Research methods:

The most important element in the research process is the methods of data collection and analysis. For quantitative research, the research methods I used are predetermined instrument based questions such as performance data, attitude data, observational data and census statistical data using Microsoft Excel. My research problem is how to develop a universal algorithm to optimize the production work order. It involves identifying the factors that influence the outcomes, and testing the factors with the help of large data. The approach that matches to my problem is quantitative approach. To collect the data, I observed four industry assembly lines and generated a random data after a careful observation by using various closed-ended and open-ended questions and focuses on numeric data so that the results and interpretations are applicable to any type of industry assembly line.

The method of data collection for my study is using Microsoft Excel as an instrument that generates a random data which reflects to the data of current manufacturing industries production line consisting of different products with each product having different parts. The purpose of collecting a random data using Excel is to analyze test the algorithm whether it has a definite outcome so that the algorithm is applicable for any type of data or manufacturing assembly line. Then the analysis includes comparison of the delay times of the original data and the optimized order data for delay time. The analysis includes the implementation of algorithm and application of descriptive statistics.

• Question # 1

Purposes and benefits of sections of Independent study

My independent study is related to quantitative research model. My study deals with development of an algorithm to arrange production work order for minimizing delay time at on the assembly line. It purely deals with the quantitative random data and uses post positivist claims for the development of sound knowledge for delay time comparison and employs experiments, and generated data aster observing a variety of manufacturing assembly lines, strategies of inquiry Purpose of the quantitative research is to do numerical summaries, generalizations across populations and comparisons between populations. This research primarily relies on quantitative data. It includes few variables which are the building blocks (Delay time, work order, number of different products) and many cases or categories (Five sets of data with each set consisting of 500 products which are different). These quantitative research methods use experimental designs. Here, every chapter follows a typical pattern.

• Introduction

Purpose: The purpose of an introduction is to provide background information to the readers for the research reported in the study. A good introduction chapter motivates the reader and audiences to go a head a read the rest of the study. It provides a plan for the research, so that readers will be able to understand how the study is different and related to other research (Creswell, 2003 p.73). It establishes the issue leading to the research by conveying information about a research problem (Creswell, 2003 p.74). In this, the problem is addressed by understanding the factors or variables which are the process time, product order that influence an outcome – delay time (Creswell, 2003 p.75). It provides the understanding of the problem that explains or relates to an outcome – delay time and helps the researcher best understand and explain the problem why the delay time is to be minimized (Creswell, 2003 p. 76). The outline of my introduction chapter is

• Introduction – I introduced in brief about my study, providing a background of what it is with a little bit of literature so as to educate the reader.

• Statement of the problem in the study – I defined the problem of my study clearly with an example of a simple manufacturing line. Sub problems are also stated in addition to the main problem.

• Objectives – I listed out the objectives of the study that I am going to achieve by solving the above problem.

• Justification of the study – I gave a justification on for what I am working on this study and supporting what I say.

• Benefits of research – I listed out the benefits of the research, who are going to benefit by using this study in long term and in short term period.

• Assumptions, limitations and delimitations – I listed out various assumptions that I am going to consider, what limits my study in solving the above problem.

• Definition of terms and – I defined the important terms that the reader should be aware of or know before proceeding into the next chapter.

• Summary -Summarizing the chapter.

Benefits: By writing an effective introduction chapter, a reader can figure out the problem leading to the study i.e., how the delay time is optimized by arranging the production work order, reviewing the literature about the problem to find whether there are any related theories that is done by other investigators, identifying deficiencies in the literature about the problem, targeting audiences and notifying the significance of the problem for this audience (Creswell, 2003 p.73).

• Literature Review

Purpose: The purpose of the literature review is to find and learn more about the topic and check whether any researcher previously has made any study or research on the same topic or on the related topic (Creswell, 2003 p.29). If explained in detail of the previous related research, the reader will try to connect the study with previous studies by filling the gaps.(Cooper, 1984; Marshall & Rossman, 1999). It provides the importance of the study when compared to the prior studies and also benchmarks the results with other findings. After the careful analysis of the research topic with the help of past research, a clear definition of the problem of the study has been stated. All or some of these reasons may be the foundation for writing the scholarly literature into a study (Miller, 1991).

My research is all about how to optimize the assembly lines. I conducted a literature review on single-model assembly lines, mixed-model assembly lines. Then, I narrowed it down to the algorithms dealing with assembly lines involving the optimization or minimization of delay times so as to clearly understand the past research, problems of the past models and clearly define the research problem.

• Introduction- I explained what I am going to talk in this chapter. And also I will tell what the various important issues I am highlighting are.

• Assembly line – I talked about assembly lines literature, presenting the information related to my study on assembly lines and the prior researcher accomplishments.

• Delay time- I talked about assembly lines literature, presenting the information related to my study on assembly lines and the prior researcher accomplishments.

• Algorithms – I spoke about key algorithms as there are so many types related to the study.

• Summary- Summarizing the chapter.

Benefits: The literature review helps the researcher to revise the research idea and shows methodological techniques to problems specific to the research problem that will help in designing the study. It also helps to suggest possible questions or hypotheses that need to be addressed (Creswell, 2003 p.46).

• Methodology

Purpose: The purpose of methodology chapter is to bring focus on survey and experimental modes of inquiry. The researcher will explain which methodology he is using, why he chose that methodology and why he chose not to use other methods. In methodology chapter, the researcher discusses about the framework elements of research such as knowledge claims, strategies of inquiry, and methods. The method that was used to collect data is explained in detail like how did to generate random data of five sets with each set consisting of 500 products using Microsoft Excel ad procedure I followed to find the real-time delay time. The reader will exactly know what was done with the collected data, to the point that he or she can replicate the study to get similar results.

• Introduction- Here I talk about the previous chapter, by concluding some important points that are going to be used in this chapter.

• Restatement of the problem – I restated the problem clearly by narrowing the problem into a specific definite path so as to make a clear understanding to the reader after dealing with the prior research work, the accomplishments etc. Because, the stated problem before might not be clear as the research work wasn’t done at that time. After knowing the literature review part, the researcher analyzes the problem and states here clearly.

• Research design- involves framework elements of the research in order to determine the type of approach followed based on the framework elements of research. Then quantitative, qualitative and mixed approaches are compared. In addition to this the validity of the method is also discussed. After knowing the type of approach, proceeded to various steps like experiments, data collection. Ethical issues are also considered before planning for data collection.

• Instrumentation – the instruments for conducting an experiment are explained in detail. Types of instruments, specifications are clearly mentioned.

• Data collection – data is collected with the help of instruments and tools

• Analysis of the data – data is ready for the analysis and using statistical tools, or any other techniques are used.

• Summary- Summarizing the chapter.

Benefits: With the help of methodology chapter, audiences can recognize the variation that exists in the qualitative, quantitative and mixed method studies, why I chose a quantitative study for the research problem I defined then it advances general guideline for procedures of the study. These guidelines include a discussion about the general characteristics of the study if the audiences are not familiar with the approach to research.

(Source: Classroom material by Dr. Lynda Kenney)

• Results

Purpose: The purpose of results chapter is to present the analysis, the tools used for obtaining results so that the reader can understand easily and can be able to interpret and learn what has been done in the study. It needs to be organized in a step-by-step manner in such a way that the collected data, statistical tests, graphs, charts, are presented for support description for the reader to interpret quickly and accurately (Leedy & Ormrod, 2005).

• Introduction

• Application of algorithm

• Results

• Summary- Summarizing the chapter.

Benefits: The benefits of the results chapter is for readers to quickly interpret the conclusions and significance with the help of the tables, graphs, charts and figures obtained from the interpretation and analysis of the data.

• Flow Chart

Purpose: The purpose of the flow chart is to explain the process of optimization of the production work order in a step-by-step process that is described in the study. Sometimes, a simulation model can also be presented for easy understanding. It depicts an outline of the algorithm in a straightforward manner.

• Introduction

• Explanation of the developed algorithm -Flow chart

• Summary- Summarizing the chapter.

Benefits: It makes the readers understand easily at a glance what has done in the entire study. It also helps the researcher to make and follow the created flow chart while analyzing the data.

• Discussion

Purpose: The purpose of the discussion chapter is to highlight the main theories and conclusions used in the research study so that a reader can easily figure out what theories the researcher used in implementing and analyzing the data. Each major conclusion is clearly explained with the help of chart, graphs and tables and compared with the results of the similar work by other investigators. Then, the researcher continually connects her findings with the theoretical frameworks. Any new or unusual results are also explained (Leedy & Ormrod, 2005). If the researcher is not sure about the significance of the results or could not understand the phenomenon of the data, he presents a speculative discussion outlining several possible outcomes by alerting the readers that such a discussion is speculative. Finally, he outlines the important results he thinks of with the study.

• Introduction

• Discussion of the results

• Advantages of the results

• Outline of important results

• Summary

Benefits: The benefits of the discussion chapter are to make grand conclusions which support the subsequent paragraphs. Here, the entire conclusions, implications or the effects due to each conclusion including the minor and major effects are presented. The discussion also includes the method of computation or derivation of the study. Such situation arises when one figure is derived from preceding figures. If the application or method is involved, then a complete example with the method is to be explained for complete understanding to the reader. Finally, explained the significance and outcomes of the study.

• Conclusions and Recommendations

Purpose: The purpose of the conclusions chapter is to make a summary of the conclusions in reference to the objectives and the problem stated in introduction chapter. The researcher also points out both what are found and what are not found. It is also the section examined by the prospective reader with limited available time (Leedy & Ormrod, 2005).

• Introduction

• Restatement of the problem

• Conclusions

• Recommendations

Benefits: Although the researcher has previously presented each of the conclusions, conclusions chapter tell us the reader the ultimate effect or the benefit of the study. In my study, I have explained how the delay time is minimized or optimized by arranging the production work order so that it is quite helpful to readers, who might easily lose track of some important conclusions as they read earlier portions of a study (Leedy & Ormrod, 2005). In addition to this, a prospective reader will able to quickly examine the research in limited time. I explained the benefits of my study that findings of this research will aid industries, retails stores by demonstrating how the algorithm is currently used, and how retail stores can assist customers to implement universal algorithm. Industries may benefit from models of evaluating arrangement of parts of a product on an assembly line.

REFERENCES

Leedy, P. D., & Ormrod, J. E. (2005). Practical Research: Planning and Design. PEARSON. Columbus. Ohio.

Creswell, J. W. (2003). Research Design: Qualitative, Quantitative and Mixed Methods Approaches. Second Edition, SAGE. Thousand Oaks. USA.

Olds, B., Moskal, B. & Miller, R. “Assessment in Engineering Education”, Journal of Engineering Education , to appear Jan. 2005.

Moskal, B., Leydens, J. & Pavelich, M. “Validity, reliability and the assessment of engineering education”. Journal of Engineering Education , Vol. 91, No. 3, 351-354, 2002.

Leydens, J., Moskal, B. & Pavilich, M. ,”Qualitative Methods Used in the Assessment of Engineering Education”. Journal of Engineering Education , 93(1), 65-72, 2004.

Hunston, Susan, 1993. Professional conflict. Disagreement in academic discourse. In: Baker, M., Francis, G.,

Tognini-Bonelli, E. (Eds.), Text and Technology. In Honour of John Sinclair. John Benjamins, Philadelphia/Amsterdam, pp. 115-134.

Dahl. T, 2008. Contributing to the academic conversation: A study of new knowledge claims in economics and linguistics, Journal of Pragmatics 40, 1184-1201

D.C. Philips & Nicholas C. Burbules (2000): Postpositivism and Educational Research. Lanham & Boulder: Rowman & Littlefield Publishers.

Campbell, D. T. & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research.. Chicago: Rand McNally.

Kirk, J., & Miller, M. L. (1986). Reliability and validity in qualitative research. Beverly Hills, Sage Publications.

Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research.

The Qualitative Report, 8(4), 597-606.

Winter, G. (2000). A comparative discussion of the notion of validity in qualitative and quantitative research. The Qualitative Report, 4(3&4).

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5(4), 465-478.

Stenbacka, C. (2001). Qualitative research requires quality concepts of its own. Management Decision, 39(7), 551-555

Patton, M. Q. (2002). Qualitative evaluation and research methods (3rd ed.). Thousand Oaks, CA: Sage Publications, Inc.