Implementing and Measuring Effective Organizational Training and Development

Training and development programs have been found to be essential to organizational effectiveness. However, a training program that cannot demonstrate improved organizational effectiveness is a waste of valuable time and money. This paper reviews the literature concerning critical elements of an effective organizational training program. Practical methods are presented in the areas of planning, implementation, and evaluation. Specific guidelines for developing an organizational analysis and training needs assessment are provided for consideration in the planning phase. Guidelines for providing training objectives, a favorable training and learning environment, effective training techniques, and strategies that ensure effective transfer of training are provided for consideration in the implementation phase. Recommendations for thorough training evaluation procedures are provided for consideration in the evaluation phase.

Implementing and Measuring Effective Organizational Training and Development

Organizations are composed of people with varying knowledge, abilities, skills, attitudes, and social behaviors that are utilized to move the organization towards its mission and goals. In order for organizations to improve and grow, the people who make up the organization must also improve and grow. Knowledge, skills, and attitudes must be developed in order to maintain a workforce that remains competitive in today’s marketplace (Cascio & Aguinis, 2005). Training and development programs have been found to produce positive results in industrial development and organizational performance and have been shown to produce an increase of employee productivity, wages, and employment longevity (Huselid, 1995; Colarelli and Montei, 1996; Kitching and Blackburn, 2002; van Eerde, Tang, & Talbot 2008).

Organizations that provide ongoing opportunities for employee development and growth place themselves at a distinct competitive advantage over those that do not. They recognize that training is a necessary component of corporate culture, where commitment to training and development is shared from the top down, and necessary time and funds are invested as needed. They ensure that training is linked to organizational objectives and strategies, and is designed to positively affect the company’s bottom line. Continuous feedback is provided through structured quality improvement practices, and the organization remains flexible to restructure systems that improve performance (Cascio & Aguinis, 2005). Training within an organization costs valuable time and money. Billions of dollars are spent by organizations each year in formal and informal training programs. Like everything else in business, it is essential to develop an organizational training and development program that provides the greatest possible return on investment (Cekada, 2010).

This paper will present a review of the literature to explore three phases of an effective organizational training and development program: the planning phase, the implementation phase, and the evaluation phase. The planning phase includes completing an organizational analysis, task analysis, and a person analysis, and utilizing a training needs assessment to determine specific training needs that are in sync with the organizations mission, goals, and cost constraints. The implementation phase includes the design of the training program that will most effectively meet the needs assessed. The evaluation phase includes the development of criteria and use of comprehensive evaluation models to determine the trainings validity and effectiveness.

Phase One: The Planning Phase

Training Utility

Training utility refers to the benefit that the training program has on organizational effectiveness. Training has been shown to be beneficial to an organization. However, if what is trained does not improve organizational effectiveness, it becomes at best a poor investment and at worst detrimental to progress. A training program is beneficial to the organization to the extent that it meets the assessed needs of the organization (van Eerde, Tang, & Talbot 2008).

Organization-Task-Person-Demographic and Values Analysis

The first task of an effective organizational training program is to determine the specific needs of the organization. A comprehensive analysis of organizational systems and their workforce is necessary to determine relevant training needs and develop a training system to meet those needs (van Eerde, Tang, & Talbot 2008). Morano (1973) recommends indentifying and surveying key leaders in each area who are as he describes are in “the best position to know” what training is needed most in their prospective areas. Morano goes on to suggest conducting a “manpower analysis” and an “organizational analysis” to determine needs that could be met through training. Although Morano’s recommendations are broad scoped, he grasped the importance of conducting a thorough analysis of what is needed and how those needs could be met through training rather than simply providing training for trainings sake (Morano, 1973).

The core model for assessing training needs has not changed much since McGehee and Thayer (1961) developed their Organization-Task-Person framework. The organizational analysis is designed to determine how training personnel can help the organization reach its goals and fulfill its mission (McClelland, 1993). Two major themes of the organizational analysis are linking training to corporate strategy and maintaining the workforce’s technical relevance (Latham, 1988). The task analysis determines what personnel must learn in order to execute their job well, and the person analysis determines who will need training and for what reason (van Eerde, Tang, & Talbot 2008). The emphasis of a task analysis should also be on what is required by the employee in the future to be effective. In addition to the organizational-task-person analysis, Latham recommends performing a demographic analysis to determine the needs of specific subgroups to include age, gender, and management levels (Latham, 1988).

Prior to evaluating organizational strategies such as company philosophy, mission statements, objectives, strengths and weaknesses, organizational design, and programs that implement the strategies, O’Reilly and Pfeffer (2000) recommend beginning with defining the organizations basic principles and values. The next step is to evaluate management practices to determine the extent that those principles and values are modeled and reflected throughout the organization. Practices that do not embrace these basic principles and values are changed or discarded. This process helps to align the company’s objectives with its basic principles and values as well as with the emotional and intellectual behavior of its employees in a way that can be used on a day-to-day basis (O’Reilly III & Pfeffer 2000).

Training Needs Assessment

A needs assessment is a structured way to determine organizational needs that can be met through training. The function of the training needs assessment is to determine why training is needed, what exactly should be trained, and who should receive it in a way that creates the greatest benefit for the organization. One of the purposes of a training needs assessment is to ensure that the training process actually is designed to meet organizational goals and needs that are assessed so that training is most likely to increase the probability of organizational performance (van Eerde, Tang, & Talbot 2008).

A training needs assessment is necessary to define goals, determine who will make up the assessment group, determine resources and financial commitment needed, review and recommend the most effective assessment methods and tools, determine time frames, schedule and implement assessment procedures, gather and analyze feedback, determine conclusions, and present findings and recommendations to the organizational leaders (McClelland, 1993). It is a structured and thorough way of determining organizational training needs. A good training needs assessment is the first step in creating an effective training program. A training needs assessment will clarify organizational goals and needs, stimulate creative thinking, formulate effective training strategies within budgetary constraints, provide increased management and participant commitment, and illuminate the most effective methods for determining and providing the training and development needed (Cekada, 2010). An effective needs assessment may also identify problems that may not be solved through training but instead require adjustments to policies, procedures, or practices (Brown, 2002). An effective training program is not one that provides the most training hours but one that is aligned with the organizations needs as defined by a thorough training needs assessment (van Eerde, Tang, & Talbot 2008).

Data collection can be accomplished through a variety of methods. Surveys and questionnaires are inexpensive, can be completed in a short time frame, and are easy to summarize. Interviews can reveal attitudes, problems, and potential solutions. Performance appraisals can identify skill deficiencies and training needs. Observations can provide data in vivo without interrupting the workflow. Tests can assist in determining deficiencies in knowledge or skills and are easily quantifiable and reportable. Assessment centers can assist in determining management training needs. Focus or discussion groups can increase participant interaction and commitment to change. Document reviews can help determine needs and can easily be collected and analyzed. Advisory committees can utilize key players to provide inside information and needs (Brown, 2002). Teams and supervisors can be utilized to provide ongoing data collection and submit results on a regular basis to HR for training recommendations (van Eerde, Tang, & Talbot 2008).

Phase Two: Training and Development

Create Training Objectives and Sequencing

Upon the completion of a thorough training needs assessment, the identified needs can be translated into clear behavioral training objectives. A key to successful training involves creating training objectives that guide the training process and specify measures that accurately evaluate results. Training objectives should clearly describe the desired behavior to be learned, the context in which the behaviors should occur, and the behavioral criteria that can be observed and measured upon completion of the training. Objectives should be listed in sequence as they are needed on the job or by importance. Reasons for sequencing should be made clear to the learner at the beginning of training (McConnell, 2003). In other words, if a particular task requires several sub-tasks to complete, it is important to sequence learning objectives for each sub-task first before working on the overall task (Cascio & Aguinis 2005).

Create an Effective Training and Learning Environment

With the development of internet and intranet technologies, computer-based instruction has provided an alternative to the traditional brick-and-mortar classroom instruction. Most companies today use the Internet to provide various levels of employee training. In a study of use of the Internet for organizational training, it was found that learners in blended learning environments that involved on-line and face to face interactions had more control over where and when they engaged in the learning process and had a wider variety of learning tools. As a result, students were more motivated to learn, and achieved higher scores over those in a traditional face-to-face classroom (Klein, Noe, & Chongwei, 2006). Virtual team training is relatively new and has not demonstrated the same success. In fact only 7% of respondents in a survey of over 400 human resource professionals reported current virtual team member and leader training to be very or extremely effective. More effective training is needed to provide skills necessary to select and use appropriate communications technologies, and to lead and manage virtual teams (Rosen, Furst, & Blackburn, 2006).

However, whether training is conducted in a traditional classroom, on the internet, or on the job, certain principles should be in place to ensure a productive learning environment. For instance, trainees should have a clear understanding of the training objectives and the expected outcomes. Training content should be meaningful and relevant to the job currently being performed. Trainees should be provided a variety of materials that stimulate various learning modalities. A sufficient amount of practice must be provided. Trainees should receive and be able to provide feedback on what is being learned, and should be able to observe and interact with other trainees (Noe & Coquitt, 2002). Others principles include providing for trainees with diverse skill levels, ensuring commitment and buy-in to training on all levels of management, and utilizing commitment strategies to increase participant buy-in and motivation (Tall & Hall, 1998).

Utilize Effective Training Techniques

A multitude of training methods or techniques exist and those chosen should be based upon the training needs assessment. Examples of training methods include group training, team training, coaching, mentoring, self-paced learning, e-learning, computer-assisted learning, distance learning, self-study, simulations, lectures, job shadowing, job rotation, and behavioral modeling. Group training occurs when three or more employees are engaged in learning common objectives. Many organizations provide group training to review HR policies and procedures and safety practices. Team training is comprised of members of a particular team with objectives on how to effectively work together as a team in order to fulfill the team’s objectives. Coaching is a training technique often used in on the job training and is usually more individualized to the employee. Mentoring occurs when more experienced employees train newcomers to adapt to the organizations culture or environment. Self-paced learning is comprised of self-contained units of instruction that the employee can review and complete at their own pace. Self-paced learning can also be computerized so that employees can complete units of instruction at their own pace via their own computer. E-learning is on-line instruction that can be utilized individually or as a group. Distance learning is utilized to provide instruction from teachers who are in other parts of the world. Distance learning can occur through video conferencing, telephone conference calls, or internet access. Simulations are provided in a laboratory or on computer and simulate the actual job that is to be performed and are usually complete with immediate feedback mechanisms to enhance learning. Job assignments and rotations often occur on the job for a specified amount of time. The shadowed employee is able to learn various aspects and multiple components of the job by asking questions and observing effective modeling in vivo from the job shadower (McConnell, 2003). Both formal and informal on-the-job type training are important for training objectives to be acquired and generalized (Verhaest & Omey, 2010).

Behavior Modeling

The behavior modeling technique based upon Bandura’s (1977) social learning theory is one of the most commonly used and researched learning methods used in organizational training. Behavior modeling involves providing clear behavioral objectives, using a model or models to demonstrate undesired and desired behaviors in various contexts, providing opportunities for learners to practice what was modeled, providing learners with clear feedback and reinforcement, and implementing procedures that ensure maximum transfer of what is learned to the actual job. A meta-analysis of 117 studies found that although declarative knowledge decreased over time, skills and behaviors remained the same and often increased with the use of behavior modeling. Skills increased when the full use of behavior modeling technique was utilized, learning points were emphasized, and training time was longer. Desired behavioral transfer to the job increased when both negative and positive models were utilized, behavior modeling contexts were created and practice by trainees, both trainees and managers were trained in the behavior model process, and effective behavioral contingencies were applied in the actual work environment to reinforce learned behaviors (Taylor, Russ-Eft, & Chan, 2005).

Ensure Effective Transfer of Training

Transfer of training refers to the how well an employee utilizes and generalizes what was learned in training to the actual job where it is needed (Cromwell & Kolb, 2004). If behaviors learned in training cannot effectively be demonstrated and increased on the job where they are needed then the cost and time spent in training was wasted (Velada, Caetano, Michel, Lyons, & Kavanagh, 2007). Cromwell & Kolb (2004) found significant transfer of training occurs when training is supported throughout the organization, supervisors are supportive during and after the training process, and trainees participate in peer support networks for a period of at least one year. Lack of time, supervisory support, and management’s commitment to the process are noted as barriers to an effective transfer of training (Cromwell & Kolb, 2004).

General considerations that will help to ensure transfer of training include:

Organizational and management support of training and transfer of training.

Training contexts should match job contexts.

As much experiential training as possible should be provided.

Supervisors should recognize and reward learned behaviors when demonstrated on the job.

A peer support system should be developed that encourages and demonstrates support for the use of behaviors learned in training that are demonstrated on the job (Machin, 2002; Martin, 2010).

Include self-management as part of training to reduce relapse of old ineffective behaviors. Self-management training involves setting personal goal, identifying attitudes and behaviors that interfere with or enhance success, creating plans to overcome interfering obstacles, developing self monitoring systems that provide observation of progress towards the plan, and the utilizing personal contingencies that reinforce effective behaviors, block ineffective behaviors, and increase motivation to utilize behaviors learned in training (Wexley & Baldwin, 1986; Gist, Bavetta & Stevens, 1990).

Recognize that self-efficacy, self-management, and performance levels may drop significantly after training, but increase to the same levels after training about one year later. Allow at least one year to transpire before evaluating transfer of training effects (Cromwell & Kolb, 2004; Vermeulen & Admiraal, 2009).

Phase Three: Evaluation

Training evaluation should occur throughout the training process. Evaluation provides information necessary to adjust, or improve effective training elements, and eliminate ineffective elements of current programs. On a practical note, a successful evaluation design has the potential to justify the existence of an organizations training and development department (Tsang-Kai, 2010).

A thorough training evaluation requires collecting and analyzing training data and results in a manner that will effectively instruct future training and development content and practices. Because this type of evaluation costs considerable time and money, many organizations revert to utilizing trainee responses from post training surveys. However, this is only one dimension of a thorough training evaluation and should not be used alone to determine training effectiveness. An effective training evaluation requires the establishment of controls, accurate methods of measurement, educated evaluators, statistical analysis, and clear evaluation criteria to determine the effectiveness of training transfer. A common problem associated with comprehensive training evaluations is that many organizations perceive them as being more theoretical than practical. In fact, one study showed that it is common for most companies to evaluate employee reactions. However, only half evaluated employee knowledge, about a fourth evaluated the transfer of training to the workplace, and only a small percentage evaluated the actual impact of the training on the organization (Tsang-Kai, 2010).

The Evaluation Design

It is commonly held that training in and of itself does not guarantee improved productivity, satisfaction, longevity, or organizational effectiveness in the workplace. Effective transfer of new knowledge, skills, and attitudes requires a variety of structural contingencies to reinforce effective behaviors and practices and to prevent relapse of ineffective habits. Training evaluators must develop outcome criteria that will accurately forecast intended results and inform future training activities (Dionne, 1996).

The evaluation design must attempt to determine the extent of change demonstrated in the level of performance that has occurred and to what extent that level of performance meets the organizational need. Standard outcomes of the study may require the use of formal experimental designs that measure statistical significance and effect size for example between an experimental and control group posttest means (Sackett & Mullen, 1993).

It is important to determine whether the requirements of the evaluation call for the measurement of change, the measurement of achievement, or both. Measuring change may be used to estimate the utility of training, provide evidence of training effectiveness, or demonstrate the monetary value of training by pairing past training with past success. The measurement of achievement involves comparing the efficacy of two different training programs to see which is more effective. If the training is only offered once, the program designers do their best to estimate the most effective approach given the budgetary and time constraints. If the training is offered repeatedly, the long term costs and constraints make comparative research more worthwhile. Finally, the evaluator who wishes to contribute to the global literature of understanding the training processes may wish to utilize a formal experimental design. Regardless of the design and methods of evaluating the training program, it is important that the evaluators “sell” organizational leaders on a thorough and useful evaluation of the training process (Sackett & Mullen, 1993).

Criteria for measuring training and job performance success

One particular model that has endured with widespread popularity, perhaps because of its practical simplicity, is Kirkpatrick’s (1959) four level training evaluation criteria model. This model has been utilized by organizations as a guide for the evaluation of training programs since it was first introduced in the late fifties (Alliger & Janak, 1989). Thirty years later Kirkpatrick (1996) maintains the value of his model, and continues to recommend his procedure for measuring the utility of the training program. Kirkpatrick’s model measures four levels of training: reaction, learning, behavior, and results. On the reaction level, the goal is to measure participants feelings about the training experience. By asking questions regarding participant’s reactions, the organization demonstrates that they are interested in the employee’s needs and are willing to do what it takes to meet those needs. Evaluating reactions to training provides a way to attend to participant’s motivation to learn. Failure to attend to motivation can easily create failure of learning, and thus failure of the training program. The second area to measure is learning. Did training create and/or increase the desired knowledge, skills, abilities, or change in attitudes? The third area to measure is behavior. Did the training create the desired change of behavior in the specific context of the work environment where it is needed? The final area to evaluate is the results. This is a measure of the effect that training had on the organization such as increased sales, higher productivity, improved quality, increased employee satisfaction, and decreased turnover. The evaluation process becomes more difficult at each level (Kirkpatrick, 1996).

Some researchers have noted problems regarding unclear training evaluation criteria in Kirkpatrick’s model and recommend revision, while others argue for entirely different models. One concern is that the framework of the model is not based upon modern theories of learning. Another concern is that Kirkpatrick over simplistically treats multi-dimensional constructs such as trainee reactions as one dimensional. Further concerns exist regarding basic assumptions suggested by the level model. For instance it is assumed in the model that evaluation levels should be arranged in ascending order for maximum effectiveness. It is also assumed that each level is causally linked to the other, and that each of the levels is positively intercorrelated. However, despite these and other concerns, the model remains one of the most utilized approaches among practitioners and researchers. (Alliger & Janak, 1989).

A meta-analysis of 34 studies and 115 correlations demonstrated significant reliabilities between training criteria using an augmented framework for training criteria based on Kirpatrick’s model. Trainee reactions were shown to be important. However, simply liking training did not necessarily transfer to performing what was trained in the workforce. However, although positive reactions to training do not guarantee transfer, negative reactions have been shown to have a significant negative affect on the training department. Questions on utility appeared to provide the best indicators of training transfer, and attentiveness to content validity was shown to be important when developing criterion strategies to minimize the potential for misinterpreting criterion relationships (Alliger, Tannenbaum, Bennett, Traver & Shotland, 1997).

An alternative measurement model was developed by Kraiger (2002) to overcome the deficiencies of Kirkpatrick’s (1959) four level model (Kraiger,). This model is based upon Kraiger’s earlier theoretical model of learning, and addresses three distinct learning outcomes: cognitive, skill-based, and affective. According to Kraiger, cognitive outcomes should measure verbal knowledge, knowledge organization, and cognitive strategies. Skill-based outcomes should measure compilations of proceduralization, composition, and automaticity. Affective outcomes should measure attitudinal outcomes and motivational outcomes such as motivational disposition, self-efficacy, and goal setting (Kraiger, Ford, & Salas, 1993).

In his later developed evaluation model, Kraiger makes a clear distinction between evaluation targets and data collection methods. Evaluation targets include training content and design, changes in learners, and organizational payoffs. Targets and methods are linked to available measurement tools. Evaluation targets focus on the implementation of the training design, training delivery, and validity. Measurement tools include the use of advisory panels, established methodologies, judgments from experts, and course ratings. For changes in learner’s targets the focus of implementation is on specific cognitive, behavioral, and affective change. Measurement tools include written tests, work samples, interviews, and surveys. For organizational payoff targets the focus of implementation is results, performance, and transfer. Measurement tools include cost-benefit analysis, ratings, and surveys. Each of the three target areas of evaluation informs the other in an interactive feedback loop (Kraiger, 2002).

Analyzing the benefits and costs of training

Organizational leaders may become concerned about training costs when there appears to be no increase in revenues or particularly when revenues appear to be declining. They may need to see that training programs are producing some sort of rate of return on their investment. In order to clearly demonstrate to administrators that the benefits of training outweigh the cost, it is important to include a cost analysis in the presentation of the training program evaluation (Clarke, 2002).

The cost-benefit analysis is the most widely used method of calculating return on investment (ROI). Using this method, return on investment is calculated by dividing the monetary value of benefits incurred by training with the actual costs of training. If the ratio exceeds 1, then the benefits are shown to outweigh the costs of the training program. Another model used to calculate ROI subtracts the monetary costs of training from the monetary value of benefits incurred. In this method, the ROI increases proportionately to the increase in the difference between calculated benefits and costs of the training program. Some organizations recommend anywhere between a 20-40 percent ROI, while others suggest that training benefits should at least outperform the rate of inflation. The net present value (NPV) is yet another method to evaluate training investment. In this method, training benefits are assigned an expected rate of return. If the net benefits are positive after deducting training costs, the training is considered successful (Murray & Efendioglu, 2007). A thorough cost analysis should seek to clearly demonstrate that the training program provides a net value to the organization in areas such as increased employee satisfaction and productivity, decreased employee turnover and recruitment costs, decreased penalties for compliance violations, and reduced insurance premiums (Clarke, 2002).

Summary

Training and development programs have been found to be essential to organizational effectiveness. However, a training program that cannot demonstrate improved organizational effectiveness is waste of valuable time and money. This paper has reviewed the literature concerning critical elements of an effective organizational training program that include thorough planning, implementation, and evaluation.

The planning phase is designed to avoid training simply for trainings sake and to ensure that the training program meets the assessed needs of the organization. The planning phase should include a thorough analysis of the organization, tasks to be learned, people that need to learn them, important demographic concerns, and values. A thorough training needs assessment is a valuable tool to determine training needs through a systematic evaluation of each of these areas.

The training and development phase is designed to create specific training objectives and sequential learning opportunities. It is important in this phase of development to create a favorable training and learning environment and utilize the most effective training techniques that will provide the maximum potential to transfer objectives into outcomes in the workplace where they are needed. Because training without transfer is a waste of time and money, the training development phase should design and implement strategies that will ensure the effective transfer of training.

Training evaluation should occur throughout the training process. Evaluation provides information necessary to adjust, or improve effective training elements, eliminate ineffective elements of current programs, and justify the existence of an organizations training and development department. A thorough and systematic evaluation design is necessary to ensure that what was trained increased the effectiveness of the organization. The design should provide target measurements for training content and design, changes in learners, and organizational payoffs that analyze the benefits and costs of training. Further research is needed to provide more accurate, practical, and cost effective methods to measure organizational training and development programs.