This paper addresses the use of total quality management principles in the evaluation of aviation training outcomes. The paradigm shift to quality management in aviation training is explored in four areas: quality management implementation, quality management tools for training managers, quality management strategies for training, and the quality training culture. Together, the four areas create a system and a process for establishing total quality management at the training manager level for the evaluation of aviation training.
Items discussed in detail include training nalysis methods, training design processes, training objectives, training assessment, the economics of training, and continuous improvement. The paper concludes with recommendations for training managers drawn from experience and from the literature. Introduction Background: The concepts of total quality management (TQM), statistical process control (SPC) and continuous quality improvement (CQI) have evolved and flourished over the last five decades. Pierce, 1991)
Though many improvements and adaptations are sure to characterize their continuing evolution, arguments can also be made that TQM, SPC and CQI are in a mature state. This does not mean that these processes have outlived their usefulness by any means. They have proven their worth and have been refined to maximize their usefulness. TQM, SPC and CQI will be discussed collectively as “quality management. ” The training of people is entering its own revolution, partly the result of quality management principles. Both TQM and CQI demand a high emphasis on employee support and training.
Training is rapidly becoming a science and a growth industry. It has taken on a new importance in business, education, athletics, government and the armed forces worldwide. This new emphasis on he quality and quantity of training is the result of the increasing complexity of systems and technology, the decreasing availability of skilled labor, the increasing demands for defect-free products and services, and quality management efforts. While quality management requires training, training management has not fully applied TQM, SPC and CQI for quality.
This paper adapts quality management philosophies, policies, practices and tactics for use by the training manager. The results should be higher training effectiveness and efficiency – better trainee skill and knowledge, for lower cost and in less time. Quality Management: Quality management refers to management processes, disciplines and tools that are coordinated to ensure that the organization consistently meets and exceeds its goals and objectives. (Capezio, 1993) For the training manager this generally means a high level of performance from trainees at the end of the training with little variability, done at low cost and in short time.
Quality management requires innovation, change, discipline, logic, motivation and commitment. It has been said that quality management, as a concept, is nothing more than a buzzword for the collection of things that managers should be doing anyway. This may be true, but the positive impact of the emphasis on quality in the world’s industries cannot be overstated. Using the tools of quality management will allow training managers to achieve outstanding results and to enlist the support of top management for training and for quality management.
Whether the training is for office clerks, shop workers, sales people, athletes, college students, airline pilots, soldiers or toddlers learning to swim, the management of training must measure the actual performance after training against the training goals and objectives. Training managers also have to manage scarce esources, mostly money and time. Purpose: This paper presents a system of quality management for trainers, training supervisors, coaches, team leaders and educators working in aviation. The principles will, however, apply in any training organization.
The work is intended as a description of practical tools and strategies that can be used in the training of people. These tools and strategies are drawn from the more general teachings of TQM, SPC and CQI, as well as from experts in the training industry. They are chosen for their applicability to aviation training, ease of use and benefits to he training manager. Together, the tools and strategies should help to create an organizational culture change that embraces quality management, much like TQM has done in so many other applications.
Contribution: The literature describing quality management is very extensive. As mentioned earlier, the concepts and processes have evolved into a mature state. Quality management principles are proven tools leading to effectiveness, efficiency and excellence. Along similar lines, the value of good training is also shown in effectiveness and efficiency. In fact, excellence cannot be achieved without it. The training literature, primarily produced by the behavioral sciences, educational sciences, social sciences and psychology, is also quite extensive.
Much of it is theoretical in nature. While this theory is important to further the knowledge of mankind, it is not of great interest or use to the training manager “where the rubber meets the road. ” Another group of writings about training concerns itself with training systems, training delivery, curriculum design and how people learn. This is also very important for the advancement of the field of training, and the practical application literature about training and learning. There is very little written about the management of training operations, particularly in relation to TQM.
TQM has been applied to educational management at universities and colleges, though this literature is also surprisingly limited. It has been hypothesized that some educators are reluctant to emphasize quality management for themselves, because the measurements of performance and statistical processes require unprecedented evaluations of the professor’s ability to teach. Whether this is true or not is a matter of debate, but the available literature remains very limited. An extension of this literature for quality management of the training profession is even more limited.
Training has less glamour than traditional education and has just recently been recognized for its importance. Airline pilots used to train only according to regulatory requirements. Today, airlines have developed their own training programs under a new set of regulations allowing “Advanced Qualification Programs” (AQP), because airline management recognizes the increases in effectiveness and efficiency that can be achieved. Similar changes are being made in the nuclear power industry, as the emphasis on quality training grows.
AQP requires many quality management principles, like statistical process control and continuous improvement, and quality training principles like task analysis and training assessment. These concepts are discussed herein. The airline and nuclear power industries are trend indicators for other training applications, because they are characterized by complex technology and zero tolerance for the effects of errors. Other industries will fall in line as the level of technology increases everywhere and the tolerance for errors decreases.
The tools and strategies presented in this paper are designed to help raining professionals regardless of the training application, though aviation examples are used. It is assumed that all training has specified goals and objectives, and a limited amount of resources. The challenges for training management are very similar across disciplines. Quality Management Implementation TQM is designed to optimize resources and performance while meeting the necessary condition of quality. TQM should emphasize the key success factors of the organization. Stein, 1994)
In aviation training, these are consistent changes in behavior affected with the allotted resources. Aviation training programs generally do not allow a great deal of variance for any of the variables, resources or performance. A high level of safety is assumed in this case, since safety management is beyond the scope of this work. TQM requires that quality be a necessary condition. Each organization must define quality in terms of its critical success factors. (Stein, 1994) The quality of the training activity is determined by the limitations on the system.
These limitations are the target of quality management. Student throughput is determined by the resource constraints. The use of resources must be optimized with quality management techniques to create and protect throughput. The following list is a non-exclusive list of quality management components drawn from the literature (Stein, 1994): 1. Orientation to continuous quality improvement 2. A customer oriented quality focus 3. People-oriented management 4. TQM must be valued throughout the organization 5. Suppliers must be involved 6. Focus on quality improvement 7. Long-term business focus and commitment 8. Unencumbered information exchang.
9. Controlled reduction of variances 10. Employee empowerment 11. Employee-based process controls 2. Comprehensive internal training 13. Valid decision support mechanisms 14. Local benchmarks in line with global benchmarks 15. Team approach to problem solving 16. Internal and external application of the customer concept 17. Prudent use of statistics 18. Prevention oriented quality controls 19. Valid and optimum scheduling 20. Philosophies, strategies, policies, practices and tactics that are in agreement 21. A dynamic system for learning, managing and adapting to change Statistical process control (SPC) is an integral part of TQM. It encompasses many of the tools that training managers can use to measure and enchmark the program outcomes. SPC compares actual outcomes to pre- established benchmarks in an objective manner. The tools are not very complicated. The data collection methods are generally manual (instructor and check airman input) , though sophisticated automated systems exist as well. SPC generally looks for trends and outliers as targets for quality management.
Continuous improvement and variance reduction are key goals of SPC. Specific SPC tools are presented later. TQM Implementation: The commencement of a quality management program in an aviation training setting requires many of the same steps as in any other industry. We can simplify the implementation process by dividing it into five phases: awareness, assessment, preparation, action planning and evaluation. Awareness of the quality management issues generally requires a change agent in the organization. The momentum created by this person or persons should help others to recognize the opportunities created by improving processes.
This awareness and subsequent commitment must also exist at the top of the organization. (Capezio, 1993) The assessment process will identify the key success factors for the training program(s), measure the general organizational performance egarding these success factors and identify benchmarks for each. These benchmarks must come from the market – customers, suppliers and competitors. Preparation for quality management in aviation training follows the strategic planning model. The data gathered in the assessment phase is used to develop the general strategies and goals.
The required resources are also identified and allocated in this phase. Development of the action plan extends the preparation phase to the details required for quality management. A team approach, using training managers, instructors, check airmen, customers and support personnel, should be used o identify roles, tactics, policies and tools. This phase incorporates the details needed for SPC within the framework of quality management. (Pierce, 1991) The evaluation phase is ongoing and should ensure continuous improvement. The key is constant dissatisfaction with the status quo and increasing goals.
The TQM implementation process must have the support of all top-level managers, particularly the CEO. These leaders will be the change agents affecting changes in attitudes, changes in relationships, and changes in processes. The quality of learning outcomes and the optimization of raining resources should be the principal goals of the change. These goals cannot be overstated or mentioned too frequently. The team has to stay focused on the main goals. TQM implementation in aviation training also requires coordination with the regulatory authorities, since most processes have to be approved by the inspector(s).
Resistance from regulators is unlikely, since their job will be easier once SPC has been adopted. Progress reports, program validation and training trends will be much easier to record. More and better data are the main benefits of quality management in aviation training to the regulator. Labor management presents another possible obstacle to TQM implementation, though this is not unique to aviation training. Most organizations have found a participative approach with labor to be best. The message to labor is that quality management is supposed to improve the quality of work life, and the productivity and quality of training.
A collaborative effort with labor can ensure this. It should also be recognized that the above TQM implementation discussion describes the ideal scenario. The real world is rarely so kind to well- intentioned training managers. While involvement of the entire organization ith top-level leadership is best, the program manager or training manager can still use many quality management principles in his/her area of responsibility. Most of the SPC tools and quality training strategies will work in isolation as well. Their benefits will still improve the training program.
Training Evaluation and Assessment: Training evaluation is the systematic collection of descriptive and judgmental information necessary to make effective training decisions related to the selection, adoption, value and modification of instructional activities. Training assessment uses this data to benchmark the program gainst its own goals and against the performance of competitors. (Goldstein, 1992) Many new training tools have been introduced recently. Aviation training organizations are making large investments in computer-based-training, task trainers and other simulation.
These investments do not guarantee that the correct knowledge, skills and abilities (KSAs) are being learned and used on the job. Data, like that gained from SPC, is needed to verify the benefits of new technology. Grove and Ostroff (1990) describe the following barriers to training evaluation: 1. Top management does not emphasize training evaluation. 2. Training directors often do not have the skills needed to conduct training evaluation. 3. The training evaluations often done by human resource professionals do not measure the correct KSAs. 4. There is a view that training evaluation can be risky and expensive, particularly if weak areas become public knowledge. Four strategies are presented for training managers seeking better evaluation and assessment:
1. Training evaluation should be used to revise the program to meet its goals and objectives. 2. Good evaluation data can demonstrate the usefulness of the training department. 3. Legal issues can also be supported with training data. Evaluation data can be used to show job relatedness to the training program. It can also be useful to show training validation during post- incident/accident litigation. . The investments in training technology have to be backed with data for shareholders and other stakeholders in the organization. The Task Analysis: Proper training requires knowing what KSAs to train. These KSAs have come from task analysis studies in aviation. A task analysis is expensive and time-consuming, but it is required for valid aviation training, for FAA- approved AQP and for scientific training evaluation. Consequently, a task nalysis is essential before quality management processes can be applied properly in a training program.
The task analysis will provide the base data for what is to be trained, measured and continuously improved. The task analysis should be first and last step of the training program development. For SPC, it will tell the training manager which data to measure. (Mitchell, 1987) The Needs Analysis: The combined task analysis data will lead to a needs analysis, which articulates the need for training within the organization. The needs analysis defines the present practices and projects what the desired esults of the training should be. This data is also essential for building the SPC program. Mitchell, 1987) The importance of the task analysis and the needs analysis are very clear in aviation training. A word of caution is warranted, however. Many organizations have overemphasized both, wasting resources that could have been used in the commencement of training. In extreme cases, consultants have been hired to conduct task and needs analysis work, but the results took so long that the training program started without them. When the analysis was complete, it was out of date and irrelevant to the already lourishing training program.
A complementary approach is presented by Goldstein (1992). This approach merges an analysis of the intended trainee population (pilots, mechanics, etc. ) with the task analysis KSAs to include the human element and to provide a clear starting point as the status quo. It is also important to align the philosophies and expectations of the various stakeholders in the training program: the trainees, the trainers and the organization. The trainees expect to learn the KSAs needed in their work. The trainers expect these KSAs to be used in the work. The organization expects results.
While these sound complementary, misunderstandings and conflicts can occur. Good communication during the implementation phase is the key. Quality Management Tools Statistical process control (SPC) requires the use of training data and benchmarking to determine the adequacy of outcomes and quality trends. In short, learning outcomes should continually improve in a system that optimizes the training resources. So, these items must be measured. The critical learning outcome from the task analysis and the organization’s resources in equipment, time and financial expenditures create the basis for measurement. Dr.
Deming, often called the father of quality management, stated that everyone must learn the basics of statistical theory and application, since this is the language of improvement. The statistics used in quality management are very basic and do not require extensive knowledge of mathematics or statistics. This is an important point, because most aviation training managers are experts in their fields, not mathematics TQM tools: The TQM tools that are most applicable to aviation training are simply pictorial displays of processes or variances. The control chart, for example, provides a clear visual display of the ariability of measures.
It plots the variances of one variable over time or over several trainees. (Sashkin, 1993) Applied to a simple aviation datum, altitude deviation in this example, the control chart can show the performance of one flight crew over a flight: [pic] The control chart can also show the variances of the pilot population for a similar condition of flight: [pic]These are obviously simplifications used to illustrate a point. The control chart can be used best in conjunction with technology that allows this data to be extracted from flight simulators without any instructor ction or by simply pressing a button.
Such data collection program exist to provide periodic training outcome reports to the training manager. The training department or standards department would set upper control limits (UCLs) and lower control limits (LCLs) for each variable. The control charts can then be analyzed for problem areas or the software can be programmed to report any problem areas as defined by the training manager. This process removes any emotion or anecdotal evidence from the training evaluation process. (Fellers, 1992) [pic]The pareto chart is another simple, effective tool for quality anagement.
The underlying principle of the pareto chart is that 80% of the training problems can be traced to 20% of the varied possible causes. To get the most out of quality management, training managers should concentrate on the leading 20% of causes. (Fellers, 1992) The following pareto chart lists recent failures noted in recurrent training for a transport category aircraft: A run chart or trend chart shows a variable over a specific period of time. This allows the training manager to see any impacts of changes in the training program. If the control chart or pareto chart indicates a problem rea, a change in training emphasis can be made.
This change is then tracked with a run chart. Once again, this data can be collected manually by instructors and check airmen or the data can be collected by software programs on the market. (Capezio, 1993) The following run chart shows the number of daily training sessions lost to simulator malfunctions: [pic]A histogram is another bar graph showing the distribution of a variable. A histogram can be used to visually display this distribution. Stein, 1994) Most distributions like trainee test scores, extra training times, tutoring sessions, etc. ould be distributed normally (shaped roughly like a bell. ) [pic]The following histogram depicts the number of CBT sessions required to pass a knowledge test: Scatter diagrams provide a pictorial view of how one variable relates to another. The results are graphed, with one variable expressed on the x-axis and the other on the y-axis. (Stein, 1994) The following scatter diagram shows the relationship between total flight hours logged by a pilot and simulator hours needed during initial training: [pic]Brainstorming and process designing tools used in TQM can also help the aviation training manager.
Capezio, 1993) While many training decisions are made for the industry by regulators, a large number of decisions have to be made in designing and improving a training program. The tools discussed previously can be used to identify areas for brainstorming and process design. A control chart may indicate a training problem in high altitude airport operations, for example. The training manager can assemble a small group of instructors and check airmen for the training meeting. Affinity diagramming involves open brainstorming concerning a single issue. Sashkin, 1993) The training manager would pose the question: “Why are we eeing more training problems involving high altitude airports? ” Each participant then writes down one answer and reads it. No discussion of the ideas is allowed until all potential ideas have been exhausted. The ideas are then arranged in related stacks according KSAs and each is evaluated on its own merit. This process removes the personality from the idea and encourages open participation. Relationship diagramming is an extension of the affinity diagram. Each selected idea is brainstormed as to its causes in another open idea generating session.
The causes are then linked to each idea, building the elationship diagram. A tree diagram can be used to identify the steps needed in a process. An open brainstorming session is used to name the actions required in completing a stated goal. The resulting tree diagram is a pictorial display of the process. This can be particularly helpful in developing new training programs, because the steps identified by the tree diagram can then be assigned and timelines can be established. The combined information will then form a GANT chart showing the entire process in a series of timelines.
This approach to project management makes the process easier and more efficient. Project management principles are very important to efficiency during training development. (Fellers, 1992) Another project management tool useful in aviation training is the arrow diagram. It also shows the schedule of activities required to achieve a goal. It can be used with the tree diagram or other similar tools like the Program Evaluation and Review Technique (PERT) or the Critical Path Method (CPM. ) (Fellers, 1992) Detailed explanations are beyond the scope of this paper and are readily available in the literature.
In an arrow diagram, arrows indicate the direction of flow of a task and the approximate length f time required to perform that task. The matrix diagram is another relationship-displaying tool. It shows the degree of correlation between two sets of data such as KSAs to be taught and the training media used to teach them. The matrix diagram simply places one variable down one axis and the other variable along the other axis. (Fellers, 1992) The crossing boxes are used to describe the relationship in terms that describe correlation: Primary, Secondary, Contributor, etc. |Sim |CBT |Grd |Text | |Systems | |Prim |Con |Sec | |SOPs |Sec | |Prim |Con | |Instr. Prim | |Sec |Con | |Proc. | | | | | |Emergencies|Prim |Prim |Prim |Prim | |Landings |Prim | | | | This simplification shows the concept. As with the other charts shown in this paper, proper use will require extensive development and comprehensive display of the variables. While the tools are very helpful, it is important to recognize that quality management uses tools, but is not the result of those tools.
Quality management benefits from quality processes. These processes are the product of good actions, good decisions, good planning, prudent controls and teamwork. The goals of the processes should be good problem solving and continuous improvement. The tools mentioned above contribute to both areas. Good problem solving also benefits from managerial principles, such as open communication, non-jeopardy incident reporting, brainstorming, empowerment and quality teams. Quality teams in aviation training organizations consists of instructors, managers, evaluators and support personnel.
Each quality team should have a finite charter to oversee the data collection, data analysis and process improvement in a certain operational area. Each organization must assign teams according to operational needs. Note that these quality teams must not match the organizational chart. Participants will need training in the use of quality management tools and proper team skills. The TQM tools discussed are only as good as the data displayed by them. Hence, data collection is critical. The training managers and operational managers already determined the critical success areas of the organization. These will require data to show performance.
Data should be collected automatically wherever possible. Training hours come from scheduling systems and payroll. They are used to show usage rates and efficiency. Trainee performance is generally still manually recorded by instructors, evaluators and on written tests, though the technology exists to collect data directly from simulators, computerized tests, CBT (computer based training) programs, aircraft and other training devices. Automatic collection is preferred because it is more efficient and effective. Human bias is eliminated as a variable. The data to be collected is also a philosophical issue.
Most aviation training efficiency measures are expressed in terms of course or hours presented, employees trained or dollars expended per training hour. All of he data also tends to focus on formal training, when, in fact, most aviation training occurs outside the formal training programs. Since the primary job of training is to promote overall learning, the data should reflect learning (KSAs) in the formal and informal setting. Once the data has been used to assess current performance, the quality team can set goals for improvement, implement tactics and track the progress toward the goals.
Goals for improvement should involve benchmarking. Internal benchmarking will track the performance of similar functions within the organization. Training programs can be compared to other raining programs; support services to other support services. Competitive benchmarking compares the training program performance to that of competitors. Data can be difficult to collect in some cases, but is generally available through industry groups. The quality team does not have to know which competitor the data came from, which allows industry groups to collect and disseminate useful data.
Most trade associations conduct and publish benchmarking surveys. Quality Management Strategies For Training The use of proper management techniques, quality teams, quality tools and benchmarking are the core of quality management. Some quality management strategies are more specific to training. These include focusing on performance, training analysis, training design, training objective evaluation, training assessment, just-in-time (JIT) training and structured OJT (on the job training). Focusing on performance: Aviation training managers must know what training can and cannot do.
Training is designed to modify behavior through the teaching of KSAs. It can increase KSAs, changing what individuals know how to do and are able to do. The training organization cannot improve the means to do the task or motivate the aviation specialists. Training managers must focus on their task of increasing KSAs. This should constantly be verified via good feedback processes. (Carr, 1992) Training analysis: The need for a good analysis was discussed in a previous section. Though most training departments understand the need for good training analysis, few have the resources or the skills to conduct one properly.
The aviation training analyst must understand aviation training and analysis work. The process usually involves the in-house trainers and an outsourced analyst or consultant. Various barriers to communication can cause problems in this process. Participants must conduct the analysis with openness and candor. Teamwork training is recommended. (Carr, 1992) Training design: A similar dilemma exists in the next step of training design. Most aviation training organizations assign the task of course development to a technical specialist, usually one of the instructors.
This person understands the KSAs required, but typically has no formal experience or education in instructional design. Instructional design requires specialized knowledge in learning theory and application. Professional instructional designers should work together with subject matter experts during the training design hase. (Carr, 1992) Training objective evaluation: Training objectives must relate back to the KSAs, must be stated in detail, must be measurable, and must be realistic. This training objective evaluation may be conducted through formal tests, performance measurement, simulation, trainee attitudes, and observation. Mitchell, 1987) Training assessment: Since training is designed to increase KSAs and to change behavior, we must be able to measure these. This assessment should take place throughout the training program, not just at the completion of the training. Trainees and instructors should receive constant feedback. Many assessment tools exist and they are covered well in the training literature. The details are beyond the scope of this paper. The important elements of assessment in terms of quality management are that they are done, are communicated, are analyzed and are acted upon. Mitchell, 1987) Just-in-time (JIT) training: The longer the delay between training and use, the less effective the training will be. This concept is not unlike just-in-time inventory management in other industries. The KSAs learned are similar to inventories and are perishable if not used soon after acquisition. Similarly to nventory stocking costs, the period between training and use represents an opportunity cost, because the organization has spent the resources before they were required to be spent. The time value of money applies.
Hence, it is recommended that aviation training be scheduled just before the applicable KSAs are needed by operations. (Carr, 1992) Structured OJT: On the job training (OJT) is by far the most common form of training in the aviation industry. We tend to think of training in terms of classroom or training device hours. In fact, most safety related training for pilots, technicians, dispatchers and ground personnel occur on the job. OJT can be very effective and terms of performance and cost. Unfortunately, most aviation OJT is not conducted in a structured manner.
Structured OJT requires prior planning, training for supervisors and instructional design. In short, structured OJT requires many of the same formal processes that any training program deserves. Analysis, design, evaluation and assessment all apply to structured OJT. (Carr, 1992) The Quality Training Culture TQM is a culture change. It requires total support and total acceptance to be optimally effective. The quality training culture must produce superb raining programs and it must do so within the limited resources governed by economics. Training for TQM: Quality management requires a permanent, ongoing training effort. Stein, 1994) The KSAs to be trained are those required to gain and maintain control over quality processes. Two obstacles must be overcome: fear and the “not invented here” syndrome. The initial training effort should address both. The training staff should be involved in the process. Many organizations designate one current staff member as the “quality management trainer. ” Outsourced training is, however, almost always recommended. (Capezio, 1993) Training wastes: Training efficiency is important in aviation. Device usage and human resource expenses are very high.
Training wastes might include the teaching of unneeded KSAs, unused device time, excess waiting time, unneeded travel or reliance on outdated technology. Training wastes should be identified by quality management analysis and they should be eliminated. This may also require a culture change as the training staff adjusts to new ways of delivering their services. (Stein, 1994) Economical training: Most aviation training is measured economically in terms of hours and dollars (resources used). The preferred economics of training maximize the performance improvement for its expenditures, while satisfying the regulatory authorities.
The strategic and scientific approach discussed as quality management for aviation training is the best way to achieve this. (Carr, 1992) The training department mission: The aviation training department should develop and maintain an overall training strategy, identify training to meet this strategy, identify KSAs to be trained, match the training programs to strategic and tactical needs by developing the courses, and deliver the training. The department should lso complete the feedback loop through the assessment methods discussed earlier. Carr, 1992) These activities go well beyond the usual aviation training department mission, which only states that “… we deliver the best training programs possible at industry-competitive costs. ” Conclusion Quality management in aviation training requires that training managers keep the processes simple, stay focused on critical areas, strive for continuous, measured improvement, and align the philosophies, policies, procedures and practices in the company. Most training managers are technical specialists, not statisticians or even professional managers.
The quality management and operational processes must be kept simple to be usable. Most data collection should be automatic and non-intrusive. Most data analysis should also be automated or centralized. Quality management does not have to be complicated. The tools and concepts discussed are truly just good management. They are applied by few aviation training organizations, however. The goals of improvement must stay focused on the critical success areas of the organizations. Most critical success areas will be some expression of efficiency or effectiveness. The training manager should never be satisfied with the status quo.
Setting higher, yet achievable, goals will motivate and stimulate the organization. There has been much discussion in the human factors and aviation safety literature about the alignment of philosophies, policies, procedures and practices. None of these four layers of operational guidelines should conflict with another. Conflicts cause confusion, loss of training effectiveness and safety hazards. The training department, through the proper analysis and data, is the front line of defense against these conflicts. This should be kept as a central theme of aviation training’s quality management efforts.