Putting Checklists in Their Place

Dennis A. Vincenzi, PhD
Aviation and Aerospace Psychology

Submitted by Edward M. Ward, Jr.
Graduate Student, Embry-Riddle Aeronautical University
December 2003

Abstract

The main objective of this research paper is to explore the different checklist philosophies and determine what human factor issues might be encountered. We will look at different checklist types, when and how they are used and in what format they should be used. We will pay particular attention to checklist color-coding, touch on the use of different checklist displays, and compare the use of a "hard copy" traditional checklist with the modern CRT displays.

Checklist Philosophies

What is a checklist? "Checklists are to ensure that the crew will properly configure the plane for flight, and maintain this level of quality throughout the flight, and in every flight" (Degani p 7). Checklists are primarily systematic procedures. Webster's dictionary defines a procedure as "an act or way of doing something". The FAA defines a checklist as:

FAR 121:3135 Cockpit Check Procedure

(a) Each certificate holder shall provide an approved cockpit check procedure for each type of aircraft.

(b) The approved procedures must include each item necessary for flight crewmembers to check for safety before starting engines, taking off, or landing, and in engine and system emergencies. The procedure must be designed so that a flight crewmember will not need to rely upon his memory for items to be checked.

(c) The approved procedures must be readily usable in the cockpit of each aircraft and the flight crew shall follow them when operating the aircraft (FAR's 2003)

We, as human factor's practitioners, must focus our attention primarily on the way in which this is accomplished. We must first define our goals and then design procedures to accomplish these goals. Checklists must be designed so that the result is (first and foremost) the accomplishment of the needed objectives in the most efficient manner possible. Checklists must be designed so that the human interface is not restricted and that the immediate action (or actions) is (are) quickly accomplished. We must understand that when checklists are used it will require an interface of a human person. Modern aircraft design; requiring the automation of systems, is a design requirement usually consisting of triple redundancy. The aircraft and avionics systems engineers can program different checks and balances but when built in logic fails, we need a human interface to provide corrective action. This is the basic and most important principal of checklist design. Checklists must work for humans!

At what period and on what occasions should a checklist be used? To correctly answer this question I would like to state the primary objectives of any checklist.

1. "Aid" the pilot in recalling the process of configuring the plane.

2. Provide a standard foundation for verifying aircraft configuration that will defeat any reduction in the flight crew's psychological and physical condition.

3. Provide convenient sequences for motor movements and eye fixations along the cockpit panels.

4. Provide a sequential framework to meet internal and external cockpit operational requirements.

5. Allow mutual supervision (cross checking) among crewmembers.

6. Enhance a team (crew) concept for configuring the plane by keeping all crewmembers "in the loop."

7. Dictate the duties of each crewmember in order to facilitate optimum crew coordination as well as logical distribution of cockpit workload.

8. Serve as a quality control tool by flight management and government regulators over the pilots in the process of configuring the plane for the flight" (Digani p.7).

Different Types of Checklists

Checklists are used for two different purposes. One purpose is to set-up or configure a system (even the whole airplane) while the other purpose is to ensure that a failure of a system is corrected. There are four basic methods used with checklist design (Orlady p.159). They are the "read, do, and verify", "do, challenge, and verify", "do and verify", and lastly the "combination where the "do and verify" checklist s are used in "non-normal or emergency checklists in conjunction with immediate action items, which must be memorized" (Orlady p.160).

Checklists must be designed for a particular action within the requirements of the number of pilots who are available and the specific equipment evolved. For example, the pilot of a single engine propeller aircraft will have to make do with the "read, do, and verify checklist" simply because there is not any other crewmember to challenge or assist him. On the other hand, aircraft with three crewmembers (two pilots and a flight engineer) have the luxury of more flexibility in what types of checklists are used and when. An important point to consider is that any checklist must be designed for a particular action to occur based on that particular type of aircraft. For example, multi-crewed airliners like the DC-9, MD-80/90 or Boeing 717 need to have their own particular checklists based on their models even though they are updates of the same basic types. Airline management cannot use a blanket checklist procedure to cover the different changes of models within types within their fleets.

When and How Checklists Should be Used

Checklists may be divided up into three areas. They are Normal Procedures, Abnormal Procedures, and Emergency Procedures. Flight Safety, Inc. teaches Normal Procedures checklists, in the "Read Do and Verify" method. All operations, from first entering a "cold" aircraft to the "After Engine Shutdown and Aircraft Secure" items are covered in the Normal Procedures types of checklists. Many operators have adapted the "flow design" ways of doing the normal procedures that follows the "do and verify" method to accomplish the required tasks. This design allows one pilot to do the required items from memory while the other pilot follows along, by reading the checklist to himself and verifies that the items are accomplished. I have used both methods and have found that the "flow design" is the faster of the two.

Abnormal Procedure checklists are used for non-standard items that occur in flight that do not require immediate emergency actions. These types of checklists usually have supplemental data, which must be looked up by the crew. Emergency checklists are set up to incorporate "Immediate Memory Items" where actions are required from memory. After the initial emergency memory items are finished, the crew will then accomplish additional items from an "Expanded Emergency Supplemental Data" checklist. Many checklists have a poor human factor's design fault as to where to place items that might be located in either the "Abnormal Procedures" checklist or the "Emergency Supplemental Data" checklist. This is especially true in problems associated with the Flight Management Computer (FMC).

An important human factor's consideration is, at what point of the flight should the different checklists be used. Establishing correct time-lines for the "Normal Procedures" checklists is not only good crew resource management but also will preclude the possibility of forgetting to do the checklist in the first place (Baron, Web). For example, leaving FL180 (eighteen thousand feet) is a great time to do the "Descent and Landing" checklist. Both crewmembers are expecting to have this occur at this stage of the flight. If the checklist is not done, then it is hoped that one of the crewmembers will notice this lapse in standard procedure .The same point can be made for pilots flying light sport aircraft. If sport pilot picks out a set time and place for a checklist to be accomplished (e.g. doing the "Before Landing" checklist on the forty five degree entry to downwind) this will usually insure that it will be done. Think about how many light aircraft gear up landings could have been avoided if this human factor's time-line approach had been used!

How should checklist be written? In the next block presentation, we will go into color-coding and checklist layout. It is interesting to note that the use of the correct font has a lot to do with reading comprehension. The use of the "Gill Median" font (in bold print) has proven to have less conflicting characters and is easier to read (Poulton p.360)

Color Coding of Checklists (multi-crewed)

Below are typical guidelines for aviation color-coding.

• Readability: Is this color sufficiently different from its background?
• Confusability: Is this color sufficiently different from other colors used in the system?
• Name ability: Does this color correspond to people's expectation for that color? For example, is red really red?
• Consistency: Does this color always mean the same thing?
• Redundancy: Is the information provided by the color-coding also provided by some other means? This is to mitigate situational, environmental, or user factors that may make the color information difficult to use. Conventions: Red typically means "stop" or "failed" whereas green means "go" and "healthy." Violating these and other established conventions is not recommended" (NAS Human Factor's Group)

When an abnormal (or emergency) situation takes place, detection is first noted by the activation of the pilot's senses (vision, auditory, touch smell). Two of these senses, vision and auditory, are the primary indicators of the requirements for checklist action. Usually, the first indication of an abnormal or emergency condition is when the Master Caution Light is illuminated or a circuit breaker pops. After the initial indication, the pilot's vision will be the primary factor for corrective action. What the crew sees (or has to see) will determine the outcome.

Usually, the first indication of a problem is the illumination of the Master Caution light followed by one or more items appearing on the Annunciator panel. The panel shown (in the above illustration) is a trade off between space and placement of the one single annunciator panel. While the First Officer has direct eye level contact with the Master Caution light and Annunciator panel, the Captain has to look over to his right for his view of the Annunciator panel.

In the example of the Astra 1125 Corporate jet, the Master Caution light has a one-color design. It will illuminate in a white hue no matter what the indicating cause from the Annunciator panel. The Annunciator panel has a two-color design. Red hue is used for emergency items while a yellow hue is used for abnormal developments.

The Emergency or Abnormal checklist process begins with the illumination of the Master Caution light (to attract the crew's attention) followed by one or more lights appearing in the Anunciator panel. The crew will deactivate the Master Caution light (usually by pushing the light switch in) and reach for their checklists. The checklist process continues as the crew will decide what item (or items) applies to the specific condition. Speed and accuracy is of the up most importance. The illustration is of a Flight Safety Checklist that shows the same layout as the Annunciator panel.

Once the light on the Annunciator panel and the same light (color coded) on the checklist are verified you can view the page numbers of the items (to the right of the checklist's Anunciator panel) and then open to the corrective actions on the checklist indexes. The index numbers are printed on both sides of the tabs to aid in location. Flight Safety, Inc. certainly used a human factor's approach to this design. The only improvements I could suggest is to color code the red emergency tab data on the sides of the pages (shown above) to red tabs (not orange). A good question is why use any red hues in cockpits. The answer is that at night in an emergency or abnormal condition, the pilots will turn on the overhead floodlights that are white in color. A red hue, at night, is not a safety factor in multi-crewed airliners.

Electronic Checklists and Their Displays

The FAA Human Factors Design Standards has published guidelines that cover, in detail, cockpit displays. The following quotes list the most important items.

"Make displays function under operational conditions. Visual displays should function under any circumstance corresponding with the operational and use philosophies of the system". (Source: Department of Defense (MIL-STD-1472F), 1999

"Make displays legible under all conditions. Visual displays shall be legible under all anticipated viewing conditions. (Source: MIL-STD-1472F, 1999)

1. Avoid unnecessary markings. Trademarks, company names, and other markings not needed to identify a display or aid in performing tasks should not be displayed on a panel face (Source: MIL-STD-1472F, 1999)

2. Ensure adequate refresh rate. Displays that require refreshing of information such as CRT's heads up displays, and collimated displays, shall be refreshed at a rate that insures that the displayed information is sufficiently current for the users to perform their tasks. (Source: MIL-STD-1472F, 1999)

3. Provide adjustable contrast and brightness. Visual displays should provide users with the capability of adjusting contrast and/brightness". (Source: Vanderheiden & Vanderheiden, 1991)

If we are to use electronic checklists and displays, we must place them in a position so that airline crews will be able to see them at critical times. The quote below details this best.

"Focal vision is the central 30 degree of the visual field. This is the area that people use to look at object in the world, moving their eyes as needed to bring images of the object on to the fovea, which is the area of highest acuity. When an object is outside of the focal area, a person will usually turn the head rather than simply move the eyes. By locating frequently used displays in the central 30 degrees of visual field, the user is not required to move his or her head to bring the information into the focal area". (Source: MIL-STD-1472F, 1999)

Another important question is, can we position electronic checklists at a place in the cockpit where all flight crews will be able to see them? With a hard copy checklist, it is possible to adjust the distances and viewing angles that will work best for the age and optic abilities of the pilots. The quote below best illustrates this situation.

"Users have individual differences in their preferred viewing distances. It is preferable for users to adjust the viewing distance to their preferred distance". (Source; Jaschinski, Heuer, & Kylian, 1998)

There has been extensive research data on the role of automation and the future of electronic checklists. In my research, the best study (that I found) is the one done by Palmer and Degani who ran three human factor's tests on how different types of checklists performed when given a set task to accomplish. Of the three types of checklists studied, the printed checklist resulted in fewer errors than any of the other electronic checklists (Palmer, Degani, Web). In my opinion, all electronic checklists have a basic built in fault. From a human factor's standpoint, electronic checklists require two human motor skills that must be done together. To effectively use electronic checklists, you must have both a visual scan on the data items and use motor control dexterity by physically scrolling down the menus. In a real emergency, this might place an excessive workload on the crew, at a critical time of flight.

Does this means that I am against automation in the cockpit? My answer to this is that automation has a definite place in modern day cockpits. One of my main concerns, of any checklist, is that flight crews can get overwhelmed with extensive manual details and not have time to "sort things out" in regards to the abnormal or emergency items. For example, the condition of "smoke and fire of unknown origin" is a detailed task to quickly rectify using any type of checklist. This condition may require the use of detailed schematics and several steps to go through. The real answer to this problem is to have built in automation to solve problems like this, and shut down the failed system automatically without any crew impute.

Another area of concern with electronic checklists is the resolution of the current displays and the ability to see the screens in a bright light condition. These faults can be developed, but I believe the paper checklist will be with us for some time. I would like to see airline management concentrate their human factor's efforts, in designing workable, color-coded paper checklist (like the Flight Safety design shown in this paper) that out performs any other types of currently available electronic checklists.

There is another area concerning checklists that I would like to see developed. The Intergraded Hazard Warning System (IHWS) has the potential to place another "pilot" in the cockpit. This system has the possibility to have a human voice, that is electronically always there (in the background much like GPWS), to avoid pilot errors when performing checklist items. For example, the "Emergency Engine Shutdown" checklist has the potential for pilots to shut down the wrong engine. Airline procedure is to have the Captain ‘guard' the good engine while the corrective action is accomplished by the First Officer. While this implies good crew coordination, it does require the Captain to direct his attention away from flying the airliner to take part in this checklist procedure. With IHWS, it is possible to put a third "pilot" into the loop who could help with checklist compliance. My position is to let the Captain fly the airliner (first and foremost) and (in this example) if the First Officer happened to move the wrong throttle, IHWS would call out something like "Verify Engine" or words to that effect.

Conclusion

This research paper has presented data on the types, designs and use of modern day checklists. We have looked into the requirements and objectives of checklist use. In addition, we have explained the different types of checklists and when to use them. Particular emphasis has been spent on the explanation of color-coding and the effects of good designs. I have shown how checklists are used and we have explored the possibility of the use of electronic checklists and some of the inherent problems that will be needed to overcome.

References

Baron Robert, AirlineSafety.com, CRM: From The Instructors Viewpoint, http://www.airlinesafety.com/editorials/CRM-InstructorsView.htm

Boeing Commercial Airplanes Division, P. O. Box 3707 Seattle, Washington 98124

Degani A, (1990) The Normal Checklist, Human Factors of Flight-Deck Checklists: NASA Contractor Report 177549, Ames Research Center, Contract NCC2-377.

Federal Air Regulations, (2003) FAR Pt. 121:315. US Government Printing Office, Washington, D.C. USA

Flight Safety International, Astra Corporate Jet Division, Wilmington, Del USA.

Jaschinski, W., Heuer. H., & Dylian, H. (1998) Preferred position of visual displays relative to the eyes: A field study of visual strain and individual differences. Ergonomics, 41 (7). 1034-1049.

MIL-STD-1472F, (1999) US Government Printing Office, Washington, DC. USA.

NAS Human Factors Group, FAA Technical Center, http://hf.tc.faa.gov/faqs.htm

Orlady, Harry and Linda, (1999) Basic Communication, Human Factors in Multi-Crew Flight Operations, Ashgate Publishing Company, Vermont 05036.

Poulton, ElCl, (1965) Letter differentiation and rate of comprehension in reading. Journal of Applied Psychology, 49(5), 358-362.

Vanderheiden, G.C., & Vanderheiden, K.R. (1991). Accessible design of consumer products: Guidelines for the design of consumer products to increase their accessibility to people with disabilities or who are aging. Madison, VI: Trace Research and Development Center.