This page is a companion to my February 7, 1995 Journal Club talk entitled
"Human Error as a Tool in Design Health Care Information Systems." It includes
a narrative version of my talk, and list of sources if you want to learn more.
Disclaimer:
This material has been posted on the World Wide Web in the spirit of free
exchange and sharing. Others are welcome to share this material with
friends and colleagues, but I request that reference to the source (me) be
maintained in any reproduction and that it be made clear that this material
is a text version of a talk given at a graduate seminar at Stanford
University School of Medicine and is not a peer-reviewed article from the
literature.
Here's a more detailed Table of Contents:
(For those of you that just want to breeze through this, here's the
executive summary)
Human Error studies the way people think and prepare for their actions,
and how they perform them. The goals are to study the cause, prediction and
prevention of human errors. I came to several conclusions while researching
this topic.
- Human Error is difficult to talk about, let alone analyze.
It's a relatively new field, still trying to define its boundaries,
terminology, and taxonomy. Nevertheless, a basic understanding of the kinds of
errors humans make can help us design better systems.
- Errors happen. This may seem trivial on the surface, but it
is important to recognize that no matter how good a system or design is, people
will always make mistakes. As
Doug
<fridsma@camis.stanford.edu>
pointed
out during my talk, some subscribe to the concept of "normal errors", errors
that are inevitable in complex systems.
- Systems can encourage Errors. Although
errors are often attributed to the action of an individual, there are often a
set of external forces and preceeding events that led up to the error. Often
these forces and events are difficult to anticipate, and yet, the reaction to
errors, especially in medicine, is often to punish the individual, rather than
examine the error-prone system as whole. By system, I mean any group of
people (e.g. physicians, nurses, programmers), objects (e.g. signs, OR equipment,
computer systems), knowledge (e.g. medical training, hospital procedures) that
come to play in a particular health care process.
- The study of Human Error reinforces existing guidelines for good
design. Many of the standard HCI Heuristics we are taught (e.g.
consistency, forgiveness, etc.) have their roots in cognitive science and the
analysis of human performance. Since we can expect users to make errors, we
should focus on error tolerance and error recovery as well as error
prevention when we design computer systems.
One of my favorite quotes comes from
Set Phasers on Stun and Other True
Tales of Design by Steven M. Casey, and reads:
New technologies will succeed or fail based on our ability to
minimize the incompatibilities between the characteristics of people and the
characteristics of the things we create and use.
This sums up
why we want to study Human Error: to understand the difference between the
actual characteristics of the world around us and the way we model them
internally (which leads to the way we interpret our perceptions and act on those
interpretations), and to use this knowledge to build better tools.
That's pretty much it. If you want more details and examples of error types,
read on...
Here are the terms introduced in this Web page. If you think they aren't
explained clearly enough, need more examples, or are lacking in some other way,
please send me e-mail.
Iatrogenic;
Skills-Based Cognition;
Rules-Based Cognition;
Knowledge-Based Cognition;
Slips;
Capture Slips;
Description Slips;
Associative Activation Slips;
Loss Of Activation Slips;
Mistakes;
Rule-based Errors;
Knowledge-based Errors;
Latent Errors;
This paper is organized as follows: we begin with an introduction to the field
of Human Error, including a model of cognitive thought that proposes a model of
how we think, and what happens when that model breaks down. We introduce some
of the vocabulary used to study human error, and, as a classic example of human
error involving computers, we look at the
Therac-25 incident. The next section
looks at what we as health care information systems designers can learn from
the field of human error, in particular, the impact it can have on interaction
design. The last section looks at how medicine as a field regards human error,
and how that can impact our task of using computers to help healers.
I became interested in the study of Human Error after taking a CS last quarter
(
CS-377 Cognitive Science for Human Computer Interaction, Assoc. Prof. Hank
Strub, Fall 1994-95). One the articles we read led me to think about how the
study of human error could help interpret user behavior during ethnographic
studies, task analyses, usability studies, and many of the other activities I
expect to be doing during my career (wouldn't it be more interesting if that
activity list had read "para-sailing, open-heart surgery, and hunting wild
yak"?).
As I started browsing through the literature, I was surprised at the estimated
rates of human error in health care settings. A population-based study
published in 1991 reported that nearly 4% of patients hospitalized in New York
State in 1984 suffered iatrogenic --doctor-caused--injuries; nearly 14%
of these were fatal. Other studies have estimated that medication errors occur
in 2% to 14% of patients admitted to hospitals, but most do not result in
injury. (Most of these statistics were quoted in the
December
1994 JAMA article by Lucian L. Leape, MD). Given the increased scrutiny
under which the health care system has come recently, it seemed interesting to
look at how the medical profession views and deals with human error, both with
and without computer involvement.
Furthermore, much of the literature on human error and performance focuses on
the field of aviation. In aviation, these studies have led to safer planes,
better designed cockpits, and a set of systems to handle errors when they
occur. Professor Strub pointed me to a new book that just came out called
Human
Error in Medicine which, as the name would indicate, focuses on health care
issues. I found it intriguing to consider whether human error could have the
same kind of impact on health care and medical informatics as it did on
aviation.
Hence, my choice of journal club topic.
The formal study of human error is relatively recent (the early articles date
back to the early 60's), and is tied closely to a several other relatively new
fields. There are three fields principly involved in studying human error:
- Cognitive Science (also known as Cognitive Engineering) is itself a
mix of different disciplines, including psychology, philosophy, neuroscience,
and artificial intelligence. Cognitive scientists attempt to understand and
model cognitive abilities such as perception, learning, language, memory,
problem solving, etc.
- Human Factors or Ergonomics look at the specifics of human
performance and how it can be improved. From a simplistic point of view, these
are the folks who figure out which arm position is the most comfortable for
typing, where to place monitors and readouts to make them easy to read, and
where to place the windshield wiper control so its easy to reach. On the
computer side, human factors engineers can help determine how to lay out the
control panels of medical devices in order to maximize user performance.
- Systems Analysis attempts to model systems and organizations in
order to understand its functions, including its relationships with other
systems and its subsystems. Researchers try to understand how various
components of a system can contribute to a problem. In this context, system can
mean an operating room team, a hospital, or the U.S. health care system as a whole. Systems
Analysis has its background in the field of Operations Research.
I placed
HCI (Human Computer Interaction) at the center because it draws from all three
of these fields and focuses them on the user of computers.
Given the variety of researchers interested in studying human error, it isn't
surprising that one finds contradictory viewpoints in the literature. Part of
the difficulty in developing a single, cohesive approach to the study of human error
stems from the various agendas at play. Lawyers, courts and the like use
errors to assign responsibility. Engineers and designers use
error to design and assess systems (or, at least, they should). Behavioral
scientists use error to understand human behavior (surprise). Medical
scientists may use errors to understand the effects of
psychopharmacological agents. And so forth...
One conviction that seems to be shared by all members of the field studying
human error from an academic standpoint is a rejection of the conventional
approach to error prevention, that of training and punishment.
Traditionally, the way to get people to do things correctly is to teach them
(droning lectures and the like). Then, if someone does something wrong, you
tell them "bad boy/girl/sentient being," tell them to go sit in the corner, and
hope they don't do it again (i.e. that they've "learned their lesson").
Students of error and human performance believe that all humans
err on occasion, and that while a human being is usually involved when
accidents occur in a professional setting, the causes of the error may be out
of the individual's control.
So, let's start with a definition: what is Human Error? Ought to be
easy, right? Wrong. Turns out there are a number of definitions and taxonomies
of human error out there, each trying to serve a different purpose or provide a
particular insight into the field. None of them struck me as particularly
relevant to human error in medicine, so keeping with the tradition of the
field, I've come up with my own definition:
"Human Error: An inappropriate action, or intention to act,
given a goal and the context in which one is trying to reach that goal." Ramon,
1995
Most seem to agree that we need a better word than
"error", because it is such a weighty term. Speaking of an "error", especially
if attributed to a human, carries insinuations of blame, responsibility, and
associated cognitive luggage. Hence, one of the tasks researchers have
undertaken is the development of a richer, more refined terminology for
speaking about human errors. Although there is no formally recognized
terminology yet, in the next section we'll look at several terms that have
gained acceptance and are being used relatively consistently in the field.
In general, researchers in the field study the Cause, Prediction
and Prevention of Human Error. I say in general because not everyone agrees
that it is possible to predict or prevent errors. Some thing that errors are
inevitable, so it is more important to focus on error-tolerance and
error-recovery. Others point out that prediction doesn't necessarily lead to
prevention: we may be reasonably certain that errors will occur in a complex
system, but still not be able to prevent them effectively.
Accidents typically occur due to mental error (even stubbing your
foot on a chair is considered a mental error since, precluding some unhealthy
hatred of furniture, you misjudged the distance between the chair and
yourself). Thus, to understand what goes wrong in these mental processes, it's
worth looking at them when things go right. I choose to present a simplified
version of a theory put forth by James Reason and summarized in the
Leape
article (I figured that any Theory of Cognition put forth by a guy whose
last name was "Reason" has got to be pretty good... that, plus he's referenced
all over the place). According to this theory, there are three basic kinds of
mechanisms of thought--
Skills-Based,
Rules-Based,
and
Knowledge-Based--that
span the range from unconscious to conscious thought processes.
Note: since most of these mechanisms refer to cognitive processes leading to
actions, I'll be referring to an "actor", the person performing the act based
on whatever cognitive process kicks in.
Skill is the ability to carry out a task; skill-based cognitive processing and
performance refers to actions that are automatic and easy due to an acquired skill. They
usually happen quickly and without express effort on the part of the actor. For
example, getting out of bed, putting on a T-shirt, or opening a door--all are
unconscious actions that we don't need to explicitly "think about" in order to
accomplish. Any decisions are usually automatic as well (which arm goes in
which hole of the T-shirt).
Most training is concerned with skill development, the end goal being the
development of an automatic process. Typically the actor needs to understand
how to execute a set of instructions, but not understand the reasons behind
them. Through training, the actor will become proficient enough--skilled
enough--to perform the actions without the need of instructions.
Any departure from this kind of skills-based processing requires either a
rules-based or knowledge-based solution to perform the task.
Rules-based processing involves matching the context and problem currently
facing the actor. These rules are typically of the "if X then Y" form, and can
be based on past experience, explicit instructions, and so forth. For example,
if you want to leave the room, you normally push the door to get out (an
automatic skill). If the door doesn't open, however, you start to go down your
list of reasons why it didn't open in order to accomplish your initial task
(leaving the room). Maybe the door pulls instead of pushes, maybe its locked,
etc.
To pursue the training analogy mentioned above, rules-based processing comes to
play when an automatic skill fails and the actor needs to fall back upon a set
of explicit instructions or rules at his disposal. The actor examines and
interprets the current situation, and choses a rule that can best solve the
problem. For example, if a lab technician attempts to a calibrate a monitoring
system, a set of rules will guide the technician's decisions: if the readout is
too high, turn this knob to the left; if its too low, turn the knob to the
right.
If rules-based processing doesn't solve the problem, we fall back on
knowledge-based processing (we tend to prefer rules-based solutions since they
require less cognitive effort on our part). This is what happens when we are
truly faced with novel or unfamilar situations, or where low-level rules aren't
appropriate (e.g. making strategic decisions, estiablishing a medical
diagnosis, or solving algebraic expressions). In general, this kind of
processing involves the processing of symbolic information (the different suits
in a deck of cards, or the graphic symbols of
algebraic notation)--the proper use of symbols in user systems can reduce the
user's cognitive effort.
As with rule-based processing, knowledge-based processing is a conscious
process. It refers to what we typically think of as "analytic thought," the
process and analysis of personal subjective knowledge. Where skill is the
ability to carry out a task, knowledge is the possion of "information, facts,
and understanding" about a task. Note that this doesn't necessarily mean you
can actually get the task done: you may know a lot about a task but still not be
able to carry it out.
So that's one model of how we think and do things. Now lets get back to the
topic at hand--Human Error, in case you forgot--to see what happens when these
processes break down. Errors in these cognitive mechanisms are called
Slips and Mistakes.
A slip is an action not in accord with the your intentions: a
good plan but poor execution, sort of like fumbling the ball on a good play.
Since they are part of automatic, unconscious actions, slips are unintended
acts due to a break in the routine. Examples of slip error mechanisms include:
-
Capture slips occur when you automatically do something you didn't mean
to, usually because you fell into a pattern you perform frequently. For
example, if you dial a particular phone number often, your fingers get used to
hitting that particular sequence of pushbuttons. There are plenty of examples
of this in medicine given the enormous amount of data entry that is done: if
certain codes are repeated frequently, the pattern, or "schema" in
HumanErrorSpeak, for that code may "take over" if you try to type a similar one
in a hurry.
-
Description slips occur when you haven't correctly told yourself what
you want to do, i.e. an "incomplete or ambiguous specification of intention."
(I have a hard enough time figuring out what I want to do when I'm
conscious--I'm surprised anything unconscious gets done at all). This usually
happens when your intended action is similar to other actions you do a lot, so
that you perform the right action but on the wrong object. For example,
sticking the salad in the oven and the cake in the refrigerator, or
accidentally sending a love letter to your advisor and your midquarter progress
report to your main squeeze because you switched e-mail addresses...
-
Associative activation slips occur when your brain makes a faulty
connection or mental association between two ideas, often when one is an
external stimulus that typically provokes a certain action. The example given
in the readings was answering the phone when you hear the doorbell. I thought
this was more a symptom of looniness, but it was cited twice as an example, so
I guess it's normal for some folks. Obviously, the authors hadn't heard the
unique and unmistakable ringing sounds of Stanford phones.
In any case, I can see how this might become an issue in other areas where
people rely on sound and other senses for information. For example, an ICU may
have several machines with audible alarms: an associative activation error
could occur if a nurse reacts to Alarm A as if it is Alarm B going off. Note
that I don't think the alarms necessarily need to sound similar--it's enough
that a bad connection between the "alarm" concept was made.
-
Loss of activation slips occur when you lose track of why you're doing
or trying to do (the "activation" of the process). This is essentially a
temporary memory loss, often due to interruption such as someone handing you
something, asking you a question, or poking you in the eye. Certain readers may
recognize this as the "now why did I get up to come into this room in the first
place?" phenomenon.
A mistake is a planning failure, where actions go as planned--but
the plan was bad. These are errors of judgement, inference, and the like, that
result in an incorrect intention, incorrect choice of criterion, or incorrect
value judgement. Mistakes are the real challenges in the analysis of human
error. Slips can often be prevented through checks built into equipment and
tools. For example, an O2/N2O ratio limiter that prevents an anesthesiologist
from accidentally administering a dangerous combination of gases. Mistakes, on
the other hand, stem from cognitive breakdowns; as we'll see, these are often
influenced by a number of external system factors, and are therefore harder to
predict and prevent.
Rule-based errors occur when the wrong rule is chosen due to the misperception
of the situation, or the misapplication of rule. For example, selecting the
wrong medication for a patient. The medication may be correctly ordered and
administered (i.e. the procedure goes off without a hitch), but it is the wrong
medication for that particular patient. Misperceptions that lead to rule-based
errors can stem from a number of sources, including external factors such as an
unclear or partially hidden read out on an ICU display, confusing patient
charts or lab result displays, and so forth. Frequently used rules may be
applied in error because the actor is familiar with them and they seem to fit
the situation.
There has been a lot of study on how stress affects performance, especially how
people process their rules. For example, one stress-induced phenomenon is
called Coning of attention, which refers to the way people concentrate
on single source of information in stressful situations. For example,
concentrating on opening an airplane door for an emergency exit in a crash
situation (the official procedure), when there's a huge hole in the bulkhead
nearby that people could use as an exit. Another phenomenon is called
Reversion under stress, where people switch to using older, more
comfortable rules in stressful situations, even if they've learned new, more
"correct" ones recently.
Knowledge-based errors are the most complex of the errors we've discussed. They
typically occur, as one might expect, from a lack of or misapplication of
knowledge. As a result, often the intention of the actor is itself
erroneous. This is where some of Tversky's bias heuristics can come to play.
The availability bias (choosing a course of action because it is the one
that comes most readily to mind) and confirmation and overconfidence
biases (fixation on a particular course of action and actively pursuing
supporting evidence or ignoring contradictory evidence) can lead an actor to
making faulty conclusions about a situation, and therefore drawing up and
executing a faulty plan to accomplish the task.
I've just listed a few errors that parallel the model of cognitive processing I
presented earlier. It isn't a comprehensive list by any means. Other taxonomies
examine different levels of error (e.g. phenomenological (errors or omission or
substitution), hypothetical internal processes (capture, overload, or bad
decisions), neuro-psychological mechanisms (forgetting, stress, attention), to
external processes (poor equipment design). But it was a one-hour journal club,
so what are ya gonna do...
It is worth briefly pointing out some of the factors that can affect our
thoughts and actions, especially since so many of these factores are around in
hospitals or other health care settings.
- Symbolic Information: the actual data you get from external sources
(the details of the situation, the specifics of the drug prescription, the
rules and instructions to follow, etc.)
- Subjective Value Formation: a person's subjective values, ethics,
attitude, the social climate, etc.
- Anatomical Properties: any physical factors impacting a person's
performance. For strenuous tasks, this may simply be the physical workload. It
also includes any injuries or disabilities.
- Physiological Functions: factors such as fatigue, sleep loss,
alcohol, drugs, illness, or other that may influence the physiological behavior
of the user.
- Psychological Mechanisms: frustration, fear, anger, orals anxiety,
and other psychological influences.
These factors come from many sources.
The physical environment can provide noise, visual stimuli, distraction motion,
heat or cold, etc. The work environment may have unproductive or inappropriate
work policies, training and education requirements, and so forth. All of these
influence the way we process information and make decisions which lead to our
actions.
Lets take a look at a classic, oft-cited example of an accident that involves
human error and computers, that of the Therac-25. The Therac-25 was a million
dollar radiation machine designed to precisely aim a beam of radiation at a
patient in order to treat tumors or cancerous growths. Patients were often
recovering from operations that had removed the bulk of a tumor, and underwent
these radiation treatments to remove what was left.
The Therac-25 was high energy radiation machine, but radiation treatment
usually involved many low-energy dosages across successive treatment sessions.
The machine was controlled through a computer (a terminal hooked up to an old
Vax mainframe, I think) located in another room (as are most radiation therapy
controls, in order to protect the technicians from unnecessary exposure).
There were two basic modes in which the Therac-25 could function. The first was
the low-energy mode mentioned above, in which an electron beam of about 200
rads was aimed at the patient and sent off in a short burst. The second mode
was an x-ray mode, which used the full 25 million electron volt capacity of the
machine. When the machine was switched into this mode, a metal thick plate
would get inserted between the beam source and the patient; as the beam passed
through the plate, it was transformed into an x-ray which would radiate tumors
and the like.
To switch to "electron mode", the technician typed "e" at the computer
terminal. To switch to "x-ray mode", the technician typed "x" at the
computer terminal. Simple, right?
Well, in 1986, Ray Cox, a Texas oil worker, went in for his usual radiation
treatment for a tumor he had removed from his left shoulder. He had been here
eight times before, so this was business as usual. They got him set up on the
table, and the technician went down the hall to start the treatment. The
technician sat down at the terminal, and hit "x" to start the process. She
immediately realized she made a mistake, since she needed to treat Ray with the
Electron beam, not the X-ray beam. She hit the "Up" arrow, selected the "Edit"
command, hit "e" for Electron beam, and hit "Enter", signifying she was done
configuring the system and was ready to start treatment.
The total time for this interaction was under 8 seconds.
It turns out that this particular sequence of actions within this timeframe had
never occured in all of the testing and evaluation of the Therac-25. If it had
occured, it would have pointed out a dangerous bug in the system. The system
presented the technician with a "Beam Ready" prompt, indicating it was ready to
proceed; she hit "b" to turn the beam therapy on. She was surprised when the
system gave her this error message (no, it wasn't a Mac, but the dialog box
below makes my point well)
She wasn't familiar with this particular message, but these particular errors
usually meant the treatment hadn't proceeded. She cleared the error to reset
the Therac-25 so she could do it again. She got the "Beam Ready" prompt and
again hit "b" to initiate the treatment. Same deal: an error message and the
system stopped. She tried it again.
Meanwhile, back in the treatment room, Ray was feeling repeated burning,
stabbing pains on his back. None of the previous treatments had been like this.
Although he cried out several times, asking (first jokingly) whether the system
was figured right, no one came to check on him. Finally, after the third
painful burst, he pulled himself off the table, and went to the nurses
station...
The problem was this: when the particular sequence of commands was executed
quickly enough (e.g. in under 8 seconds), the arm correctly withdrew as it
should be in Electron beam mode, but the beam switch never occured. Although
the machine told the operator it was in Eletron beam mode, it was actually in a
hybrid proton beam mode. As a result, the system was delivering a radiation
blast of 25,000 rads with 25 million electron volts, more than 125 times the
normal dose. The particular sequence of steps executed by the technician had
moved the metal plate from the beam's path, but left the power setting on
maximum!
Ray Cox's health deteriorated rapidly from radiation burns and other
complications from the treatment overdose. He kept in good humor about his
condition, joking that "Captain Kirk forgot to put his machine on stun". He
died four months later.
It is worth noting that the problem wasn't actually diagnosed until 3 weeks
later, when it happened again to another patient. At this point, the senior
technician realized something about the sequence of steps being taken must be
triggering this flaw. After investigation, he found the problem with the plate,
and reported it to the manufacturer. Subsequent investigation showed that there
had been similar overdoses in Georgia, Canada and Washington.
Lets look at how the systems which house computing and other manipulated
devices can contribute to accidents.
The Therac-25 incident serves as a good example of what are called Latent
Errors. These are "accidents waiting to happen", errors that are virtually
offered by the system. The term recognizes the fact that accidents often result
from a series of errors, often spread out over time. For example, there were a
number of checks in the Therac-25 case that might have prevented the radiation
overdose. There was an audio intercom system between the treatment room and the
control room, but it was broken. There was a video hookup between the two, but
the monitor was disconnected on that particular day. Obviously, the system
itself noticed the error but the message to the user was not evocative enough
to accurately reflect the condition or gravity of the error. Thus, although the
technician noticed she had made a mistake, she assumed the treatment hadn't
happened because that's what there error messages usually meant. Thus, her
subsequent action was erroneous as well because she chose an inappropriate
procedure due to her misinterpretation of the situation (rule-based error).
This kind of progression of multiple errors in procedures, performance and
equipment is seen in many accidents in complex systems. In the 1986 Chernobyl
reactor accident, for example, most of the safety systems had been turned off
because they were running some test experiments. Similar situations occured at
Three-Mile Island (1979) and Bhopal (1984). In all of these cases, a particular
context and sequence of events, sometimes spanning months (e.g. the intercom in
the Therac-25 case had been broken for while--not just that morning), lead up
to the accident. The net effect is that the system virtually elicits the
error from the user.
Therac-25 is a classic example of an error, but errors occur all over in
medicine, with or without the use of computers. Consider:
- Diagnostic Process: Failure to employ indicated tests; Misreading
lab results; Failure to act on the results of monitoring or testing.
- Treatment: Technical error in performance; Error in preparation the
treatment (e.g. dosage); Delayed treatment or inappropriate care.
- Preventive (failure to provide prophylactic treatment): Inadequate
monitoring, Inadequate follow-up of treatment.
- Other: Failure to communicate; Equipment failure; Situated
environments (OR and ICU)
And yet given the acknowledged complexity of the
health care process, it is surprising that medicine has only such a superficial
approach to handling errors and accidents.
Errors typically only become concerns when accidents occur. They typically
result in some sort of punishement, usually aimed to prevent that particular
individual from performing that particular error again. To understand this
approach, it is worth comparing how the fields of Aviation and Medicine
respectively approach the study of human error.
In medicine, the emphasis is on perfection in diagnosis and treatment. Often in
the public eye, physicians are expected to perform their tasks flawlessly as
infallible performers, and any error that occurs is often seen as a failure of
character more than anything else.
In aviation, it is assumed that errors will occur, that they are part of the
accepted risk of flying. Even the best pilots will make errors in judgement or
action, and consequently, aviation systems are designed to try to absorb these
errors through buffers, automation, and redundancy. Procedures are standardized
as much as possible, so that pilots have specific protocols and checklist to
help minimize the occurence of errors.
Medical settings may be more complex than their aviation counterparts. In the
cockpit, for example, there is an officially established hierarchy of command
among the 5 or so crew members. Surgical teams, for example, may be much larger
than a cockpit crew. In the larger context, there are many more facets to the
power structure in the health care setting. A wide variety of players--nurses,
technicians, patients, hospital administrators, national boards, state and
federal regulatory agencies, insurance companies--pursue their goals, creating
a complex set of influences that can lead up to a particular event or accident.
In medicine, error analysis often focuses on individuals and incidents.When an
error occurs, the reaction is to find the immediate cause and correct it; error
correction is typically punishment in the form of malpractice tort litigation.
The underlying assumption seems to be "if we find out who did it and spank them
hard enough, they won't do it again". Thus error analysis usually focuses on
the assignment of blame in malpractice lawsuits, rather than seeking out the
root cause of the problem (often a harder task). Furthermore, because of the
extreme sensitivity to legal impact of error, physicians are often unwilling to
openly discuss slips or mistakes they make in an effort to analyze what
systemic influences may have brought on these errors.
In aviation, there is an on-going effort to track errors and accidents and
attempt to learn from them. There are formal organizations responsible for this
task (see below). Further, because of the assumption that errors occur,
cross-check and verification systems can be put into place to attempt to catch
them before they cause too much harm. Because of the fear of malpractice,
physicians are typically resistant to attempts to document or monitor their
performance and skills.
It should be pointed out that tracking and analyzing errors is hard to do in
medicine. There is no equivalent to a "black box" that records actions and
decisions. Furthermore, unlike aviation where the laws of aerodynamics allow
detailed simulation of how a particular process should occur, there often is no
definitive physiological model of how an particular treatment should progress
in the human body (there are exceptions--see
David
Gaba's work ). To the extent that it is possible, some error analysis
occurs through weekly Mortality and Morbidity reports, incident reports, etc.
But these analyses are less formal than in aviation, and tend to focus on
unique and usual clinical cases or events. Again the burden is placed on the
physician: it is up to the physician to keep track of these incidents, draw
conclusions, and decide whether to alter behavior.
In medicine, certification emphasizes training and education rather than
performance. Board certification often involves taking a separate exam, rather
than monitoring physician performance in the health care setting. Part of this
stems from the way we educate physicians. Compared with aviation, there is less
emphasis on protocol, and more learning by osmosis: we give medical students some
basic tools and thrust them into the health care environment, assuming they
will pick up the knowledge as they go and hope they find good role models to
learn from. Because most of the training occurs in a real health care setting,
there is little opportunity for practice and error, and mistakes (whether by
students or their mentors) are rarely admitted.
In aviation, there are several institutions that monitor the field and concern
themselves with minimizing the occurence of errors in flying. The
Federal Aviation
Administration (FAA) regulates all aspects of flying, including the
educational requirements for pilot training and the correct procedures for
flying aircraft. The National Transportation Safety Board (NTSB) investigates
all accidents related to commercial aviation: they attempt to determine the
specific cause for the crash and attempt to determine whether anything can be
done to avoid a similar accident in the future. These reports are included in a
number of commonly available publications (Aviation Monthly Safety
Summary & Report, NTSB Reporter), and some accident reports are even
available online.
The Air Safety Reporting System
(ASRS) has been set up to allow anonymous reporting of dangerous situations.
They receive over 5000 such notifications per year. It is ironic that the
public expects and accepts such institutionalized safety mechansism in
aviation, but prefers to treat physicians as infallible high priests for whom
mistakes are impossible...
In medicine, there is little standardization across the field, and what
standardization does exist is often limited to the confines of a particular
hospital. As a rule, there is no institutionalized focus on safety, and no
effort to analyze the root causes of accidents (except maybe anesthesia).
Given our newfound understanding of how systems impact human performance, can
we use this knowledge to build better systems?
No.
Just kidding.
But recognizing error for the purpose of design is different than retrospective
analysis of accidents and mishaps. We would like to synthesize system design
principles based on the study of Human Error, principles that help us design
systems that are easy to use and make it difficult for users to make mistakes.
Furthermore, if we admit that we cannot predict and prevent all errors, we must
also design for error tolerance. As it turns out, there are certain broad
themes in the study of human error that can be generalized and used as design
guidelines. This, coupled with a basic knowledge of error analysis, can help
designers build better systems.
It is interesting to note that many of the design principles and heuristics
taught in HCI, Product Design, and similar fields have their roots in the
analysis of Human Error. I found it very rewarding to see that the
recommendations put forth by Apple Computer HCI Guidelines and the early Xerox
STAR work could be supported scientifically by referring to the Human Error and
Cognitive Psychology and Science literature. For example, Don Norman points out
some specific examples of system design principles that are derived from
classes of Human Error (
Norman,
1982 ). Here are some of his points:
- Capture errors imply need for better feedback. In the
Therac-25 incident, the technician dismissed the error dialog under the
assumption that it was essentially analogous to the other error dialogs she'd
seen (i.e. the more familiar of two sequences took over). If the system had
given better feedback as to its internal state (i.e. dangerous level of
radiation was administered), the technician (we hope) would not have dismissed
the error so readily.
- Description errors suggest the need for better system configuration.
These errors occur because an action wasn't "specified" accurately enough,
and a different but similar action took place (e.g. the cake in the refrigerator
and salad in the oven example). These errors are common when operations involve
throwing switches, pushing buttons, etc. and when these operations are similar.
For example, knobs for adjusting different display scales on an EKG machine might
be confused if they look the same and are placed next to each other. One solution
would be to differentiate between the control functions using symbols or more
diverse control placement.
- The inevitability of errors implies a need for reversible actions.
If you take it for granted that even expert users will make mistakes, your
system should be forgiving when they are made. In particular, user
actions should be reversible. On familiar desktop
systems, this is usually supported via the "Undo" command. I have yet to see a
medical software system that supports undo for the majority of its actions...
- Activation problems can be avoided through visible affordances
(that offer users the opportunity to interact with the
system) and feedback (that informs the user of the results of an
interaction). For example, you suffer a loss of activation slip if you
are interrupted while going to fetch something, and then can't remember what
you wanted to get. If you had a check list in your hand which listed things you
needed to retrieve, you would be more likely to remember what you came to get.
An online system could provide similar support by integrating a (visible) model
of the workflow to help the user accomplish all steps the task.
And many
more...
Granted, there is no panacea. But any student of design (interface or other)
will tell you that design isn't a process that lends itself to template
solutions: there is no fixed set of rules that will guarantee success. At best
they can provide some steering aids to help us get the job done. However,
understanding the underlying theory can help us handle situations that aren't
covered by the guidelines. This may be harder than it sounds. It often seems
that designers don't use their knowledge of cognitive science and psychology,
maybe because it doesn't come in these convenient, pre-packaged heuristic forms.
Good question. Here are some things to think about.
- "Error awareness." Hopefully this document and my journal club have
raised your awareness to the variety of errors that can occur in medical work
settings, and provided you with a basic vocabulary and framework
to think and talk about errors. This includes analysing your usability studies
using some of the formal cognitive frameworks we've discussed. Any discussion
of a user's workplace and tasks should consider the errors that occur in that
context. This may be of particular interest to the SIGIR group, which has
obvious interests in modelling user context and tasks.
Ideally, you want to keep a system wide "look-out" for errors, try to find
and identify errors on a routine basis. However, we need to do this without
violating any rules (spoken or unspoken) about accepted behavior in medical
settings. Hopefully, this document can help that process.
- Take a systemic view. Although cognitive theory can explains
what happens in an accident, it is systems analysis that often explains
why. Keep an eye out for larger system problems, since they may be the
ones that really need fixing. When designing health care information systems,
you will come across user errors--remember that they are rarely single,
independent events, but instead part of a larger sequence. When you do find a
problem, try to reason back from the immediate cause to see if you can redesign
and reorganize the process to avoid such problems in the future.
Your analysis of users' tasks, workflow and information needs may point
out problems in the way work is done that exist before you bring in a
computer. You should consider whether the computer systems can help solve these
problems, or whether they need to be solved at a different level (e.g.
administrative, regional, policy, etc.). Remember: "Don't swap at
mosquitoes--drain the swamp!"
- Use errors as tools to analyze your design. You may already be
doing this through User-Centered Design: usability studies give you feedback on
your system design by demonstrating errors that users can make. But remember
that although usability testing can point out some errors, it won't find them
all. Try to prevent errors but design your system to handle them when they do
occur (e.g. forgiveness of user actions, redundancy in data storage, etc.)
- Be willing to redesign. This is probably the biggest challenge to
software designers, but it is central to good design. You must be willing to
throw away some of your work in order to improve the overall system. In some
cases, you may need to redesign the whole system; be willing to look beyond the
immediate causes of the error and consider the larger influences at work.
Granted, this is the holy grail of User-Centered Design, and is subject to
various resource constraints (time, funds, sanity, caffeine), this is the
process to strive for.
- Use simulations when possible. Aviation has benefited from the use
of flight simulators to test new cockpit designs. In many ways this is what
usability studies are for: they attempt to model the user's work environment
using a new tool (interface). In certain cases, however, you may be able to use
a more robust simulation of actual work environments.
David
Gaba's anesthesia simulator, for example, could be used to test out new
designs and interfaces to monitoring devices and information displays.
- Automate data collection for error analysis. One of the great
challenges in the analysis of Human Error is obtaining enough data to
understand how an accident occured. Computers have the ability of recording
user actions automatically, much the way the "black boxes" in airplanes record
all vital flight information. Consider building such tracking features into
your software in order to analyze problems when they do occur.
- Perform structured evaluations to estimate human performance.
Cognitive Science and Psychology offer a number of tools to measure human
performance. Many of these can be used to predict how a user will perform with
a particular interface. For example, Fitt's Law states that the distance to
move your finger across a distance D to a target of size S is I x log2(D/S +
0.5), where I is a speed constant (usually 100 ms). This could be used to
estimate the time it takes to enter codes on a new keypad or screen layout, for
example. Similarly, GOMS and Cognitive Walkthroughs are two ways
to analyze user interfaces by doing cognitive analyses: break tasks down into
small, cognitive sub-components, and measuring how long each sub-component
would take to perform based on assumptions about human performance and
cognitive abilities.
- Anticipate error through better coding. If you assume that
accidents will happen (e.g. network goes down, power cord gets kicked out of
the socket, etc.) Error proofing can include online protocol guidelines or
checklists to provide a visual reminder of what steps need to be taken to
accomplish a particular task, check values to guard against data entry errors
(e.g. lab values within certain ranges), standardization of tasks (e.g.
prescription scales so that nurses don't need to keep track of invidual
physician drug dosage preferences), etc.
The best overview of this topic I found was a JAMA article entitled
Error
in Medicine. For a more in depth survey that looks at various approaches
and research work, there's a new book entitled
Human
Error in Medicine, which I found helpful in preparing my talk.
Set
Phasers on Stun and Other True Tales of Design, Technology, and Human Error
a fun, light read into many of the major technology related accidents over the
past 50 years or so. Not very academic, but fun background reading.
Peter Neuman has written a book called
Computer
Related Risks, which lists various technology related accidents, both minor
and major, that occur throughout the world. The book is largely based on
materials collected through the RISKS forum (comp.risks
newsgroup), which is moderated by Neuman. Check out the newsgroup to see what
kind of fiascos computers have led to recently :)
On a current events note, the January 25, 1995 Stanford Campus Report had an
article on some work Donald Redelmeier, MD did in investigating physician
decision biases which could lead to error. An interesting finding was the
doctors faced with complex decisions will choose the status quo or the distinct
option. In their experiment, they did a baseline study where physicians were
presented with a case and asked to choose whether to treat the patient with
Drug A or leave the patient untreated. In the study group, they showed the same
scenario, but asked whether to treat the patient with Drug A, Drug B, or leave
the patient untreated. Drug A and Drug B were essentially equivalent. In the
later group, physicians were measurably more likely to choose to leave the
patient untreated than in the initial 2-option group.
- Human error in computer systems (Bailey)
- Human error in medicine (Bogner)
- Human Factors Issues in Monitoring (Botney & Gaba)
- Effects of Message Style on Users' Attributions toward Agents (Brennan & Ohaeri)
- Developing an Error Prevention Methodology Based on Cognitive Error Models (Bruno, Welz, & Sherif)
- Set phasers on stun and other true tales of design, technology, and human error (Casey)
- A catalog of errors (human-machine interaction) (Fraser, Smith, & Smith)
- Tasks, Errors and Mental Models (Goodstein, Andersen, & Olsen)
- How to Design Usable Systems (Gould)
- Conceptual modeling for expert system user interface development (Hekmatpour, Vassigh, & Norman)
- The Design of Reliable HCI: The Hunt for Hidden Assumptions (Hollnagel)
- A Methodology for Objectively Evaluating Error Messages (Isa, Boyle, Neal, & Simons)
- Representations in distributed cognitive tasks (Jiajie & Norman)
- Error in Medicine (Leape)
- Designing for Error (Lewis & Norman)
- Computer Related Risks (Neumann)
- Steps Toward a Cognitive Engineering: Design Rules Based on Analyses of Human Error (Norman)
- How goes cognitive science? (Norman)
- Approaches to the study of intelligence (artificial intelligence) (Norman)
- Collaborative computing: collaboration first, computing second (Norman)
- Design principles for cognitive artifacts (Norman)
- How might people interact with agents (Norman)
- Three Mile Island: A Normal Accident (Perrow)
- Human error (Reason)
- Organizations: rational, natural, and open systems (Scott)
- Human error (cause, prediction, and reduction): analysis and synthesis (Senders & Moray)
It would appear that anesthesiology is the one area in medicine that has shown
a concerted ffort to address issues of human error and performance. Part of
this focus is probably due to the clear potential for death or brain death
during surgery. These deaths are at about about 1 in 200,000 now, down from 1
in 10,000 a decade ago; many believe this is partially due to improvements in
ergonomics, operating room layout, monitoring interfaces, and other system-wide
refinements that were suggested by the study of error and performance in
anesthesia.
Dr. David Gaba <gaba@hpp.stanford.edu> at the Palo Alto VA has developed
an anasthesia simulator to help, among other things, train anesthesiologists.
Such a simulation could also be used to test performance using new kinds of
monitoring equipment or other interfaces. For example, running formal usability
tests on a new screen layout to evaluate how effective the layout is in
conveying relevant information.
Note:This simulator is a commercial product, and is not available across
the Internet. There is a separate web page about
the
Anesthesia Simulator Center that can tell you more about the product.
There are a number of things I'd like to add or see added to this page, but
I'm a little burnt out at the moment. Some of them are listed below. If you
have other suggestions or pointers to related information resources,
let me know.
- A catalog or table of categorized error examples. While I found the
descriptions of different error categories pretty straightforward, I sometimes
had trouble coming up with examples (especially in medicine) or using
those categories to analyze accidents that I see happen. This is particularly
true of slips. I would love a big list of anecdotes from medicine, aviation or
elsewhere that would show how a particular error happened (e.g. pictures of
monitors or displays where the facilitated an error).
- A list of journals and magazines where one could find recent work in this area.
- A historical perspective that shows how other fields deal with
human error. Alot of this stuff has actually been, in one form
or another, what psychology deals with -- errors are often the side effects of
psychological events or mechanisms (e.g. a Freudian slip). This document
focuses on Human Error as a separate field, but it (possibly unfairly) ignores
the role of the study of error in psychology. Thanks to Hank Strub for
pointing out this gap in content -- any contributions would be welcome.
Thanks to David Gaba, Tom Rindfleisch, Mike Walker, Larry
Fagan, Eric Horvitz, and MIS crew for helping me put this journal club. Thanks to
Rich Acuff for some details and corrections on the aviation section. Special thanks to
Wanda Pratt, David Gaba and Hank Strub for critiquing the content and interactivity of this web page.