The Fundamentals of a Human Factors Program

Edmond-Israelski-web Rob-Ngungu-web Eric-Schaffer web
Ed Israelski Rob Ngungu Eric Schaffer
Convener of IEC and ISO Joint Working Groups on Usability Engineering Standards

Director Regulatory Affairs and Product Development Strategy, Austen BioInnovation Institute

CEO and Head of Human Factors International Laboratories

Device manufacturers with a weak approach to human factors and usability engineering will need to increase their focus on this function, as regulatory bodies place a greater emphasis on product design safety and effectiveness and the orthopaedic industry becomes more competitive.

Human factors/usability engineering intentions, risk and testing are part of FDA’s premarket approval process and are sometimes required for a 510(k). Following stricter protocols—and recalls—in the U.S., the European Union, Japan and China are raising regulatory expectations surrounding this function. Experts note that the release of new guidelines and the greater attention placed on human factors internationally should drive manufacturers to implement practices that demonstrate the value of human factors to the entire organization.

Prudent manufacturers already know that the deployment of human factors and usability engineering go beyond meeting regulatory requirements. Manufacturers can leverage their programs to cut costs, mitigate risks and understand how they stack up to competition in the market.

“A lot of manufacturers that haven’t incorporated human factors as part of their culture are doing so now because it’s required by regulators,” says Ed Israelski, Ph.D., Convener of IEC and ISO Joint Working Groups on Usability Engineering Standards. “They think they can do the bare minimum, and that’s a mistake. There are great business reasons for doing good human factors. Your product will sell better; will be more competitive; will have less costs associated with supporting it; you’ll have fewer complaints, returns, recalls and fewer field actions. Also, you’ll spend less time training people if it’s a good, easy to use product.”

Rob Ngungu, Director Regulatory Affairs and Product Development Strategy, Austen BioInnovation Institute, illustrated this point in his OMTEC 2015 presentation, Applying Human Factors to Usability Engineering.

 
 How are Human Factors Defined?
 

Human factors:

“… the application of knowledge about human capabilities (physical, sensory, emotional, and intellectual) and limitations to the design and development of tools, devices, systems,
environments, and organizations…”
(ANSI/AAMI HE75:2009)

Usability:

“Characteristics of the user interface that establishes effectiveness, efficiency, ease of
user learning and user satisfaction.”
(ISO/IEC 62366”2007)

              

“It’s useful to conduct some type of comparable usability study with a protocol and three or four competing devices that you put a series of users through,” Ngungu says.

When Austen BioInnovation brought in nurses and physicians from local hospitals to test different brands of a similar product, they learned that stakeholders gravitated to the easiest brand to use and ignored the other devices, which then sat unused. When manufacturers compare failure data from the field or FDA, the number of adverse events might mean that the product does indeed have a serious flaw—or it might mean that one product is being used more than the other, and therefore has a greater probability for adverse events.

Hospitals will restock the product that is more highly used; therefore, manufacturers should consider marketing and sales data when understanding the usability of a device. “In some cases, you may not have complaints; however, you’re losing market share,” Ngungu says.

Manufacturers need to implement a holistic approach to their human factors programs. This outlook often falls short in proper planning for people, design and testing.

People Requirements
When it comes to people, first, there is a lack of commitment from leadership to understand human factors and how the function should be incorporated into the company culture.

“Executives are starting to get behind human factors engineering efforts, which is great,” says Eric Schaffer, CEO and Head of Human Factors International Laboratories. “The problem is that they don’t know what it means. They know the word—whether it’s user experience, customer experience or patient experience—but they don’t know what it takes to do it, and they don’t understand that it’s a profession like any other.”


A critical element to the integration of human factors within a manufacturer is management’s recognition that the company needs the right expertise. Human factors engineering is a discipline that, like any other, takes training and knowledge. 

“In reality, you need professional human factors engineers,” Schaffer says. “When we see operations handled in a mature way, you have professionals that are properly trained and certified. You have them working in an organization structure that integrates them into projects. They’re working against a defined methodology with specific deliverables and activities, using a set of tools and techniques that are well established in the field. We know how to do the work, but it’s another thing to see it properly applied in an organization. Very often I see organizations approach it with a lick of paint. You might get away with that on your website, but for the medical device that operates in complex ecosystems with lives at stake, it is not okay.”  

Manufacturers also fail to recognize that a successful outcome—in this case, a well-designed, intuitively-used device—requires input from several departments.

“You cannot rely on the engineering effort alone in order to pull off a sound usability/human factors engineering program,”Ngungu says. “These devices are used by nurses, doctors and even in some cases the patients. There are various disciplines, including industrial engineering and marketing folks, who know what an acceptable standard would look like. Most companies will bring in just the engineering side and not the clinical side, who know how this device is used, and no one from the marketing side, who knows how to present it.”

Design Requirements
Manufacturers struggle to choose the point during the design process wherein they should incorporate human factors. The answer is: from the start. Usability specification should be included in design input and output requirements. 

Ultimately, the foundation of any sturdy usability program relies on a thorough understanding of the way that people are actually going to use your device. To do this, manufacturers need to define the users. The questions that should be answered include: Who is the user? Who is the patient? What is the complexity of the environment in which the product is being used? What is the workflow before and after use of the product? Do you need to take into account global considerations? Can user error result in injury, or even death?

Data from prospective users and competitive intelligence needs to be incorporated in the design. “Look at all of the FDA data and the issues that similar devices have encountered, and make sure those are captured,” Ngungu says. “It doesn’t mean that you’re going to be able to hit all of them. It means that it is part of your design goal. Maybe you only get 50 percent of the way there, but it’s better than not having it in the first place. You would rather have 100 requirements that you threw in and hit 20 of them than have zero requirements and hit zero. There is value in making an attempt so that you know it was a consideration that went into the thought process.” 

As Ngungu puts it, there’s no such thing as 100 percent safe. In designing the various components of a device and its packaging, manufacturers need to define how much risk they’re willing to accept. In determining acceptance criteria, he recommends the 90 percent standard, in which 90 percent of the population of users should be able to execute each needed step without fault. Experts say that manufacturers also do not account for all of the possible risk factors and do not perform a thorough-enough risk analysis during the design process. 

Testing Requirements
FDA’s intense interest around human factors continues to focus on who will use the device, user-related risks and validation testing of the product. R&D, clinical and marketing all need to be aware of what processes need to take place for a successful regulatory approval. 


3 Documents to Read to Enhance
Your Usability Program

Regulatory bodies around the world are showing greater interest in human factors. Whether you’re a novice or an expert on the topic, it’s essential you stay up to date on the latest changes set forth by government and standard bodies. Ed Israelski, Ph.D., Convener of IEC and ISO Joint Working Groups on Usability Engineering Standards, recommends that device manufacturers familiarize themselves with these three important documents. 

1. FDA Draft Guidance: Applying Human Factors and Usability Engineering to Optimize Medical Device Design.
This document, issued in 2011 and expected to be finalized by the end of 2016, calls on manufacturers to consider more in their human factors studies, such as sample size and the introduction of a delay or learning delay period between when you train people for a usability study and when you actually test them.

“[FDA] raised the bar in terms of the strictness of the protocol and what you measure,” Israelski says. “Each observation in a usability study needs to be probed deeply. They really don’t want numerical acceptance criteria, which were reflected in the form of usability objectives. FDA wants you to use qualitative data; every observation you make in your early formative studies, and in your validation study at the end. Validation needs to be examined carefully for its implications for further design improvement and whether any residual risk that exists is acceptable. They want it to be a thorough analysis of everything you observe. They want you to probe subjectively, even when people do everything correctly.”

2. ANSI/AAMI HE 75:2009 Human Factors Engineering – Design of Medical Devices.
This extensive 500-page standard covers good design principles for medical devices, and is currently being updated.

3. IEC 62366-1 Application of
Usability Engineering to Medical Devices
.

This version was released in early 2015 and is the updated standard of the 2007 version, recognized worldwide.

“It’s the most current description of the process for usability engineering/human factors engineering,” Israelski says. “It is much improved; the older version was not as clear.”

According to Israelski, a second part to this standard is expected to be released in early 2016 and will provide tutorials and examples of the usability engineering process.

“[IEC 62366-2] will make IEC 62366-1 easier to use,” Israelski says. “Part two also directs that you must use IEC 62366 to produce a product that is focused on safety and clinical effectiveness, but you could also use human factors usability engineering for business purposes, to be more competitive and have a product that doesn’t get returned to you by unhappy users. Part two will give you more material to broaden the scope of your human factors work to first address safety and effectiveness, and second, to address business reasons for having good usability.”


First, for time and cost purposes, it is crucial for manufacturers to properly distinguish what tests will be performed to collect user expectations and clinical outcomes data. The distinction between the two can be difficult for manufacturers. Clinical trials focus on effectiveness and safety. Usability studies focus on user expectations and safety.

“This is critical because the rules for clinical trials are so stringent,” Ngungu says. “People create protocols and think they have an absolute usability protocol, and it has nothing to do with a clinical trial, but they’ll take it to a hospital or institute that refuses to accept it unless they go through an institutional review board (IRB). Then, they end up changing the test to meet the IRB requirement and lose all of the emphasis that they were looking for, which was usability. They end up collecting data that is great for the IRB and the institution, but doesn’t do any good in terms of usability of the device.”

Also, if you’re caught collecting clinical data without IRB approval, you cannot use that data to support a submission and you’re violating human subject protection rules.

Successful usability tests account for all of the questions mentioned above in regard to the user and the effectiveness and safety of the device. FDA in particular is asking manufacturers for greater detail regarding sample size and the learning delay period between when a user is trained and then tested.

“The manufacturer will often know that this is a problem area, and will seek to train people not to do it, and will emphasize it in instructional materials. Manufacturers insist that they just don’t have the time or budget to continue to refine their design,’” Israelski says. “But the very first thing you want to do is design out the likelihood of user error, not just instruct people to avoid it or train them to avoid it through labeling.”

Manufacturers cannot rely on users memorizing instructions for use or even quick reference cards. While these guides are necessary, experts like Israelski and Ngungu say that the tendency for users to return to these manuals is rare.

“Experience tells us that your number one winner is intuitiveness,” Ngungu says. “How intuitive is the design? Not how good are my instructions.”

Manufacturers that address these common mistakes and consider these points may achieve not only a greater level of understanding of human factors requirements, but a tool to measure their device’s viability and lifespan in the market.


Hannah Corcoran is Editorial & Media Assistant at ORTHOWORLD. She can be reached by This email address is being protected from spambots. You need JavaScript enabled to view it..