AMA #3: Does ISO 14971 require benefit-risk analysis of ALL individual residual risks?
A common practice in the medical industry is to document benefit-risk analysis of all individual residual risk as an added column in risk assessments. Is this necessary?
Dear colleagues, hello! 👋
Recently, I received the following question:
Many auditors insist on documentation of benefit-risk analysis of every individual residual risk in the risk management file. Does ISO 14971 require that all individual residual risk(s) be analyzed for benefit-risk?
This is an interesting question because there is an ambiguity in how Clause 7.4 in ISO 149711 is written within the framework of Clause 7.
Let us briefly review the risk control process prescribed by ISO 14971 to gain a better perspective on this question.
Here is the structure of Clause 7 - Risk control:
Clause 7.1: Risk control option analysis
Clause 7.2: Implementation of risk control measures
Clause 7.3: Residual risk evaluation
Clause 7.4 Benefit-risk analysis
Clause 7.5 Risks arising from risk control measures
Clause 7.6 Completeness of risk control
Let us say we have identified 100 individual risks2 during risk analysis. Let us also assume that we have estimated each of these 100 individual risks as a unique combination of a probability of occurrence level and a severity level. Now, we also assume that our initial risk evaluation (Clause 6) for each of these 100 individual risks leads us to conclude that they are not acceptable according to the risk acceptability criteria defined in our risk management plan.
This will lead us to initiate risk control activities according to Clause 7. Our first step will be to consider risk control options according to Clause 7.1.
The following figure partially illustrates the sequence of steps needed during risk control for each of these 100 individual risks3. The red arrow indicates when we would need to analyze benefit-risk of an individual residual risk during the risk control phase.
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffac2d970-c67a-4728-8a6c-b7e42817b6e0_750x655.png)
As shown in Figure 1, there are two scenarios where we would need to consider benefit-risk analysis of an individual residual risk according to Clause 7.4:
If at first in 7.1, risk control options analysis, we find that none of these potential risk control options are practicable4, or;
After implementing risk control options, an individual residual risk still remains unacceptable according to pre-defined criteria in the risk management plan when we evaluate this residual risk in Clause 7.3. Then we go back to Clause 7.1 but we can not find any other practicable risk control options for further risk control. As a result, we will need to go to Clause 7.4 again for benefit-risk analysis of this specific residual risk.
It should be clear that the innermost loop of the flowchart above will need to be executed for each of the 100 individual risks identified during risk analysis.
Let us now reframe the core question:
Is the intent of clause 7.4 in ISO 14971 to require that all individual residual risks are analyzed for benefit-risk?
Let us now take a closer look at Clause 7.4, Benefit-risk analysis
If a residual risk is not judged acceptable using the criteria established in the risk management plan and further risk control is not practicable, the manufacturer may gather and review data and literature to determine if the benefits of the intended use outweigh this residual risk.
If this evidence does not support the conclusion that the benefits outweigh this residual risk, then the manufacturer may consider modifying the medical device or its intended use (go back to 5.2). Otherwise, this risk remains unacceptable.
If the benefits outweigh the residual risk, then proceed to 7.5.
The results of the benefit-risk analysis shall be recorded in the risk management file.
NOTE See ISO/TR 249715 for guidance on performing a benefit-risk analysis.
Compliance is checked by inspection of the risk management file.
Note the use of the term “may” in the clause, which is used to describe permission (e.g., a permissible way to achieve compliance with a requirement or test. Therefore, the benefit-risk analysis of an individual residual risk is not a mandatory “shall” requirement for compliance with ISO 14971.
The partial flowchart in Figure 1, on the other hand, would seem to suggest that we must do a benefit-risk analysis of any individual residual risk that remains unacceptable when further risk control is not practicable. It does not allow us to keep moving forward in Clause 7 when an individual residual risk that is deemed unacceptable and no further risk controls are practicable.
The other choice we can make is to consider modifying the device or its intended use and go back to Clause 5.2 to repeat the whole process. If we decide not to do that either, then the particular individual residual risk remains unacceptable.
Clause 7.4, as written, therefore contains an ambiguity in the context of other steps outlined in the risk control process. When following a linear sequence of steps, as illustrated in the partial flowchart in Figure 1 above, it seems that we must do a benefit-risk analysis in Clause 7.4 for those individual residual risks that remain unacceptable and no further risk controls are practicable. We can also consider modifying the device or its intended use. However, Clause 7.4 does not explicitly stop us from continuing forward with an unacceptable individual residual risk.
So, the short answer to the original question is no, ISO 14971 does not require benefit-risk analysis of all individual residual risks. This conclusion is further supported by the rationale in Annex A, section A.2.7.4:
There can be particular hazardous situations for which the risk exceeds the manufacturer’s criteria for risk acceptability. This subclause enables the manufacturer to provide a high-risk medical device for which they have done a careful evaluation and can show that the benefit of the medical device outweighs the risk. However, this subclause cannot be used to weigh residual risks against economic advantages or business advantages (i.e. for business decision making).
In short, Clause 7.4 offers an option for a manufacturer to consider in case there are a handful of residual risks that remain unacceptable and no further risk control is practicable. It is not a mandatory requirement of the standard.
So why do some auditors ask for documentation of benefit-risk analysis for every individual residual risk?
In my experience, I have seen manufacturers jump through hoops to satisfy some auditors, especially from some of the notified bodies, who expect to see documentation of a benefit-risk analysis of every individual residual risk.
A common practice in the industry is to add an extra column in risk assessments to show that a benefit-risk analysis for every individual risk has been done, indicated by a generic statement “benefits of the intended use outweigh this residual risk”.
This practice does not makes sense, and in most cases, does not accurately reflect that a benefit-risk analysis has indeed been done for every individual residual risk. Auditors simply move on when they see this column and do not generally ask for additional evidence.
So why create this unnecessary and impractical documentation? As explained above, ISO 14971 does not mandate that benefit-risk analysis of every individual residual risk needs to be done.
One possible explanation is that auditors see Figure B.1 in Annex B of ISO 14971:2019 and interpret that to be a normative requirement. Even then, it is hard to understand why Clause 7.4 would be required for every individual residual risk. It becomes relevant only when an individual residual risk is deemed unacceptable and no further risk control is practicable.
The other possible explanation is that auditors interpret some of the regulatory requirements, particularly EU-MDR6 (or EU-IVDR) as requiring all individual residual risks to be reduced to as far as possible (AFAP) without adversely affecting the benefit-risk ratio7. One way to “justify” an otherwise unacceptable residual risk is through benefit-risk analysis pathway in Clause 7.4 in ISO 14971. In this case, the manufacturer’s risk acceptability criteria would need to be AFAP without adversely affecting the benefit-risk ratio to comply with EU-MDR requirements.
Still, however, it does not make any practical sense to require documentation of benefit-risk analysis of all individual residual risks. As a practical matter, a manufacturer could summarize benefit-risk analysis of only those individual risks that are deemed unacceptable and no further risk control is practicable. A summary description in a risk management report approved by appropriate authorities should be considered sufficient.
In conclusion
ISO 14971 does not require benefit-risk analysis of all individual residual risks.
Clause 7.4 provides an option to manufacturers to accept an otherwise unacceptable residual risk if benefits of the intended use of the device outweigh that specific residual risk based on a careful evaluation of available data and literature. It is not a mandatory requirement because the clause uses “may” and not a “shall” language.
Another alternative available to manufacturers according to Clause 7.4 is to consider modifying their medical device and/or its intended use.
One reason an auditor may be asking for documentation of benefit-risk analysis of all individual residual risks is to ensure compliance to regulatory requirements such as those in EU-MDR (or EU-IVDR). It is a good idea to consult with your regulatory affairs specialists to fully understand the requirement and find the most efficient way to document compliance.
I hope you will find value in the AMA feature and consider submitting your own questions!
ISO14971: Medical devices - Application of risk management to medical devices, third edition (2019-12).
ISO 14971 defines the term “risk” as combination of the probability of occurrence of harm and the severity of that harm. In the context of identification of hazards and hazardous situations required by Clause 5.4, an individual risk can be identified as a unique combination of a hazard, hazardous situation and harm. As an example, if there are 5 potential hazards, each associated with 5 different hazardous situations, each of which can lead to 5 different harms, we would have a total of 125 (5x5x5) individual risks.
Note that the partial flowchart presented in Figure 1 is extracted from the full flowchart in Figure B.1 of Annex B in ISO 14971:2019. Note further that any information presented in the Annexes is considered informative only and not a normative requirement of the standard.
The term “practicable” is not defined in ISO 14971. However, according to guidance in ISO/TR 24971, Annex C, section C.4, practicability of risk controls is not to be confused with practicality. Further, practicability has two components: technical practicability and economic practicability. Technical practicability refers to the ability to reduce the risk regardless of cost. Economic practicability refers to cost and availability implications for a medical device in the context of the overall benefit for public health and society as a whole. Thus practicability of risk control measures should be considered in the context of stakeholder expectations, and are expected to involve a considerable level of judgment.
ISO/TR 24971: Medical devices - Guidance on the application of ISO 14971, second edition (2020-06).
EU-MDR: Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC.
See paragraph 2 and 4(a) in Annex I of EU-MDR, General Safety and Performance Requirements, Chapter I, General Requirements.
Hi Naveen,
I have read your article and here are my comments.
I totally agree with you, that the 7.4 chapter about benefit-risk-evaluation of individual risks (!) is only necessary, if the residual risk is not acceptable and there are no more countermeasures possible.
I also agree with you that there are sometimes auditors insisting on all individual benefit-risk-evaluations and also medical device manufacturers that are doing it voluntarily because they think they have to do it.
In both cases my personal assumption is, that they did not read chapter 7.4 carefully and simply overlooked the restriction to the "non-acceptable risks". Especially auditors are often not well trained or experienced with ISO 14971 and thus have a lack of understanding.
Your hypothesis is, that auditors in Europe might refer to the statement in Annex I of the MDR about reducing risks "as far as possible without adversely affecting the benefit-risk ratio". And this might lead to the interpretation that every single residual risk needs a benefit-risk-evaluation.
The question is: what does MDR instead mean by "reducing risks as far as possible without adversely affecting the benefit-risk ratio"? I think they wanted to express: If you introduce more and more and more countermeasures, there will get the point where the risk is close to zero, but so is also the benefit! This does not mean, that you formally have to do a benefit-risk-evaluation. It just means that you stop with countermeasures if they will have a heavy impact on the benefit of the device.
What you have to do in every row of the risk table: do not judge the residual risk by the acceptance graph only, evaluate also whether it is reduced as far as possible. But I would never do a benefit-risk-evaluation for every residual risk in every row.
The benefit-risk-ratio is by the way already determined at the beginning of the risk management process when designing the ratio of the red and green area in the risk graph. The more benefit the device has, the more risks you can except. And this is expressed not only qualitatively but also quantitatively based on the classes and their defintions. By that a residual risk that lays in the green area, already is evaluated indirectly for benefit-risk-ratio.
Best regards
Christian
In speaking with BSI people, I specifically asked about this and the answer may surprise you...
According to BSI (and other consultants who have sat with the Commission asking the same question), EVERY risk has to have a B-RA. Now, the question as to HOW is interesting.
There is division on whether to have a B-RA immediately written inline with the risk on your risk management documents (FMEA, PHA, etc.) OR if a collection of acceptable risks can be all combined within your risk management documentation as a "risk catch-all." Some companies prefer to have their B-RA right at the end of each line item and some like to have a reference to a document that talks about the acceptable risk.
I realize, Naveen, this contradicts your post somewhat, but realize that what is written in the regulations and what the Commission is asking for are sometimes two different things. I have numerous examples of that in the MDR. ISO 13485 and ISO 14971 have some of that going on, as well. For now, I'm told, make sure you have a B-RA for every line item, whether right there in that line item or referenced to an overall document that calls out the risk OF that line item.