AMA #3: Does ISO 14971 require benefit-risk analysis of ALL individual residual risks?

A common practice in the medical industry is to document benefit-risk analysis of all individual residual risk as an added column in risk assessments. Is this necessary?

Dear colleagues, hello! 👋

Recently, I received the following question:

Many auditors insist on documentation of benefit-risk analysis of every individual residual risk in the risk management file. Does ISO 14971 require that all individual residual risk(s) be analyzed for benefit-risk?

This is an interesting question because there is an ambiguity in how Clause 7.4 in ISO 14971¹ is written within the framework of Clause 7.

Let us briefly review the risk control process prescribed by ISO 14971 to gain a better perspective on this question.

Here is the structure of Clause 7 - Risk control:

1. Clause 7.1: Risk control option analysis

2. Clause 7.2: Implementation of risk control measures

3. Clause 7.3: Residual risk evaluation

4. Clause 7.4 Benefit-risk analysis

5. Clause 7.5 Risks arising from risk control measures

6. Clause 7.6 Completeness of risk control

Let us say we have identified 100 individual risks² during risk analysis. Let us also assume that we have estimated each of these 100 individual risks as a unique combination of a probability of occurrence level and a severity level. Now, we also assume that our initial risk evaluation (Clause 6) for each of these 100 individual risks leads us to conclude that they are not acceptable according to the risk acceptability criteria defined in our risk management plan.

This will lead us to initiate risk control activities according to Clause 7. Our first step will be to consider risk control options according to Clause 7.1.

The following figure partially illustrates the sequence of steps needed during risk control for each of these 100 individual risks³. The red arrow indicates when we would need to analyze benefit-risk of an individual residual risk during the risk control phase.

Figure 1: Partial flowchart illustrating risk control for each individual risk identified during risk analysis. Source: Figure B.1, Annex B, ISO 14971:2019

As shown in Figure 1, there are two scenarios where we would need to consider benefit-risk analysis of an individual residual risk according to Clause 7.4:

1. If at first in 7.1, risk control options analysis, we find that none of these potential risk control options are practicable⁴, or;

2. After implementing risk control options, an individual residual risk still remains unacceptable according to pre-defined criteria in the risk management plan when we evaluate this residual risk in Clause 7.3. Then we go back to Clause 7.1 but we can not find any other practicable risk control options for further risk control. As a result, we will need to go to Clause 7.4 again for benefit-risk analysis of this specific residual risk.

It should be clear that the innermost loop of the flowchart above will need to be executed for each of the 100 individual risks identified during risk analysis.

Let us now reframe the core question:

Is the intent of clause 7.4 in ISO 14971 to require that all individual residual risks are analyzed for benefit-risk?

Let us now take a closer look at Clause 7.4, Benefit-risk analysis

If a residual risk is not judged acceptable using the criteria established in the risk management plan and further risk control is not practicable, the manufacturer may gather and review data and literature to determine if the benefits of the intended use outweigh this residual risk.

If this evidence does not support the conclusion that the benefits outweigh this residual risk, then the manufacturer may consider modifying the medical device or its intended use (go back to 5.2). Otherwise, this risk remains unacceptable.

If the benefits outweigh the residual risk, then proceed to 7.5.

The results of the benefit-risk analysis shall be recorded in the risk management file.

NOTE See ISO/TR 24971⁵ for guidance on performing a benefit-risk analysis.

Compliance is checked by inspection of the risk management file.

Note the use of the term “may” in the clause, which is used to describe permission (e.g., a permissible way to achieve compliance with a requirement or test. Therefore, the benefit-risk analysis of an individual residual risk is not a mandatory “shall” requirement for compliance with ISO 14971.

The partial flowchart in Figure 1, on the other hand, would seem to suggest that we must do a benefit-risk analysis of any individual residual risk that remains unacceptable when further risk control is not practicable. It does not allow us to keep moving forward in Clause 7 when an individual residual risk that is deemed unacceptable and no further risk controls are practicable.

The other choice we can make is to consider modifying the device or its intended use and go back to Clause 5.2 to repeat the whole process. If we decide not to do that either, then the particular individual residual risk remains unacceptable.

Clause 7.4, as written, therefore contains an ambiguity in the context of other steps outlined in the risk control process. When following a linear sequence of steps, as illustrated in the partial flowchart in Figure 1 above, it seems that we must do a benefit-risk analysis in Clause 7.4 for those individual residual risks that remain unacceptable and no further risk controls are practicable. We can also consider modifying the device or its intended use. However, Clause 7.4 does not explicitly stop us from continuing forward with an unacceptable individual residual risk.

So, the short answer to the original question is no, ISO 14971 does not require benefit-risk analysis of all individual residual risks. This conclusion is further supported by the rationale in Annex A, section A.2.7.4:

There can be particular hazardous situations for which the risk exceeds the manufacturer’s criteria for risk acceptability. This subclause enables the manufacturer to provide a high-risk medical device for which they have done a careful evaluation and can show that the benefit of the medical device outweighs the risk. However, this subclause cannot be used to weigh residual risks against economic advantages or business advantages (i.e. for business decision making).

In short, Clause 7.4 offers an option for a manufacturer to consider in case there are a handful of residual risks that remain unacceptable and no further risk control is practicable. It is not a mandatory requirement of the standard.

So why do some auditors ask for documentation of benefit-risk analysis for every individual residual risk?

In my experience, I have seen manufacturers jump through hoops to satisfy some auditors, especially from some of the notified bodies, who expect to see documentation of a benefit-risk analysis of every individual residual risk.

A common practice in the industry is to add an extra column in risk assessments to show that a benefit-risk analysis for every individual risk has been done, indicated by a generic statement “benefits of the intended use outweigh this residual risk”.

This practice does not makes sense, and in most cases, does not accurately reflect that a benefit-risk analysis has indeed been done for every individual residual risk. Auditors simply move on when they see this column and do not generally ask for additional evidence.

So why create this unnecessary and impractical documentation? As explained above, ISO 14971 does not mandate that benefit-risk analysis of every individual residual risk needs to be done.

One possible explanation is that auditors see Figure B.1 in Annex B of ISO 14971:2019 and interpret that to be a normative requirement. Even then, it is hard to understand why Clause 7.4 would be required for every individual residual risk. It becomes relevant only when an individual residual risk is deemed unacceptable and no further risk control is practicable.

The other possible explanation is that auditors interpret some of the regulatory requirements, particularly EU-MDR⁶ (or EU-IVDR) as requiring all individual residual risks to be reduced to as far as possible (AFAP) without adversely affecting the benefit-risk ratio⁷. One way to “justify” an otherwise unacceptable residual risk is through benefit-risk analysis pathway in Clause 7.4 in ISO 14971. In this case, the manufacturer’s risk acceptability criteria would need to be AFAP without adversely affecting the benefit-risk ratio to comply with EU-MDR requirements.

Still, however, it does not make any practical sense to require documentation of benefit-risk analysis of all individual residual risks. As a practical matter, a manufacturer could summarize benefit-risk analysis of only those individual risks that are deemed unacceptable and no further risk control is practicable. A summary description in a risk management report approved by appropriate authorities should be considered sufficient.

In conclusion

ISO 14971 does not require benefit-risk analysis of all individual residual risks.

Clause 7.4 provides an option to manufacturers to accept an otherwise unacceptable residual risk if benefits of the intended use of the device outweigh that specific residual risk based on a careful evaluation of available data and literature. It is not a mandatory requirement because the clause uses “may” and not a “shall” language.

Another alternative available to manufacturers according to Clause 7.4 is to consider modifying their medical device and/or its intended use.

One reason an auditor may be asking for documentation of benefit-risk analysis of all individual residual risks is to ensure compliance to regulatory requirements such as those in EU-MDR (or EU-IVDR). It is a good idea to consult with your regulatory affairs specialists to fully understand the requirement and find the most efficient way to document compliance.

---

I hope you will find value in the AMA feature and consider submitting your own questions!

---

1. ISO14971: Medical devices - Application of risk management to medical devices, third edition (2019-12).

2. ISO 14971 defines the term “risk” as combination of the probability of occurrence of harm and the severity of that harm. In the context of identification of hazards and hazardous situations required by Clause 5.4, an individual risk can be identified as a unique combination of a hazard, hazardous situation and harm. As an example, if there are 5 potential hazards, each associated with 5 different hazardous situations, each of which can lead to 5 different harms, we would have a total of 125 (5x5x5) individual risks.

3. Note that the partial flowchart presented in Figure 1 is extracted from the full flowchart in Figure B.1 of Annex B in ISO 14971:2019. Note further that any information presented in the Annexes is considered informative only and not a normative requirement of the standard.

4. The term “practicable” is not defined in ISO 14971. However, according to guidance in ISO/TR 24971, Annex C, section C.4, practicability of risk controls is not to be confused with practicality. Further, practicability has two components: technical practicability and economic practicability. Technical practicability refers to the ability to reduce the risk regardless of cost. Economic practicability refers to cost and availability implications for a medical device in the context of the overall benefit for public health and society as a whole. Thus practicability of risk control measures should be considered in the context of stakeholder expectations, and are expected to involve a considerable level of judgment.

5. ISO/TR 24971: Medical devices - Guidance on the application of ISO 14971, second edition (2020-06).

6. EU-MDR: Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC.

7. See paragraph 2 and 4(a) in Annex I of EU-MDR, General Safety and Performance Requirements, Chapter I, General Requirements.