Case study: Using post-market data to evaluate changes in risk level

How you can estimate P1 and P2 from reported adverse events to estimate the probability of harm and monitor changes in risk levels.

What if we could develop a method to directly estimate a probability of occurrence of harm (POH) associated with device malfunctions using actual post-market data?

This could not only help us monitor changes in the risk level for an existing device, but it could also provide a set of baseline data to use in the development of the next generation of devices.

A common challenge during the post-market phase of a medical device is to determine if there is a change in previously estimated risk levels based on complaints and adverse events.

It is a requirement of ISO 14971, the international standard for application of risk management to medical devices, to review information collected during the post-market phase to determine if the risk levels have changed and if any individual risk is no longer acceptable¹. If risk levels have changed to a level no longer acceptable, then timely action is required to ensure that a medical device continues to remain safe and effective throughout its life cycle.

However, this proves to be a challenging exercise in the industry. One reason is the common practice of using an FMEA² as the only method of risk analysis, which fails to correctly and completely identify the risk of harm arising from each hazardous situation. A hazardous situation may arise even when a device is operating in the normal mode and there is no failure. Further, not all failure modes will necessarily result in a hazardous situation. Therefore a direct one-to-one link between failure modes and different hazardous situations is not feasible in an FMEA.

Another industry practice is to use the P1, P2 method³ of estimating the probability of occurrence of harm (POH) in an FMEA to comply with ISO 14971 requirements for risk analysis. FMEA is a technique for analyzing individual failure modes. Therefore, the estimated risk level arising due to a failure mode is applicable only to that specific failure mode/cause combination. In practice, a device malfunction may occur due to a combination of different failure modes through a sequence of events. Therefore, it is not feasible to isolate a specific failure mode/cause combination when a particular device malfunction is reported as a complaint.

As a result, it is generally not feasible to determine if previously estimated risk levels have changed based on actual complaints and adverse events. The best we can do is to monitor the frequency of adverse events and look for unusual trends or outliers.

It is like flying blind, with no way to course-correct in time, because we don’t know how far we have deviated from expected levels!

In this case study, publicly available data is used to develop a method of estimating POH using the P1, P2 approach. Although there are several limitations to this approach, it is one way of potentially creating a more direct link between risk estimation during product development and post-market surveillance.

Case Study: HeartWare ventricular assist device (HVAD) system

The HVAD system is indicated for patients with end-stage heart failure either as a bridge therapy to heart transplant (BTT), or as destination therapy (DT)⁴. As shown in the following figure, it includes an implantable mechanical pump to drive blood from the left ventricle to the body, a driveline that connects to an external controller, dual power sources and a data monitor.

Figure 1: HVAD system components. Image source: fda.gov

Medtronic acquired the HVAD system from HeartWare, International Inc. in 2016⁵ to expand its $5 billion portfolio of Cardiac Rhythm and Heart Failure products. But, in a short span of 5 years, Medtronic announced discontinuation of the sale and distribution of the HeartWare Ventricular Assist Device (HVAD) from the market. This announcement came after multiple recalls associated with a pump malfunction and a higher risk of death or other neurological adverse events compared to another commercially available device. In short, the benefit-risk evaluation of the HVAD system was no longer favorable to support a claim of continued safety and effectiveness.

In December 2020, Medtronic shared details of the pump malfunction involving delay to restart, or failure to restart after a pump stop event⁶.

An internal pump component from three (3) specific lots puts a subset of the finished pumps at higher risk of delay to restart or failure to restart. The risk exists only when the pump is stopped, for example, in a controller exchange when an attempt is made to restart the pump. A delay to restart or failure to restart could occur at any time after a pump stop, even if the pump initially started at the time of implant. If a pump has successfully restarted after a pump stop event, a delay to restart or failure to restart could be experienced in the future.

The implanted pump is expected to provide continuous circulatory support to the patient, and a delay or failure to restart after a pump stop event could lead to a life-threatening situation, including death. This issue is so critical that the system is designed to have multiple redundancies such as dual sources of power (AC or DC adapter plus a battery, or two batteries) to be connected to the controller at all times. The pump is also designed to run with dual stators, but it can continue to operate momentarily with a single stator when the electrical contact is interrupted.

Indeed, Medtronic reported 26 complaints between March 1, 2017 and November 16, 2020, which were directly attributed to this specific malfunction:

As of November 16, 2020, Medtronic has identified two (2) deaths, nine (9) cases of critical harm (such as cardiac arrest or reoperation for pump exchange), seven (7) cases of major harm (such as hospitalization or prolonged implant procedure due to interoperative pump exchange), and eight (8) cases of negligible harm (such as a potentially life-threatening event in which the patient recovered without long term effects, or a patient experienced a delay in implant).

A total of 506 HVAD systems were manufactured and distributed in this timeframe with the impacted components from the 3 specific lots. Medtronic estimated a failure rate of 5.7% for pumps in this subset, compared to a failure rate of 0.087% for general population pumps when operating normally (dual stator) and a failure rate of 0.4% when operating in single stator.

We will use this information to develop a method for applying the P1, P2 approach to estimate the POH. Let us dive in.