“Overconfidence in the performance of AI models in the development phase is a common problem.”
In this Let’s Talk Risk! conversation, David Giese highlights the growing need for a rigorous, disciplined development process with the goal of meeting regulatory requirements for marketing authorization. He shares how AI developers are often over-confident in the performance of their models, but disappointed when these models are subjected to rigorous scrutiny during the regulatory review.
Software as a Medical Device (SaMD), including AI/ML devices, are growing exponentially in MedTech. Rapidly evolving technology offers an opportunity to develop innovative products, but it also presents new risks. Security and Cybersecurity concerns are on the rise, as well as, concerns about transparency and explainability of AI/ML models.
David points to new guidance documents from the FDA and emerging standards. As an example, software developers should consider using IEC 81001-5-1 - Health software and health IT system safety, effectiveness and security - along with IEC 62304 - Medical device software - Software life cycle processes.
Listen to this Let’s Talk Risk! conversation with David Giese, which also includes an open discussion with a live audience. Jump to a section of interest using these timestamps.
00:01:30 Introduction
00:02:30 Current state of AI/ML applications in MedTech
00:04:05 Different types of AI models - adaptive vs. generative AI
00:06:25 Challenges in meeting regulatory expectations for AI in MedTech
00:08:18 Use of synthetic data in AI model development
00:09:30 Best practices for AI development and regulatory submissions
00:12:02 Cybersecurity challenges for SaMD and AI/ML devices
00:14:03 New skills risk practitioners need to develop
00:15:09 Open discussion and audience Q&A
00:27:20 Closing comments and key takeaways
If you enjoyed this podcast, consider subscribing to the Let’s Talk Risk! newsletter.
Suggested links:
LTR: EU vs. FDA - Aligned but different in approach to safety of AI/ML devices
LTR: A new paradigm for building connected medical devices
FDA: Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions, Issued September 2023.
About J. David Giese
J. David Giese is currently President and Partner at Innolitics, a consulting firm specializing in development of new medical software and regulatory submissions to the FDA. His team has helped develop and receive FDA clearance for over 60 medical devices, both SaMD and SiMD, over the last 12 years. His expertise include application development, custom software development, engineering design, project management, user experience design, cybersecurity, software testing and technical writing. He holds Bachelors and Masters degrees in Biomedical Engineering.
Let’s Talk Risk! with Dr. Naveen Agarwal is a weekly live audio event on LinkedIn, where we talk about risk management related topics in a casual, informal way. Join us at 11:00 am EST every Friday on LinkedIn.
Disclaimer
Information and insights presented in this podcast are for educational purposes only. Views expressed by all speakers are their own and do not reflect those of their respective organizations.
Share this post