RCA Radio is a podcast bringing you the latest news and insights in regulatory, compliance, and quality assurance. A transcript for “Episode 001: AI & Machine Learning in Medical Device” follows. Listen to the episode here and on your favorite listening apps, including Apple Podcasts and Spotify.
Erika Porcelli Hello and welcome to RCA Radio, a podcast covering the latest news and challenges in regulatory compliance and quality assurance facing the pharmaceutical and medical device and biologics industries. I’m your host, Erika Porcelli. I’m the vice president of client relations at Regulatory Compliance Associates. In this episode, we’re covering everything you need to know about the FDA’s proposed regulatory framework for artificial intelligence and machine learning based software as a medical device. For today’s episode, I’m joined by Lisa Michels. Lisa serves as general counsel and regulatory affairs expert for RCA. Lisa, good morning and welcome.
Lisa Michels Good morning, Erika. Thank you so much.
Erika Porcelli So there’s been a lot of talk in the industry the last couple of years on artificial intelligence. What is the history behind and the purpose of the FDA’s discussion paper and request for feedback on the proposed regulatory framework?
“Due to the constant emergence of new novel and cutting edge medical devices and software technologies, the FDA has had this goal to establish more robust regulatory oversight for these evolving products to ensure patient safety.”
Lisa Michels This discussion paper, which has been issued by the FDA, is an output of a number of FDA initiatives that have been culminating over the course of the last several years. And they were initiated in an effort to keep pace with the ever changing regulatory landscape.
Of course due to the constant emergence of new novel and cutting edge medical devices and software technologies, the FDA has had this goal to establish more robust regulatory oversight for these evolving products to ensure patient safety. So to meet this objective, the FDA has put forth this discussion paper to gauge what the industry’s feedback would be on this approach. And they’ve proposed this framework that they’re referring to as a total product life cycle based regulatory framework, for the artificial intelligence and the machine learning based software devices. And where the history of this proposed framework comes from is based on the organization known as the International Medical Device Regulators Forum, also more known as the IMDRF.
And this is an organization that many countries belong to. The FDA is one of the members of this harmonized group. And their guidance in terms of risk categorization, the principles that they’ve defined in terms of the benefit risk methodology and the risk principles for software really lay the foundation for this framework as well as the FDA’s initiatives related to guidance documents issued on software modifications, and their other efforts in the area of digital health initiatives. So this position paper is posed to industry to solicit feedback and comments on how these products should be regulated.
Erika Porcelli So I think in the name of the proposed regulatory framework, we talk about artificial intelligence and machine learning. And I’m going to make some assumptions that this could potentially cause some confusion in the industry because they are somewhat similar. Could you give us maybe an overview of what artificial intelligence means?
Lisa Michels Sure. So artificial intelligence, it essentially refers to products that incorporate certain software programs or algorithms that are able to analyze and continually adapt to new data. And that’s why they’re referred to as artificial intelligent, because the technology of the software and the algorithms is very intelligent. And the techniques and models that are used in order for the software to be able to analyze and continually adapt to this new data. It may be based on how the software is programmed in terms of its ability to do statistical data analysis. Relying on expert systems that rely on a decision tree, if-then statements, and other machine learning techniques. And essentially the artificial intelligence algorithms are the software that learn from data that’s new data that’s received, and then act on that data.
“[Artificial intelligence] essentially refers to products that incorporate certain software programs or algorithms that are able to analyze and continually adapt to new data.
Machine learning is an artificial intelligence technique, and it’s used to essentially design and train the software algorithms to learn and act on that data.”
Erika Porcelli So what is machine learning then? Because they can be somewhat interchangeable I suppose. But how would you, I guess define that?
Lisa Michels So they are very similar. Obviously they’re connected. Machine learning is an artificial intelligence technique, and it’s used to essentially design and train the software algorithms to learn and act on that data. And there are certain types of machine learning techniques that the software developers use to create either locked algorithms, or adaptive algorithms. And the algorithms themselves when they’re locked, they essentially do not change. And the adaptive algorithms are those that can change over time.
Erika Porcelli So do you see one or the other being more prevalent, or does it really just depend on the circumstances?
Lisa Michels It depends on the circumstances of what the device is and how it’s designed. I will talk about each, if that would be helpful, I can discuss the locked algorithm and then give you a little more background on the adaptive algorithm.
Erika Porcelli Yeah, that would be great. Thank you.
Lisa Michels So the locked algorithm is one that doesn’t change each time an algorithm is used. And the manufacturer controls the intervals when the locked algorithms may be changed based on specific training data, and process validation. So that they can ensure that the system performs and functions as intended.
Locked algorithms are those that provide the same result each time the same input is provided, and they apply a fixed function. So some of the examples or techniques are having a static look up table, decision trees, or a complex classifier. And those are applied to a given set of inputs. Whereas an adaptive algorithm is one that does change over time based on new data or input.
And these types of algorithms actually introduce a bit more risk because they’re designed to evolve on their own based on receiving new data. And this in the FDA’s eyes, can raise some concerns about safety. The adaptive algorithms are able to learn from new user data presented to the algorithm through real world data. And because it’s a continuing learning algorithm, it changes its behavior using a defined learning process. And the adaptive algorithm, when it changes over time for a given set of inputs, that output could actually be different before and after those changes are implemented.
“[Adaptive] algorithms actually introduce a bit more risk because they’re designed to evolve on their own based on receiving new data. And this in the FDA’s eyes, can raise some concerns about safety.”
So these algorithm changes are typically implemented and validated through a very well defined and in some cases a fully automated process. And the purpose is to obviously aim at improving performance based on this analysis of the newer, additional data.
And that adaptation process that I’ve mentioned, there are essentially two steps there. Number one is the learning stage, and two is the updating stage. And the algorithm learns how to change this behavior from the addition of these new input types or by adding new cases to an already existing database. And then the second stage of the update occurs when the new version of the algorithm is deployed. So as a result, given the same set of inputs at a time A versus time B before and after the update, that algorithm may differ.
Erika Porcelli So we have talked about artificial intelligence and machine learning, and I think it would be really important for us to understand the difference between that and compared to software as a medical device. Can you expand on that?
Lisa Michels Sure. So the artificial intelligence and machine learning technologies, again, have the real time ability to adapt. They can optimize device performance based on this real world feedback. And the intent of course is to continually improve the performance and ultimately enhance the patient’s safety, the patient care.
So when artificial intelligence and machine learning based software is intended to treat, diagnose, cure, mitigate, or prevent diseases or conditions, then they’re essentially also medical devices, and they meet the definition of software as a medical device. And this is a term that comes both from the FDA perspective and the International Medical Device Regulators Forum.
“When artificial intelligence and machine learning based software is intended to treat, diagnose, cure, mitigate, or prevent diseases or conditions, then they’re essentially also medical devices.”
There’s a slight difference in terms of how it’s defined, but essentially they’re harmonized for the point of this discussion position paper. And the International Medical Device Regulators Forum actually set up this risk based framework for the software as a medical device, and it uses it as the foundation for determining the risk classification for a this type of software. And it goes from lowest to highest, class one, two, three and four.
Erika Porcelli So I think that’s an interesting point you raise. In your opinion, how does artificial intelligence and machine learning fit within the FDA’s current approach to pre-market review of new software and/or modifications to existing software?
Lisa Michels So currently, the artificial intelligence and machine learning technologies don’t fit into the current regulatory framework for traditional software. Whether that software is standalone software or medical devices that the software may accompany the medical device. The new software devices and modifications that may be made to traditional software that basically could affect the safety or effectiveness of the device, typically require submitting a new 510(k), or if it’s a PMA device, a PMA supplement. Anything that impacts the safety or effectiveness of the device will require that.
This new technology is very different of the fact that the software itself, the algorithms have the ability to adapt and change. It’s happening real time. So in terms of fitting into the current framework, it’s the open situation where the current FDA guidance in terms of assessing whether or not a 510(k) or a PMA supplement may be necessary for a modification to the software. It’s slightly different.
The FDA still recommends that their existing guidance, which is the guidance on deciding when to submit a 510(k) for a software change for an existing device. They also refer to it as the software modifications guidance document. But the artificial intelligence machine learning software modifications will actually require a very specifically tailored pre-market review and post-market monitoring process in order for this to work. And the current process doesn’t really afford that.
So that’s why the FDA is proposing this new total product life cycle approach so that they can ensure that effective safeguards are going to be established for this new novel technology.
“The FDA is proposing this new total product life cycle approach so that they can ensure that effective safeguards are going to be established for this new novel technology.”
Erika Porcelli So what are some of the types of artificial intelligence and machine learning based SAMD modifications? What should people be aware of?
Lisa Michels So this of course will be a specific to a manufacturer’s type of device, its indications for use. Every piece of software is different and how it’s used in device may be very different. So there’s not really a cookie cutter answer for the types of modifications that may be made to artificial intelligence or machine learning based software medical devices.
But in general, the FDA has categorized three different general types of changes that they envision. And of course, this position paper is put forth so that the FDA can garner additional feedback from industry as to whether there are any other types of modifications that are not mentioned or referenced that should be provided so that when the FDA eventually finalizes a draft version of some guidance document and provides more recommendations, they will feel like they’re covering all of the most critical types of modifications that they may be made.
But typically, the types of changes would either involve changes to clinical and analytical performance. So performance based changes. Or inputs that are used by the algorithm and their clinical association to obtain a particular output. Those may change or be altered. Or, if the intended use of the software as a medical device is changed, then obviously that would be considered more of a significant modification.
So all of these three different types are essentially just the very broad categories that would fit into the types of changes that would be somewhat commensurate with how we view traditional software, and how we look at modifications to traditional software that would potentially require either doing a new regulatory submission. Or in some cases, if it’s not affecting safety and effectiveness, changes that could be handled internally through what is known to industry as a letter to file.
Erika Porcelli So a few minutes ago, you had spoken about the total product life cycle approach. What are the primary elements of this approach?
Lisa Michels So this approach is intended to strike a balance between the iterative improvement capabilities of the AI/ML software to ensure patient safety. So this approach requires that the ongoing algorithm changes are implemented according to a pre-specified performance objective criteria. And they must comply with predefined algorithm change control protocols.
Of course, this process also requires a robust validation process to be implemented. And this is to improve performance, safety, and effectiveness of the artificial intelligence, machine learning software.
And lastly, the inclusion of real world monitoring. So at a really high level, the general principles behind this total product life cycle approach is there’s four key areas. And the first is the expectation that the FDA assumes as they do with any other medical device company, that you are meeting quality system requirements. So 21 CFR part 20. And they basically developed this new terminology called good machine learning practices. And this is still to be determined. It’s yet to be outlined as to what this constitutes. But that’s the whole point behind this position paper is to help develop some of the specific requirements for what constitutes good machine learning practices.
“The whole point behind this position paper is to help develop some of the specific requirements for what constitutes good machine learning practices.”
The second principle of this total product life cycle approach is to have a initial pre-market assurance of safety and effectiveness. So this would be commensurate with the current process where a manufacturer has to submit a 510(k) or some pre-market submission in order to be for commercialization. So it has to be cleared via 510(k) or a different regulatory pathway before the company can actually commercialize it. So in alignment with that methodology, this approach would also incorporate an initial pre-market review. Once it would be approved or cleared, again, depending on the regulatory pathway, then the modifications made to that initial clearance or approval for this type of software would then have to be assessed. And that’s how this approach is outlining the general themes on how modifications should be addressed and whether they constitute submitting a new information to the FDA.
So this approach, one way that the FDA is proposing is to have a proposed plan if you will, come from the manufacturer that once the device is once they cleared or approved initially through this pre-market assurance of safety and effectiveness performed by the FDA, what is the proposed plan for the manufacturer to have this established pre-specification process and an algorithm change protocol?
So the FDA is suggesting at this early stage that this would be an optional way for the manufacturer to be able to explain to the FDA and get their input guidance on whether or not that proposed process would be appropriate for their device in terms of how the FDA would basically use, what they would use as a baseline in terms of assessing modifications after it goes through this initial pre-market assurance of safety and effectiveness.
And then lastly, the FDA expects that the manufacturers of these type of devices will have total transparency. Not only to the FDA, but also to patients and users. And that they perform some type of real world performance monitoring of these types of devices. However, at this early stage, a lot of this is very tentative. And in order for this process to work effectively, it will take a lot of input from industry and experts to determine how this will actually work in the real world and what are the specific types of methodologies, or processes, or frameworks that the FDA would be willing to accept.
The problem with it is that a number of these devices are very different of course, as you can imagine, because they may be have a different indication for use. They’re used in different parts of the body. It’s different technologies. So there’s not just a one right answer for everyone to be following, because all these devices are different.
So this is all information that has to be worked out as the FDA moves forward with this initiative, to really come up with a fair and consistent way of applying these general principles across the board for these types of devices. And again, using this risk based framework for the devices based on the risk level that may be assigned.
Erika Porcelli What are the regulatory implications of using this approach?
Lisa Michels So again, based on the risk level of the device, how the device is classified, the current regulatory process requires a 510(k) or PMA De Novo in some cases to be submitted as the pre market filing before they can receive their clearance or approval and start to commercialize it. So once a software device such as the ones we’re talking about are cleared or approved, those modifications must also be assessed from a risk based perspective to determine whether a new filing is necessary.
So in terms of the regulatory implications, it’s not so easy as it may have been in the past. Not that the process is easy, but in terms of being able to just use consistent guidance in applying the defined methodology or flow charts that the FDA has put forth in some of its guidance, for assessing the types of changes that you can make a decision if it’s this type of change, 510(k) is going to be required or it may be a change where it can be documented internally and via the letter to file process, and a 510(k) would not be necessary.
So what the regulatory implications of this new process are going to be is that there’s going to have to be extremely heavy oversight by medical device manufacturers that produce these types of products to make sure that they are constantly assessing these types of changes, and being very transparent with the FDA and the patients or users. Of course medical device companies have to constantly assess their devices already, so that’s not necessarily a new hurdle. But because the process just doesn’t specifically fit into the current process, the implications are such that this could take some time to smooth out all of the rough edges for these medical device companies that produce these types of products.
“There’s going to have to be extremely heavy oversight by medical device manufacturers that produce these types of products to make sure that they are constantly assessing these types of changes, and being very transparent with the FDA and the patients or users.”
So it could in some sense put them in somewhat of a slight disadvantage as they develop some of these technologies because the process is not well defined at this point in order to accommodate all of these different devices that incorporate this artificial intelligence or machine learning.
So that is, I think, the biggest challenge for these types of devices. The good thing of course is that the FDA has recognized that devices are moving in this direction. Much of the digital health initiatives and mobile medical app devices and things of that nature as we continue to expand the technologies of these devices and how they’re used. It’s great that the FDA is focusing on this effort. But the reality is that it will take some time before a really defined useful process is going to be put in place for some of these manufacturers. And in the meantime, we have to rely on existing FDA guidance documents, risk based methodologies, software validation, methodologies and so forth that we currently use based on the risk level. But it will be very interesting to see how this evolves over time and how long it will take before we have a consistent and useful methodology for these novel devices.
Erika Porcelli Absolutely. I agree. Do you have a sense of what the FDA is expecting manufacturers to include for review and the initial pre-market assurance of safety and effectiveness?
Lisa Michels So that is still to be determined. The FDA is suggesting as part of the new framework that manufacturers would be given the option of submitting a plan for modifications during this initial pre-market review. And this predetermined change control plan would include very specific details on the types of anticipated modifications that the manufacturer may have to the software. And what the FDA is referring to for these control plans is referred to as a software as a medical device pre-specification or SPS. Again, the specific details of what that constitutes is not yet defined. But the fact that the FDA in this position paper has said that this would be an option is very interesting because if this is the direction that the guidance needs to go in terms of gathering input from industry as to what makes sense for these pre-specification documents, it shouldn’t be optional. It should be mandatory.
So that’s a very interesting point that I noted in the discussion paper, and we’ll see how that shakes out. But in the discussion paper, the FDA also talks about an algorithm change protocol that should be submitted in this initial pre-market review to assure safety and effectiveness. And this would essentially outline the manufacturer’s specific methods so they can ensure that the risks of their anticipated types of modifications are going to be controlled appropriately.
And in terms of the elements or at least suggested elements of what needs to be in an algorithm change control protocol, the discussion paper actually includes as one of the exhibits a very high level section on what they envision the elements to be. But again, this is not even a draft guidance at this point. So there’s nothing in this discussion paper that is binding. It’s just a reflection of what the FDA currently suggests. And I think that this will definitely evolve as the FDA receives feedback from industry and provides input to some of their questions, which in fact are asking industry, what do you think are the appropriate elements for these pre-specification documents in the algorithm change protocols? So it will be very interesting to see all of the feedback that is put forth from industry, and how the FDA takes that input and actually prepares guidance based on that feedback.
Erika Porcelli Yeah. It’ll be very interesting to see how things unfold over the next several months in this area.
Lisa Michels Definitely.
Erika Porcelli So what is the approach suggested for modifications after the initial review with an established software pre-specification and algorithm change protocol?
Lisa Michels So again, right now the FDA is … the current process is to assess changes that could impact the safety or effectiveness of the device. And this risk base approach is what manufacturers should use when performing a risk assessment so that they can properly evaluate whether these risks are appropriately mitigated. And again, this is going to be different across the board because based on the type of the modifications, it may require submitting a new submission, or the changes may be minimal and they don’t affect safety and effectiveness, and they can be handled internally through a letter of file.
So the approach right now is to follow the current practice of using the FDA guidance to assess changes, modifications based on this risk approach. And then the pre-specification and the algorithm change control documents that had been suggested to be included in this initial review. As those become more refined, we’ll have a better idea for how the FDA will focus its review on those changes. And essentially, they will have to match.
So again, the feedback is going to be critical coming from industry so that the FDA can prepare appropriate standards for industry to follow. So that this review process is going to be consistent across the board for manufacturers of devices that may be very different. Again, based on how the software is incorporated, how the device is used. The intended use of the device. And I think that’s a pretty big challenge.
“The feedback is going to be critical coming from industry so that the FDA can prepare appropriate standards for industry to follow. So that this review process is going to be consistent across the board for manufacturers of devices that may be very different.”
Erika Porcelli Yeah, I agree. I agree. Do you have a sense of what the FDA’s expectations for manufacturers to comply with transparency requirements and real world performance monitoring?
Lisa Michels So this is still yet to be determined. The FDA essentially talks about transparency, about the function of the device and the modifications of the medical device as a key aspect of their safety. So especially those that change over time. So these are some of the adaptive algorithms that we talked about earlier.
Again, this is preliminary, but what the FDA alludes to is the transparency could potentially consist of providing updates or reports to the FDA. And as the changes occur, obviously labeling changes may be necessary to describe those modifications and to be very transparent, not only with the FDA, but also with users or patients.
So again, this is a huge effort in order to keep transparency on the forefront. And what transparency actually consists of again could be very different. Without the FDA either mandating specific types of reporting or labeling requirements, it’s difficult to determine at this early stage what those will be. But the real world performance monitoring is really going to require the manufacturers to establish a very robust process for monitoring the real world performance of the devices, how the software is working. As it changes, if there are new performance standards that are met. How the manufacturers incorporate this to determine the enhancements to the performance of the device.
So again, this is a very preliminary, and the FDA is currently alluding to the involvement of manufacturers in either an annual reporting process or in some of the pilot programs that the FDA has initiated, just to ensure that this ongoing safety and effectiveness in the development of new devices of this type or modifications of this artificial intelligence and machine learning software.
So it’s again, very open in terms of how this will actually come to the FDA’s expectations. It’s still to be determined.
Erika Porcelli Well I think in closing, and I don’t think I mentioned this at the beginning, but we’re coming up on the deadline for submitting comments, which is June 3rd, 2019. I know that RCA will be submitting feedback and comments, and we would certainly encourage all of our listeners to do the same.
Lisa, I really want to thank you for taking the time to provide us with your insight. It has been invaluable and hopefully it has helped others understand the document in a little bit more detail. And to learn more about artificial intelligence and machine learning, please visit our website at www.rcainc.com. And thank you to our listeners for tuning into this episode of RCA Radio, and be sure to subscribe to be the first to know when we upload a new episode. And again Lisa, thank you so much for your time.
Lisa Michels My pleasure, Erika. Thank you.