AI in the Eyes of Pragmatic Reviewers
Last time we discussed the definition of clinical evaluation. NMPA's definition focuses on the confirmation of the finished medical device, reflecting the pragmatic review approach from a regulatory perspective. So, what is Artificial Intelligence in the eyes of pragmatic reviewers?
In the NMPA context, "AI" usually refers to the use of machine learning algorithms to process medical data to achieve the intended use of a medical device. We often hear terms like Artificial Intelligence, Machine Learning (ML), Traditional Machine Learning, Deep Learning (DL), Artificial Neural Networks (ANN), Big Data & Data Mining. Sometimes it's hard to distinguish them. Below is a brief clarification of the relationships between these common terms in the context of medical devices.
Two common concepts to clarify:
Artificial Neural Networks (ANN) are mathematical models that mimic the structure and function of biological neural networks (especially the brain), composed of numerous artificial neurons connected together. They can change their internal structure based on external information—in layman's terms, they have learning capabilities. Single-layer/shallow neural networks belong to traditional ML, while neural networks implementing deep learning consist of multiple layers (hidden layers ≥ 3).
Big Data & Data Mining are not algorithms or mathematical models. Big Data refers to massive, multi-source, heterogeneous data that is difficult to process with traditional data processing software. Big Data can be both the material for training AI/ML and the object processed by AI/ML. Data Mining refers to the process of discovering patterns/associations in data (which can be implemented using ML algorithms) to acquire knowledge; it is not a technology that replaces ML.
In the professional field of medical devices, AI mainly assists doctors in reading scans, interpreting ECG/EEG, triaging, and providing diagnostic ideas. Based on the chronological emergence of AI technologies, we can roughly divide them into the following schools:
These technologies are mentioned less now, but they haven't disappeared; they've just become "basic skills."
How do they learn? Similar to a "master teaching an apprentice." Medical experts first tell the AI: "Look, this nodule is 5cm in diameter with spiculated margins, it looks bad..." The expert feeds massive cases and diagnostic results to the AI, and the AI learns to score the risk.
Representatives: Support Vector Machine (SVM), Random Forest, Logistic Regression.
What do they do in devices? Early premature heartbeat detection, simple risk scorecards. Advantages: simple, obedient, not prone to "nonsense." Disadvantages: relies on experts manually extracting features; cannot see through complex information on their own.
The main force of "AI" in current devices, characterized by "brute force."
How do they learn? No longer requires experts to tell the AI what features to analyze. Massive annotated imaging data is directly fed to the AI. The AI silently builds thousands of "neurons," extracting features layer by layer automatically—the first layer looks at lines, the second at shapes, the third at textures... After training, running the data through the network detects abnormalities.
Representatives: CNN (Convolutional Neural Network).
What do they do in devices? Pulmonary nodule detection, diabetic retinopathy screening, MRI image reconstruction. Due to extreme complexity, humans find it hard to understand the constructed network, often called a "black box."
As the synonym for large models, GPT is essentially still deep learning, but it reads text instead of images.
How do they learn? It read massive articles and books across the internet and mastered the ultimate skill of "picking up the conversation" (predicting the next word). After "special training" (fine-tuning) with medical literature, it becomes a medical-savvy AI.
What do they do in devices? Basic application (Secretary version): turning messy clinic recordings into neat medical records. Advanced application (Expert version, regulatory red line): analyzing symptoms, reading test results, and even giving diagnostic suggestions. GPT has a bad habit called "Hallucination" (making things up), potentially inventing non-existent medical guidelines. Therefore, using GPT for auxiliary diagnosis requires very thorough clinical evaluation.
AI is a big basket; any software or hardware simulating human intelligence can be thrown into it. In the medical device field, AI currently specifically refers to Machine Learning. SVM is the traditional routine, relying on humans to teach features. Deep Learning is high-end martial arts, summarizing formulas from standard answers, suitable for reading images and waveforms. LLM/GPT is the latest gameplay of deep learning, dealing specifically with text—smart but prone to making things up, carrying extremely high risk when used for diagnosis.
For clinical evaluation, the more "black box" (deep learning) or "prone to making things up" (GPT) a technology is, and the more it directly provides diagnostic suggestions (auxiliary decision-making), the more sufficient the clinical evidence must be prepared. Not only should multi-center, multi-reader clinical trials be considered, but various application scenarios must be fully considered during AI training with appropriate samples fed to it. Only then can AI be reliable in all situations.
Author: Dr. Zhang Jiange (ORCID: https://orcid.org/0000-0003-2182-8686)
Hot Line
(+86)010-82667388
1508, Building A of TYG CENTER No C2 East Third Ring North Road, Chaoyang District, Beijing,China
Follow our WeChat to stay up-to-date
on NMPA regulation
Follow our WeChat
Copyright@2021 Shangpin All Rights Reserved
Beijing Regis Success Medical Tech. Co., Ltd. Technical support：ShangPin
