"Practice artificial intelligence" -- solve the dilemma of small sample size and realize the non-invasive evaluation of TP53 in pancreatic cancer

Recently, Xiaohua Qian 's team from School of Biomedical Engineering, Shanghai Jiaotong University creatively proposed using “Spiral transformation” to solve the small sample size challenge of artificial intelligence in tumor imaging. This technique has successfully solved the problem of multimodal MRI in predicting TP53 status of pancreatic cancer, and has also been verified in predicting HPV status of cervical cancer based on CT. With the title of "Combined Spiral Transformation and Model Driven Multi-modal Deep Learning Scheme for Automatic Prediction of TP53 Mutation in Pancreatic Cancer", the findings were published online in IEEE Transactions on medical imaging (TMI, a top imaging journal) on November 4, 2020. The first author of this paper is a sophomore Xiahan Chen, Xiaohua Qian as the corresponding author.

Pancreatic cancer is a malignant form of cancer with one of the worst prognoses. The poor prognosis and resistance to therapeutic modalities have been linked to TP53 mutation. Pathological examinations, such as biopsies, cannot be frequently performed in clinical practice; therefore, noninvasive and reproducible methods are desired. However, automatic prediction methods based on imaging have drawbacks such as poor 3D information utilization, small sample size, and ineffectiveness multimodal fusion. In this study, we proposed a model-driven multi-modal deep learning scheme to overcome these challenges. A spiral transformation algorithm was developed to obtain 2D images from 3D data, with the transformed image inheriting and retaining the spatial correlation of the original texture and edge information. The spiral transformation could be used to effectively apply the 3D information with less computational resources and conveniently augment the data size with high quality. Moreover, model-driven items were designed to introduce prior knowledge in the deep learning framework for multi-modal fusion. The model-driven strategy and spiral transformation-based data augmentation can improve the performance of the small sample size. A bilinear pooling module was introduced to improve the performance of fine-grained prediction. The experimental results show that the proposed model gives the desired performance in predicting TP53 mutation in pancreatic cancer, providing a new approach for noninvasive gene prediction. The proposed methodologies of spiral transformation and model-driven deep learning can also be used for the artificial intelligence community dealing with oncological applications.

Fig. 1. The framework of the proposed model. The pipeline includes modules for spiral transformation, feature extraction, feature fusion, and output. Three parts, mutation prediction loss, intra-modal feature selection loss, and inter-modal prediction constraint loss are combined to supervise training process.

Full text link:

https://ieeexplore.ieee.org/document/9248055

Neural Network Method for Automatic Quantitative Assessment of Motor Ability of Parkinson's Disease Based on Smartphone Videos

Motor disorder is a typical symptom of Parkinson’s disease (PD). Neurologists assess the severity of PD motor symptoms using the clinical rating scale, i.e., MDS-UPDRS. However, this assessment method is time-consuming and easily affected by the perception difference of assessors. In the recent outbreak of coronavirus disease 2019, telemedicine for PD has become extremely urgent for clinical practice.

Therefore, Xiaohua Qian's team from the School of Biomedical Engineering of Shanghai Jiaotong University cooperated with Dr. Chencheng Zhang from Bomin Sun's team, director of functional neurosurgery of Ruijin Hospital Affiliated to Medical College of Shanghai Jiaotong University, On November 19, 2020, and published a research article online entitled "Sparse Adaptive Graph Revolutionary Network for Leg Agility Assessment in Parkinson's Disease" in IEEE Transactions on Neural Systems and Rehabilitation Engineering (TNSRE), the top journal of rehabilitation. This paper proposes a deep learning model which is suitable for automatic and objective evaluation of leg flexibility tasks in MDS-UPRDS based on video (smartphone video). The first author of this paper is Rui Guo, a sophomore doctoral student, and the corresponding author is Xiaohua Qian.

Leg agility (LA) assessment is one of the vital parts of the MDS-UPDRS. In this assessment, the neurologist requires the patient to raise each leg from the ground with the maximum amplitude and speed and then scores them 0–4 according to the patient’s movement speed, amplitude, hesitation and pause, gradual reduction of amplitude, and other factors. However, there are many limitations to the clinical practice of the MDS-UPDRS. First, although the action assessment in the MDS-UPDRS is direct and detailed, the quantitative assessment still depends on the clinical experience of neurologists, so subjectivity and inter-rater variability inevitably exist. In addition, a complete assessment usually takes more than half an hour to finish. This time-consuming assessment imposes a substantial burden on clinicians as the number of PD patients increases. Second, the scarcity of neurologists in the movement disorder and neurodegenerative disorder fields make it difficult for performing regular follow-ups to monitor PD patients’ disease status.

To solve these problems, the paper developed an automated and objective assessment method of the leg agility task in the MDS-UPDRS using videos and a graph neural network. In this study, a sparse adaptive graph convolutional network (SA-GCN) was proposed to achieve fine-grained quantitative assessment of skeleton sequences extracted from videos. Specifically, the sparse adaptive graph convolutional unit with a prior knowledge constraint was proposed to perform adaptive spatial modeling of physical and logical dependency for skeleton sequences, thus achieving the sparse modeling of the discriminative spatial relationships. Subsequently, a temporal context module was introduced to construct the remote context dependency in the temporal dimension, hence determining the global changes of the task. A multi-domain attention learning module was also developed to integrate the static spatial features and dynamic temporal features, and then to emphasize the salient feature selection in the channel domain, thereby capturing the multi-domain fine-grained information.

Fig. 2. Architecture of the proposed SA-GCN.

Finally, the evaluation results using a dataset with 148 patients and 870 samples confirmed the effectiveness and reliability of our scheme, and the method outperformed other related state-of-the-art methods. Our contactless method provides a new potential tool for automated PD assessment and telemedicine.

Full text link:

https://ieeexplore.ieee.org/document/9264203

Introduction to Xiaohua Qian’s team

Associate professor and doctoral supervisor from School of Biomedical Engineering, Shanghai Jiaotong University

Research interests:

1) Image processing and machine learning (deep learning) algorithm development

2) Medical image (video) processing and analysis, and health big data mining and analysis