Machine/Deep Learning

Semantic Segmentation Solution for SCD Diagnosis

Red blood cell (RBC) segmentation and classification from biomedical images is a crucial step for the diagnosis of sickle cell disease (SCD). We are developing and deploying a semantic segmentation framework to simultaneously detect and classify RBCs from raw microscopic images. The deformable U-Net (dU-net) in our proposed solution utilizes an extra deformable layer to make the prediction model more robust towards variations in the size, shape and viewpoint of the cells. Testing on preliminary data consisting of 314 images from 5 different SCD patients shows that dU-net framework achieves best segmentation/classification accuracy within an integrated workflow, outperforming both traditional unsupervised methods and classical U-Net structure. ​​​

Architecture of the dU-Net in this work.

Binary segmentation results by different methods: (a) raw image, (b) Ilastik, (c) region growing, (d) U-Net, (e) dU-Net, (f) ground truth.

Attenuation Correction for Brain PET Imaging Using Deep Neural Network Based on Dixon and ZTE MR Images

Positron Emission Tomography (PET) is a functional imaging modality widely used in neuroscience studies. To obtain meaningful quantitative results from PET images, attenuation correction is necessary during image reconstruction. For PET/MR hybrid systems, PET attenuation is challenging as Magnetic Resonance (MR) images do not reflect attenuation coefficients directly. To address this issue, we present deep neural network methods to derive the continuous attenuation coefficients for brain PET imaging from MR images. With only Dixon MR images as the network input, the existing U-net structure was adopted and analysis using forty patient data sets shows it is superior than other Dixon based methods. When both Dixon and zero echo time (ZTE) images are available, we have proposed a modified U-net structure, named GroupU-net, to efficiently make use of both Dixon and ZTE information through group convolution modules when the network goes deeper. Quantitative analysis based on fourteen real patient data sets demonstrates that both network approaches can perform better than the standard methods, and the proposed network structure can further reduce the PET quantification error compared to the U-net structure.

PET reconstruction error images (unit: SUV) using the Dixon-Atlas method (first column), the ZTE-Seg method (second column),the DixonZTE-Unet method (third column) and the DixonZTE-GroupUnet method (last column)

The bar plot of the mean relative PET error for the patient data sets with both Dixon and ZTE images. Standard deviations of the absolute error for all the patients are plotted as the error bar.

Deep Learning -Enabled System for Rapid Pneumothorax Screening on Chest CT

We are building a deep learning-enabled system for detecting the presence of pneumothorax (i.e. free air) on chest CT images. The pilot system has been tested in a pseudo-online manner with perfect sensitivity. The system will be further integrated into the clinical workflow at the point of completion of each chest CT scan, making it useful for patients with pre-existing chest diseases that place them at higher risk for pneumothorax, and for patients presenting in the emergency department for rapid triage.​​​

Illustration of the need for a rapid screening system to re-prioritizing the work list for hospital management.​​

Visualization of pneumothorax detection results (all positive) from nine patients, shown in axial view.

Self-paced learning method to overcome the lack of samples for supervised machine learning

•As there is ever-increasing need for annotated training samples to feed the learning-based models (such as deep convolutional networks), we developed a multi-stage self-paced learning framework to increase the samples by tens of folds, based on the refinement of the unlabeled samples. Those increased amount of training samples, which will be very expensive both in cost and labor work, can lead to more accurate and robust AI programs.

Iterative Low-dose CT Reconstruction with Priors Trained by Artificial Neural Network

Dose reduction in computed tomography (CT) is essential for decreasing radiation risk in clinical applications. Iterative reconstruction algorithms are one of the most promising way to compensate for the increased noise due to reduction of photon flux. Most iterative reconstruction algorithms incorporate manually designed prior functions of the reconstructed image to suppress noises while maintaining structures of the image. These priors basically relied on smoothness constraints cannot exploit more complex features of the image. The recent development of artificial neural networks and machine learning enabled learning of more complex features of image, which has the potential to improve reconstruction quality. In this work, K-sparse autoencoder (KSAE) was used for unsupervised feature learning. A manifold was learned from normal-dose images and the distance between the reconstructed image and the manifold was minimized along with data fidelity during reconstruction. Experiments on 2016 Low-dose CT Grand Challenge was used for the method verification, and results demonstrated the noise reduction and detail preservation abilities of the proposed method.

End-to-end Lung Nodule Detection in Computed Tomography​​

•As there is ever-increasing need for annotated training samples to feed the learning-based models (such as deep convolutional networks), we developed a multi-stage self-paced learning framework to increase the samples by tens of folds, based on the refinement of the unlabeled samples. Those increased amount of training samples, which will be very expensive both in cost and labor work, can lead to more accurate and robust AI programs.

Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes

We are exploring the how the multi-modality medical imaging can help advancing the accuracy and robustness of computer aided detection. To this purpose, we built multi-modal Convolutional Neural Networks (CNN) that performs fusion across CT, MR and PET images at various stages. For the task of detecting and segmenting the soft tissue sarcoma, multi-modal deep learning system shows much superior performance than single-modal systems, even using images of lowered quality. The capability of maintaining high segmentation accuracy on low-dose images with added modality of the proposed system provides a new perspective in medical image acquisition and analysis.

Illustration of the structure for (a) Type-I fusion networks, (b) Type-II fusion network and (c) Type-III fusion network. The yellow arrows indicate the fusion location.

(a) Ground truth shown as yellow contour line overlaid on the T2 image. (b) Result from Type-II fusion network based on PET+CT+T1. (c) Result from single-modality network based on T2. (d-f) Results from single-modality network based on PET, CT and T1, respectively.