Research
Multi-sequence MRI synthesis
Project 1. IMPORTANT-Net
Background
Magnetic resonance imaging (MRI) is highly sensitive for lesion detection. Sequences obtained with different settings can capture specific characteristics of lesions. Such multi-parametric MRI information has been shown to aid radiologist performance in lesion classification, as well as improving the performance of artificial intelligence models in various tasks. However, obtaining multi-parametric MRI makes the examination costly from both financial and time perspectives, and there may also be safety concerns for special populations, thus making acquisition of the full spectrum of MRI sequences less durable.
Methods
In this study, a sophisticated Integrated MRI Multi-Parametric increment fusiOn generatoR wiTh AtteNTion Network (IMPORTANT-Net) is developed to generate absent sequences/parameters. First, the parameter reconstruction module is used to encode and restore the existing MRI parameters to obtain the corresponding latent representation information at any scale level. Then the multi-parametric fusion with attention module enables the interaction of the encoded information from different parameters through a set of algorithmic strategies, and applies different weights to the information through the attention mechanism after information fusion to obtain refined representation information. Finally, a increment fusion scheme embedded in a V-shape generation module is used to combine the hierarchical representations to generate specified absent MRI parameter.
Results
Results showed that our IMPORTANT-Net is capable of synthesizing absent MRI, outperforms comparable state-of-the-art networks and more importantly benefit downstream tasks.
Project 2. Multi-sequence MRI synthesis
Background
Magnetic resonance imaging (MRI) is a robust cancer detection, characterization, and diagnosis tool due to its high sensitivity. Different MRI sequences and parameters reveal specific features of lesions. In breast imaging, for example, T1-weighted imaging before and after contrast, T2-weighted imaging with and without fat suppression, and diffusion-weighted imaging (DWI) offer insights into enhancement abnormalities, cyst visualization, and tissue microstructure.
Multiple studies have demonstrated that utilizing multi-sequence MRI enhances the diagnostic accuracy of radiologists and artificial intelligence models in classifying and segmenting lesions. However, obtaining multi-sequence MRI can be costly and time-consuming, especially when acquiring sequences such as DCE that require contrast agents, which can increase toxicity. Deep learning models can synthesize some sequences, effectively addressing these issues and streamlining breast cancer screening protocols.
Methods
We propose image-to-image models to achieve multi-sequence MRI synthesis, where the input can be either a single MRI sequence or multiple MRI sequences. The model can convert or fuse source sequences into a target sequence.
Results
Our deep learning algorithms are capable of analyzing multi-sequence MRI data, determining the rank of contributions for each sequence, and extracting specific regions within each sequence. Additionally, the algorithms can synthesize MRI sequences, such as DCE and T2, which helps to reduce toxicity and scanning costs.
