Predicting breast cancer types on and beyond molecular level in a multi-modal fashion
NPJ breast cancer
2023
16
Tianyu Zhang, Tao Tan, Luyi Han, Linda Appelman, Jeroen Veltman, Ronni Wessels, Katya M Duvivier, Claudette Loo, Yuan Gao, Xin Wang, Hugo M Horlings, Regina GH Beets-Tan, Ritse M Mann
Abstract
Accurately determining the molecular subtypes of breast cancer is important for the prognosis of breast cancer patients and can guide treatment selection. In this study, we develop a deep learning-based model for predicting the molecular subtypes of breast cancer directly from the diagnostic mammography and ultrasound images. Multi-modal deep learning with intra- and inter-modality attention modules (MDL-IIA) is proposed to extract important relations between mammography and ultrasound for this task.