Skip to content

2Be3-Net : Combining 2D and 3D convolutional neural networks for 3D PET scans predictions

Radiomics – high-dimensional features extracted from clinical images – is the main approach used to develop predictive models based on 3D Positron Emission Tomography (PET) scans of patients suffering from cancer. Radiomics extraction relies on an accurate segmentation of the tumoral region, which is a time-consuming task subject to inter-observer variability. On the other hand, data-driven approaches such as deep convolutional neural networks (CNN) struggle to achieve great performances on PET images due to the absence of available large PET datasets combined to the size of 3D networks. In this paper, we assemble several public datasets to create a PET dataset large of 2800 scans and propose a deep learning architecture named “2Be3-Net” associating a 2D feature extractor to a 3D CNN predictor. First, we take advantage of a 2D pre-trained model to extract feature maps out of 2D PET slices. Then we apply a 3D CNN on top of the concatenation of the previously extracted feature maps to compute patient-wise predictions. Experiments suggest that 2Be3-Net has an improved ability to exploit spatial information compared to 2D or 3D only CNN solutions. We also evaluate our network on the prediction of clinical outcomes of head-and-neck cancer. The proposed pipeline outperforms PET radiomics approaches on the prediction of loco-regional recurrences and overall survival. Innovative deep learning architectures combining a pre-trained network with a 3D CNN could therefore be a great alternative to traditional CNN and radiomics approaches while empowering small and medium-sized datasets. 

Ronan Thomas, Elsa Schalck, Damien Fourure, Antoine Bonnefoy and Inaki Cervera-Marzal, 2Be3-Net : Combining 2D and 3D convolutional neural networks for 3D PET scans predictions, Proc. of the 2nd International Conference on Medical Imaging and Computer-Aided Diagnosis, 2021.

Watch Ronan present the paper on YouTube.

Click here to access the paper in its preprint form.

Share on linkedin
Share on twitter
Share on email

Releated Posts

INTERNSHIPS 2022

This document presents internships supervised by our software engineering department or by our research & development department. Each project is an opportunity to feel both empowered and responsible for your own professional development and for your contribution to the company.
Read More

Multimodal Classifier For Space Target Recognition

We propose a multi-modal framework to tackle the SPARK Challenge by classifying satellites using RGB and depth images. Our framework is mainly based on Auto-Encoders to embed the two modalities in a common latent space in order to exploit redundant and complementary information between the two types of data.
Read More