Deep Learning-Based Multimodal Brain Tumor Analysis: A Comprehensive Review of Classification, Segmentation, Detection, and Prognosis

Abstract

Brain tumors are still one of the most difficult diseases in neuro-oncology due to their diverse morphology, their varying grades, and the lack of clarity in demarcating tumor and normal tissues. Structural and functional information for diagnosis, treatment planning, and prognosis is provided by magnetic resonance imaging (MRI), which may be combined with computed tomography (CT), positron emission tomography (PET), and single-photon emission computed tomography (SPECT). Over the past decade, deep learning has revolutionized brain tumour analysis by shifting the paradigm from handcrafted features and manual delineation to automated convolutional neural networks (CNNs), Vision Transformers (ViTs), and hybrid architectures that learn directly from imaging data. This paper summarizes recent advances across four related challenges: tumour classification, tumour segmentation, disease forecasting, and object detection. The literature is organized by methodological family, covering transfer learning, attention mechanisms, capsule networks, graph networks, transformer-based encoders, generative and semi-supervised learning methods, and federated and privacy-preserving approaches. We also summaries the data types, imaging modalities, and evaluation metrics commonly used to benchmark these systems, and compare reported performance trends from 2019 to 2025. Lastly, we consider persistent clinical challenges such as domain shift between institutions, missing-modality robustness, limited interpretability, and regulatory readiness, as well as emerging trends like foundation models, self-supervised pretraining, and federated diffusion-based data synthesis, that will impact the future of clinically deployed systems.