arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Baris Turkbey

36 papers · 2016–2026

2016: 12017: 22018: 12019: 12020: 32021: 32022: 22023: 32024: 92025: 72026: 4
2016peak 9 in 20242026

Frequent co-authors

Papers

showing 1–25 of 36 RSS

Deeply-Supervised CNN for Prostate Segmentation

Qikui Zhu, Bo Du, Baris Turkbey +2 more

cs.CV · · cited 219× · arXiv ↗ · more like this ·

Prostate segmentation from Magnetic Resonance (MR) images plays an important role in image guided interven- tion. However, the lack of clear boundary specifically at the apex and base, and huge variation of shape and texture between the images from different patients make the task very challenging. To overcome these problems, in this paper, we propose a deeply supervised convolutional neural network (CNN) utilizing the convolutional information to accurately segment the prostate from MR images. The proposed model can effectively detect the prostate region with additional deeply supervised layers compared with other approaches. Since some information will be abandoned after convolution, it is necessary to pass the features extracted from early stages to later stages. The experimental results show that significant segmentation accuracy improvement has been achieved by our proposed method…

VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging

Yufan He, Pengfei Guo, Yucheng Tang +11 more

cs.CV · · cited 103× · arXiv ↗ · more like this ·

Foundation models for interactive segmentation in 2D natural images and videos have sparked significant interest in building 3D foundation models for medical imaging. However, the domain gaps and clinical use cases for 3D medical imaging require a dedicated model that diverges from existing 2D solutions. Specifically, such foundation models should support a full workflow that can actually reduce human effort. Treating 3D medical images as sequences of 2D slices and reusing interactive 2D foundation models seems straightforward, but 2D annotation is too time-consuming for 3D tasks. Moreover, for large cohort analysis, it's the highly accurate automatic segmentation models that reduce the most human effort. However, these models lack support for interactive corrections and lack zero-shot ability for novel structures, which is a key feature of "foundation". While reusing pre-trained 2D…

Multi-Domain Image Completion for Random Missing Input Data

Liyue Shen, Wentao Zhu, Xiaosong Wang +9 more

cs.CV · · cited 95× · arXiv ↗ · more like this ·

Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e.g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI). However, due to possible data corruption and different imaging protocols, the availability of images for each domain could vary amongst multiple data sources in practice, which makes it challenging to build a universal model with a varied set of input data. To tackle this problem, we propose a general approach to complete the random missing domain(s) data in real applications. Specifically, we develop a novel multi-domain image completion method that utilizes a generative adversarial network (GAN) with a representational disentanglement scheme to extract shared skeleton encoding and separate flesh encoding across multiple domains. We further illustrate that the learned…

VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge

Vishwesh Nath, Wenqi Li, Dong Yang +22 more

cs.CV · · cited 80× · arXiv ↗ · more like this ·

Generalist vision language models (VLMs) have made significant strides in computer vision, but they fall short in specialized fields like healthcare, where expert knowledge is essential. In traditional computer vision tasks, creative or approximate answers may be acceptable, but in healthcare, precision is paramount.Current large multimodal models like Gemini and GPT-4o are insufficient for medical tasks due to their reliance on memorized internet knowledge rather than the nuanced expertise required in healthcare. VLMs are usually trained in three stages: vision pre-training, vision-language pre-training, and instruction fine-tuning (IFT). IFT has been typically applied using a mixture of generic and healthcare data. In contrast, we propose that for medical VLMs, a fourth stage of specialized IFT is necessary, which focuses on medical data and includes information from domain expert…

Convolutional neural network based deep-learning architecture for prostate cancer detection on multiparametric magnetic resonance images

Yohannes Tsehay, Nathan Lay, Holger R. Roth +6 more

Artificial Intelligence · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · · cited 68× · DOI ↗ · more like this ·

Prostate cancer (PCa) is the second most common cause of cancer related deaths in men. Multiparametric MRI (mpMRI) is the most accurate imaging method for PCa detection; however, it requires the expertise of experienced radiologists leading to inconsistency across readers of varying experience. To increase inter-reader agreement and sensitivity, we developed a computer-aided detection (CAD) system that can automatically detect lesions on mpMRI that readers can use as a reference. We investigated a convolutional neural network based deep-learing (DCNN) architecture to find an improved solution for PCa detection on mpMRI. We adopted a network architecture from a state-of-the-art edge detector that takes an image as an input and produces an image probability map. Two-fold cross validation along with a receiver operating characteristic (ROC) analysis and free-response ROC (FROC) were used…

Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentation

Yingda Xia, Dong Yang, Wenqi Li +15 more

eess.IV · · cited 60× · arXiv ↗ · more like this ·

Federated learning (FL) enables collaborative model training while preserving each participant's privacy, which is particularly beneficial to the medical field. FedAvg is a standard algorithm that uses fixed weights, often originating from the dataset sizes at each client, to aggregate the distributed learned models on a server during the FL process. However, non-identical data distribution across clients, known as the non-i.i.d problem in FL, could make this assumption for setting fixed aggregation weights sub-optimal. In this work, we design a new data-driven approach, namely Auto-FedAvg, where aggregation weights are dynamically adjusted, depending on data distributions across data silos and the current training progress of the models. We disentangle the parameter set into two parts, local model parameters and global aggregation parameters, and update them iteratively with a…

When Unseen Domain Generalization is Unnecessary? Rethinking Data Augmentation

Ling Zhang, Xiaosong Wang, Dong Yang +7 more

cs.CV · · cited 49× · arXiv ↗ · more like this ·

Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, in clinically realistic environments, such methods have marginal performance due to differences in image domains, including different imaging protocols, device vendors and patient populations. Here we consider the problem of domain generalization, when a model is trained once, and its performance generalizes to unseen domains. Intuitively, within a specific medical imaging modality the domain differences are smaller relative to natural images domain variability. We rethink data augmentation for medical 3D images and propose a deep stacked transformations (DST) approach for domain generalization. Specifically, a series of n stacked transformations are applied to each image in each mini-batch during network training to account for the contribution of domain-specific shifts in…

Gaze2Segment: A Pilot Study for Integrating Eye-Tracking Technology into Medical Image Segmentation

Naji Khosravan, Haydar Celik, Baris Turkbey +9 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

This study introduced a novel system, called Gaze2Segment, integrating biological and computer vision techniques to support radiologists' reading experience with an automatic image segmentation task. During diagnostic assessment of lung CT scans, the radiologists' gaze information were used to create a visual attention map. This map was then combined with a computer-derived saliency map, extracted from the gray-scale CT images. The visual attention map was used as an input for indicating roughly the location of a object of interest. With computer-derived saliency information, on the other hand, we aimed at finding foreground and background cues for the object of interest. At the final step, these cues were used to initiate a seed-based delineation process. Segmentation accuracy of the proposed Gaze2Segment was found to be 86% with dice similarity coefficient and 1.45 mm with Hausdorff…

Cross-modal Attention for MRI and Ultrasound Volume Registration

Xinrui Song, Hengtao Guo, Xuanang Xu +6 more

cs.CV · · cited 6× · arXiv ↗ · more like this ·

Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registration. Our proposed cross-modal attention block effectively maps each of the features in one volume to all features in the corresponding volume. Our experimental results demonstrate that a CNN network designed with the cross-modal attention block embedded outperforms an advanced CNN network 10 times…

Domain Generalization with Adversarial Intensity Attack for Medical Image Segmentation

Zheyuan Zhang, Bin Wang, Lanhong Yao +5 more

eess.IV · · cited 2× · arXiv ↗ · more like this ·

Most statistical learning algorithms rely on an over-simplified assumption, that is, the train and test data are independent and identically distributed. In real-world scenarios, however, it is common for models to encounter data from new and different domains to which they were not exposed to during training. This is often the case in medical imaging applications due to differences in acquisition devices, imaging protocols, and patient characteristics. To address this problem, domain generalization (DG) is a promising direction as it enables models to handle data from previously unseen domains by learning domain-invariant features robust to variations across different domains. To this end, we introduce a novel DG method called Adversarial Intensity Attack (AdverIN), which leverages adversarial training to generate training data with an infinite number of styles and increase data…

NV-Generate-Body: 3D Latent Diffusion Model forControllable Anatomy, Tumor, and Demographic Body Mask Synthesis

Can Zhao, Pengfei Guo, Dong Yang +8 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Curating large labeled CT datasets is expensive and constrained by patient privacy. We present NV-Generate-BodyMask, a 3D latent diffusion model that synthesizes whole-body CT segmentation masks with precise, joint control over organ anatomy, tumor burden, and patient demographics. The model operates in the compressed latent space of a frozen mask autoencoder, generating $256^3$ volumetric label maps conditioned on a 19-dimensional vector via Adaptive Group Normalization and cross-attention. A size-adaptive ROI loss with spatial halo weighting and a paired tumor injection pipeline enable accurate tumor size conditioning despite the scarcity of annotated tumor cases. Trained on 8,571 masks across 26 cohorts spanning 132 labels, the model achieves Spearman $ρ= 0.70$--$0.92$ for organ size, 85--100\% tumor recall for lung, pancreatic, colon, and bone lesions, and strong demographic…

NV-Reason-CT: 3D Visual Language Model for CT Analysis

Andriy Myronenko, Dong Yang, Yucheng Tang +13 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

We present NV-Reason-CT, a generative vision--language model for chest and abdominal CT combining native 3D visual encoding with radiologist-guided reasoning. The model couples a native 3D vision transformer with a language model, passing all visual tokens and their explicit 3D coordinates into language decoding without further spatial token merging. This retains volumetric spatial information within the vision encoder and through the language model's positional encoding during joint processing with text. We train on a curated corpus of approximately 550,000 multimodal instruction examples from 70,111 unique CT image inputs, combining standardized reports, abnormality-focused and anatomy-specific questions, multi-turn interactions, and radiologist-authored reasoning from recorded and transcribed expert CT interpretations. Expert annotations provide direct supervision and guide…

Inter-reader variability for quality assessment of synthetically degraded prostate MR images aligned with PIQUAL version 2

Kang‐Lung Lee, Jobie Budd, Shu-Huei Shen +9 more

Computer Vision and Pattern Recognition · Diagnostic and Interventional Imaging · · cited 0× · DOI ↗ · more like this ·

PURPOSE: The purpose of this study was to evaluate expert radiologists' perceptions of the quality of magnetic resonance imaging (MRI) of the prostate for images that underwent controlled degradation. MATERIALS AND METHODS: Seven-levels of controlled degradations were applied to 10 single-slice image sets, with T2-weighted images degraded using Gaussian noise and blur, and diffusion-weighted images degraded using noise, blur, and elastic deformation. Twelve image sets of varying apparent-diffusion coefficient (ADC)-T2-weighted anatomical mismatch were also included. Ten expert radiologists independently assessed image quality using prostate imaging quality (PI-QUAL) aligned criteria. Inter-reader agreement was evaluated using proportion agreement and Gwet's agreement coefficient 1 (AC1). RESULTS: For T2-weighted images, image quality adequacy declined with increasing noise and blur…

MAISI-v2: Accelerated 3D High-Resolution Medical Image Synthesis with Rectified Flow and Region-specific Contrastive Loss

Can Zhao, Pengfei Guo, Dong Yang +7 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Medical image synthesis is an important topic for both clinical and research applications. Recently, diffusion models have become a leading approach in this area. Despite their strengths, many existing methods struggle with (1) limited generalizability that only work for specific body regions or voxel spacings, (2) slow inference, which is a common issue for diffusion models, and (3) weak alignment with input conditions, which is a critical issue for medical imaging. MAISI, a previously proposed framework, addresses generalizability issues but still suffers from slow inference and limited condition consistency. In this work, we present MAISI-v2, the first accelerated 3D medical image synthesis framework that integrates rectified flow to enable fast and high quality generation. To further enhance condition fidelity, we introduce a novel region-specific contrastive loss to enhance the…

Text2CT: Towards 3D CT Volume Generation from Free-text Descriptions Using Diffusion Model

Pengfei Guo, Can Zhao, Dong Yang +9 more

eess.IV · · cited 0× · arXiv ↗ · more like this ·

Generating 3D CT volumes from descriptive free-text inputs presents a transformative opportunity in diagnostics and research. In this paper, we introduce Text2CT, a novel approach for synthesizing 3D CT volumes from textual descriptions using the diffusion model. Unlike previous methods that rely on fixed-format text input, Text2CT employs a novel prompt formulation that enables generation from diverse, free-text descriptions. The proposed framework encodes medical text into latent representations and decodes them into high-resolution 3D CT scans, effectively bridging the gap between semantic text inputs and detailed volumetric representations in a unified 3D framework. Our method demonstrates superior performance in preserving anatomical fidelity and capturing intricate structures as described in the input text. Extensive evaluations show that our approach achieves state-of-the-art…

Comparing and Combining Artificial Intelligence and Spectral/Statistical Approaches for Elevating Prostate Cancer Assessment in Bi-Parametric MRI: A Pilot Study

Rulon Mayer, Yuan Yuan, Jayaram K. Udupa +5 more

Artificial Intelligence · Preprints.org · · cited 0× · DOI ↗ · more like this ·

Background Prostate cancer management optimally requires non-invasive, objective, quantitative, accurate evalua-tion of prostate tumors. Current research applies visual inspection and quantitative approaches, such as artificial intelligence (AI) focused in deep learning (DL) to evaluate MRI. Recently, a different spectral/statistical approach successfully evaluated spatially registered bi-parametric MRIs for prostate cancer. This study aimed to further as-sess and improve the spectral/statistics approach through benchmarking and combining with AI. Methods A zonal-aware self-supervised mesh network (Z-SSMNet) was applied to the same 42-patient cohort from previous spectral/statistical studies. The probability of clinical significance of prostate cancer (PCsPCa) and detec-tion map with affiliated tumor volume, eccentricity were computed for each patient. Linear and logistic regression…

NIMG-52. ARTIFICIAL INTELLIGENCE-BASED VOLUMETRIC KINETICS FOR EVALUATING TREATMENT RESPONSE IN IDH-MUTANT GLIOMA TREATED WITH IMMUNOTHERAPY

Zach Sergi, Mason J. Belue, Kelly Fernandez +6 more

Modeling and Simulation · Neuro-Oncology · · cited 0× · DOI ↗ · more like this ·

Abstract BACKGROUND Interpretating radiographic findings of glioma patients receiving immunotherapy is challenging. Repeated imaging is often required for clinical decisions, potentially delaying next line of treatment. Here we assess a glioma segmentation AI model to evaluate treatment response in IDH-mutant glioma patients receiving nivolumab. METHODS Clinical magnetic resonance imaging (MRI)s from twenty-nine patients enrolled in a clinical trial (NCT03718767) were retrospectively analyzed. All patients had at least two sequential MRIs, including baseline study. T2, T1 (pre/post-contrast), and T2 FLAIR MRI sequences were pre-processed using publicly available AI-based whole-brain segmentation models. Lesion contours were derived from a validated 4-class brain tumor segmentation AI model, distinguishing healthy brain parenchyma, peri-lesional edema, non-contrast enhancing volume…

Large-Scale Multi-Center CT and MRI Segmentation of Pancreas with Deep Learning

Zheyuan Zhang, Elif Keles, Gorkem Durak +35 more

eess.IV · · cited 0× · arXiv ↗ · more like this ·

Automated volumetric segmentation of the pancreas on cross-sectional imaging is needed for diagnosis and follow-up of pancreatic diseases. While CT-based pancreatic segmentation is more established, MRI-based segmentation methods are understudied, largely due to a lack of publicly available datasets, benchmarking research efforts, and domain-specific deep learning methods. In this retrospective study, we collected a large dataset (767 scans from 499 participants) of T1-weighted (T1W) and T2-weighted (T2W) abdominal MRI series from five centers between March 2004 and November 2022. We also collected CT scans of 1,350 patients from publicly available sources for benchmarking purposes. We developed a new pancreas segmentation method, called PanSegNet, combining the strengths of nnUNet and a Transformer network with a new linear attention module enabling volumetric computation. We tested…

Distance Map Supervised Landmark Localization for MR-TRUS Registration

Xinrui Song, Xuanang Xu, Sheng Xu +4 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

In this work, we propose to explicitly use the landmarks of prostate to guide the MR-TRUS image registration. We first train a deep neural network to automatically localize a set of meaningful landmarks, and then directly generate the affine registration matrix from the location of these landmarks. For landmark localization, instead of directly training a network to predict the landmark coordinates, we propose to regress a full-resolution distance map of the landmark, which is demonstrated effective in avoiding statistical bias to unsatisfactory performance and thus improving performance. We then use the predicted landmarks to generate the affine transformation matrix, which outperforms the clinicians' manual rigid registration by a significant margin in terms of TRE.

Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation

Pengfei Guo, Dong Yang, Ali Hatamizadeh +14 more

eess.IV · · cited 0× · arXiv ↗ · more like this ·

Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. The inherent privacy-preserving property of FL algorithms makes them especially attractive to the medical field. However, in case of heterogeneous client data distributions, standard FL methods are unstable and require intensive hyperparameter tuning to achieve optimal performance. Conventional hyperparameter optimization algorithms are impractical in real-world FL applications as they involve numerous training trials, which are often not affordable with limited compute budgets. In this work, we propose an efficient reinforcement learning (RL)-based federated hyperparameter optimization algorithm, termed Auto-FedRL, in which an online RL agent can dynamically adjust hyperparameters of each client based on the current training progress.…

Align then Refine: Text-Guided 3D Prostate Lesion Segmentation

Cuiling Sun, Linkai Peng, Adam Murphy +9 more

cs.CV · · arXiv ↗ · more like this ·

Automated 3D segmentation of prostate lesions from biparametric MRI (bp-MRI) is essential for reliable algorithmic analysis, but achieving high precision remains challenging. Volumetric methods must combine multiple modalities while ensuring anatomical consistency, but current models struggle to integrate cross-modal information reliably. While vision-language models (VLMs) are replacing the currently used architectural designs, they still lack the fine-grained, lesion-level semantics required for effective localized guidance. To address these limitations, we propose a new multi-encoder U-Net architecture incorporating three key innovations: (1) an alignment loss that enhances foreground text-image similarity to inject lesion semantics; (2) a heatmap loss that calibrates the similarity map and suppresses spurious background activations; and (3) a final-stage, confidence-gated multi-head…

Reasoning Visual Language Model for Chest X-Ray Analysis

Andriy Myronenko, Dong Yang, Baris Turkbey +10 more

cs.CV · · arXiv ↗ · more like this ·

Vision-language models (VLMs) have shown strong promise for medical image analysis, but most remain opaque, offering predictions without the transparent, stepwise reasoning clinicians rely on. We present a framework that brings chain-of-thought (CoT) reasoning to chest X-ray interpretation. Inspired by reasoning-first training paradigms, our approach is designed to learn how experts reason, not just what they conclude, by aligning intermediate steps with observable image evidence and radiology workflow. Beyond accuracy, the explicit reasoning traces support clinical auditability: they reveal why a conclusion was reached, which alternatives were considered, and where uncertainty remains, enabling quality assurance, error analysis, and safer human-AI collaboration. Our model couples high-fidelity visual encoding with a two-stage training recipe: a reasoning-style supervised fine-tuning…

Scaling Artificial Intelligence for Prostate Cancer Detection on MRI towards Organized Screening and Primary Diagnosis in a Global, Multiethnic Population (Study Protocol)

Anindo Saha, Joeran S. Bosma, Jasper J. Twilt +26 more

eess.IV · · arXiv ↗ · more like this ·

In this intercontinental, confirmatory study, we include a retrospective cohort of 22,481 MRI examinations (21,288 patients; 46 cities in 22 countries) to train and externally validate the PI-CAI-2B model, i.e., an efficient, next-generation iteration of the state-of-the-art AI system that was developed for detecting Gleason grade group $\geq$2 prostate cancer on MRI during the PI-CAI study. Of these examinations, 20,471 cases (19,278 patients; 26 cities in 14 countries) from two EU Horizon projects (ProCAncer-I, COMFORT) and 12 independent centers based in Europe, North America, Asia and Africa, are used for training and internal testing. Additionally, 2010 cases (2010 patients; 20 external cities in 12 countries) from population-based screening (STHLM3-MRI, IP1-PROSTAGRAM trials) and primary diagnostic settings (PRIME trial) based in Europe, North and South Americas, Asia and…

VHU-Net: Variational Hadamard U-Net for Body MRI Bias Field Correction

Xin Zhu, Ahmet Enis Cetin, Gorkem Durak +13 more

eess.IV · · arXiv ↗ · more like this ·

Bias field artifacts in magnetic resonance imaging (MRI) scans introduce spatially smooth intensity inhomogeneities that degrade image quality and hinder downstream analysis. To address this challenge, we propose a novel variational Hadamard U-Net (VHU-Net) for effective body MRI bias field correction. The encoder comprises multiple convolutional Hadamard transform blocks (ConvHTBlocks), each integrating convolutional layers with a Hadamard transform (HT) layer. Specifically, the HT layer performs channel-wise frequency decomposition to isolate low-frequency components, while a subsequent scaling layer and semi-soft thresholding mechanism suppress redundant high-frequency noise. To compensate for the HT layer's inability to model inter-channel dependencies, the decoder incorporates an inverse HT-reconstructed transformer block, enabling global, frequency-aware attention for the recovery…

OpenPros: A Large-Scale Dataset for Limited View Prostate Ultrasound Computed Tomography

Hanchen Wang, Yixuan Wu, Yinan Feng +11 more

physics.med-ph · · arXiv ↗ · more like this ·

Prostate cancer is one of the most prevalent and deadly cancers among men, motivating the development of accurate and accessible imaging technologies for early detection. Ultrasound computed tomography (USCT) reconstructs quantitative tissue parameters such as speed-of-sound (SOS) and is a promising low-cost alternative to existing modalities. However, prostate USCT remains challenging due to limited-angle acquisition, strong tissue heterogeneity, bone-induced wave distortion, and the lack of large-scale, anatomically realistic datasets for method development and evaluation. We introduce OPENPROS, the first large-scale benchmark dataset for limited-angle prostate USCT, designed to systematically evaluate machine learning methods for quantitative inverse problems. OPENPROS contains over 280,000 paired samples of realistic 2D SOS maps and corresponding ultrasound full-waveform data…