Early Access

Glaucoma is a leading cause of irreversible blindness worldwide, with asymptomatic early stages often delaying diagnosis and treatment. Early and accurate diagnosis requires integrating complementary information from multiple ocular imaging modalities. However, most existing studies rely on single- or dual-modality imaging, such as fundus and optical coherence tomography (OCT), for coarse binary classification, thereby restricting the exploitation of complementary information and hindering both early diagnosis and stage-specific treatment. To address these limitations, we propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset comprising scanning laser ophthalmoscopy fundus images, circumpapillary OCT images, and visual field pattern deviation maps, annotated with four disease stages, enabling effective exploitation of multimodal complementary information and facilitating accurate diagnosis and treatment across disease stages. To effectively integrate cross-modal information, we propose hierarchical attentive masked modeling (HAMM) for multimodal glaucoma classification. Our framework employs hierarchical attentive encoders and light decoders to focus cross-modal representation learning on the encoder. The attention module, named multimodal-channel graph attention (MCGA), boosts glaucoma classification performance by emulating two key clinical reasoning steps: first, it uses a multi-head modality gating mechanism to replicate ophthalmologists’ confidence scoring of fundus, OCT, and VF modalities; then, MCGA leverages a relational graph attention network to cross-examine structural-functional consistencies of weighted modalities. The experiments on GLEAM demonstrate that tri-modal fusion significantly outperforms single-modal and dual-modal configurations. Moreover, our proposed HAMM achieves superior performance compared with state-of-the-art multimodal learning methods. The dataset and code are publicly available via https://github.com/microewing/HAMM.

青光眼是全球不可逆失明的主要原因，其无症状早期阶段常常延误诊断和治疗。早期准确诊断需要整合来自多种眼部成像模式的互补信息。然而，大多数现有研究依赖于单模态或双模态成像，如眼底和光学相干断层扫描（OCT），用于粗略的二元分类...

Osteoarthritis (OA) is a debilitating joint disease in which early microstructural and compositional changes in articular cartilage (AC) are challenging to detect with current diagnostic tools. We hypothesized that multispectral imaging (MSI) could capture optical signatures that predict key AC properties. To test this, we developed and evaluated an MSI-based approach for estimating tissue thickness, proteoglycan (PG) content, collagen fiber orientation, and cell morphological properties (area and circularity). Reflectance images of bovine patellar AC were acquired using a custom-built MSI system operating at six wavelengths (550–970 nm), and machine learning models were trained to predict the targeted markers. The models yielded reliable estimates for AC thickness, whereas for PG content, collagen fiber orientation, and cell circularity, they achieved moderate accuracy. This work suggests that MSI holds promise as a label-free tool for characterizing AC and detecting early degenerative changes.

中文摘要翻译待生成

Spatial Transcriptomics (ST) technology detects gene expression from tissue biopsies, playing an emerging role in cancer diagnosis and precision medicine. However, the high cost of ST technology limits its broader application. Recently, deep learning approaches have provided insight into predicting gene expression based on H&E-stained histopathology images. Nevertheless, the relationship between morphological features and gene expression is highly complex. To address these challenges, we propose DiffBulk, a novel two-stage framework that leverages conditional diffusion models to learn expressive image representations enriched with gene expression information. In the first stage, we introduce a gene-to-image conditional diffusion model equipped with a permutationinvariant open-embedding gene encoder, which enables unified training across diverse gene panels. In the second stage, diffusion-derived features are fused with representations from a pathology foundation model, effectively bridging the domain gap and improving downstream gene expression prediction. We evaluate DiffBulk on high-quality Xenium ST data curated from the HEST dataset and the CrunchDAO challenge, constructing tile-level pseudo-bulk datasets for training and evaluation. Extensive experiments demonstrate that DiffBulk consistently outperforms state-of-the-art baselines across all metrics for gene expression prediction. These findings highlight the potential of diffusion-based gene-image representation learning and suggest promising directions for future research.

空间转录组学（ST）技术从组织活检中检测基因表达，在癌症诊断和精准医学中发挥着新兴作用。然而，ST技术的高成本限制了其更广泛的应用。最近，深度学习方法为基于H&E染色组织病理学图像预测基因表达提供了思路。然而，m…

Multiple instance learning (MIL) has emerged as the dominant paradigm for whole slide image (WSI) analysis in computational pathology, achieving strong diagnostic performance through patch-level feature aggregation. However, existing MIL methods face critical limitations: (1) they rely on attention mechanisms that lack causal interpretability—the ability to explain why predictions vary across demo...

中文摘要翻译待生成

3D reconstruction of the Right Ventricle (RV) from echocardiograms is crucial for accurate clinical evaluation of cardiac function. However, existing methods are hindered by the complex RV anatomy and the incomplete spatial information inherent in 2D multi-view echocardiograms. Therefore, we propose RVDeformer, a sparse point cloud-guided framework for RV 3D reconstruction. RVDeformer reformulates the reconstruction task as a mesh deformation problem, learning to deform a predefined template mesh to match the target structure under the guidance of the sparse anatomical point cloud. Specifically, this framework employs the end-to-end neural network RVDeformNet to extract the features of the point cloud and template mesh for predicting the displacement of each mesh vertex. We design a point cloud-mesh fusion module that can effectively align and fuse features from the two modalities to enhance the representation ability of the model. We conduct extensive validation on a clinical dataset of 1,278 cases and demonstrate that RVDeformer outperforms existing state-of-the-art methods, achieving a Chamfer Distance (CD) of 2.24±0.55 mm, an F1-score of 0.74±0.10 at the 3 mm threshold, and a Volumetric Similarity (VS) of 91.53±2.28%, with significant potential for clinical applications. The code is available at https://github.com/onezh95/RVDeformer.

从超声心动图中进行右心室（RV）的三维重建对于准确评估心脏功能至关重要。然而，现有方法受限于复杂的RV解剖结构以及二维多视图超声心动图固有的空间信息不完整。因此，我们提出了RVDeformer，一种稀疏点云引导的RV三维重建框架。RVDeformer重新表述了...

Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular hemodynamics marker to detect heart failure. In clinical practice, Right Heart Catheterization is considered a gold standard for assessing cardiac hemodynamics while non-invasive methods are often needed to screen high-risk patients from a large population. In this paper, we propose a multimodal learning pipeline to predict PAWP marker. We utilize complementary information from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal features from CMR scans using tensor-based learning. We propose a graph attention network to select important EHR features for prediction, where we model subjects as graph nodes and feature relationships as graph edges using the attention mechanism. We design four feature fusion strategies: early, intermediate, late, and hybrid fusion. With a linear classifier and linear fusion strategies, our pipeline is interpretable. We validate our pipeline on a large dataset of 2, 641 subjects from our ASPIRE registry. The comparative study against state-of-the-art methods confirms the superiority of our pipeline. The decision curve analysis further validates that our pipeline can be applied to screen a large population. The code is available at https://github.com/prasunc/ hemodynamics.

肺动脉楔压（PAWP）是检测心力衰竭的重要心血管血流动力学标志物。在临床实践中，右心导管检查被认为是评估心脏血流动力学的金标准，但通常需要无创方法从大量人群中筛查高风险患者。本文提出了一种多模态学习流程来预测PA...

With the rapid development of pathology foundation models, there is a growing demand for efficient fine-tuning strategies tailored to downstream tasks. However, existing parameter-efficient fine-tuning approaches are largely task-agnostic and exhibit limited generalization to histopathological images, particularly for unseen cancers and stains, due to substantial stain variability and the complexity of tissue microenvironments. To address these challenges, we present Histopathology-induced Adapter (HiAdapter), which incorporates domain-specific insights into staining and imaging mechanisms of histopathology. HiAdapter reconstructs stain-invariant representations via a Stain-invariant Adapter (S-Adapter) and integrates morphological features through a Morphology-aware Adapter (M-Adapter), effectively bridging the gap between low-level optical properties and high-level tissue semantics. Additionally, we introduce a Pathology Prototypical Contrastive Loss (PPCLoss) to reduce inter-class similarity and mitigate intra-class heterogeneity, enhancing feature discriminability. Extensive experiments using three pathology foundation models (CTransPath, CONCH and UNI) across six benchmarks, including two public datasets, an osteosarcoma tissue classification dataset (56,178 patches) and a chondrosarcoma necrosis classification dataset (3,867 patches) for unseen cancers generalization, as well as an IHC-stained dataset (4,967 patches) and an HIF1A IHC-stained dataset (4,433 patches) for unseen stains generalization, demonstrate the effectiveness of HiAdapter in both efficiency and accuracy. HiAdapter achieves an average improvement of 2.15 in F1 and 1.55 in accuracy over the second-best performer, maintaining strong biological and diagnostic interpretability. External validation on an independent osteosarcoma dataset (9,535 patches) and WSI-level survival analysis (178 slides) further confirm the superior generalizability and underscore the potential for patient-level diagnosis and prognosis in clinical practice. Our code is available at HiAdapter.

中文摘要翻译待生成

Volume electron microscopy (vEM) has revolutionized the nanoscale reconstruction of synapses in neural circuits. However, large-scale vEM techniques relying on serial sectioning suffer from severe anisotropy, where axial resolution is far worse than lateral resolution. This anisotropic imaging induces discontinuities in biological architectures across 3D space, compromising reconstruction accuracy...

中文摘要翻译待生成

Abnormality detection is a crucial yet challenging task in medical image analysis. Distinguishing abnormalities from normal data by learning to reconstruct normal-only data alleviates the reliance on labeled datasets. However, many studies, even if unsupervised, rely on a labeled validation set to select the best model for inference from multiple training iterations. For many diseases labeled data are unavailable and substantially time consuming to obtain. To address this, AUC p - a novel metric that supports abnormality detection for unsupervised and self-supervised methods is proposed. Instead of evaluating the realism of reconstructed images to select the best of model for inference, it focuses on actual detection performance and without requiring an annotated test set. Assuming the pseudo ground truth of all unannotated samples in the test set as abnormal/positive and using traditional AUC calculation, AUC p scores are derived. Given a large and representative training set of normal samples, we show mathematical and empirical evidence that model selection using AUC p scores improves disease detection in terms of unsupervised and self-supervised methods over conventional metrics. Using two unsupervised methods for neurologic disease detection and self-supervised methods on diverse datasets, our results demonstrate that the AUC p score effectively identifies the optimal model for inference, significantly enhancing abnormality and disease detection. The corresponding implementations are available in https://github.com/mahfuzmohammad/AUCp.

异常检测是医学图像分析中一项关键且具有挑战性的任务。通过学习仅重构正常数据来区分异常与正常数据，减轻了对标记数据集的依赖。然而，许多研究即使是无监督的，也依赖于标记的验证集从多次训练迭代中选择最佳推理模型。对于许多疾病，标记数据...

Synthetic medical image data can unlock the potential of deep learning (DL)-based clinical decision support (CDS) systems through the creation of large scale, privacy-preserving, training sets. Despite the significant progress in this field, there is still a largely unanswered research question: “How can we quantitatively assess the similarity of a synthetically generated set of images with a set of real images in a given application domain?”. Today, answers to this question are mainly provided via user evaluation studies, inception-based measures, and the classification performance achieved on synthetic images. This paper proposes a novel measure to assess the similarity between synthetically generated and real sets of images, in terms of their utility for the development of DL-based CDS systems. Inspired by generalized neural additive models, and unlike inception-based measures, the proposed measure is interpretable (Interpretable Utility Similarity, IUS), explaining why a synthetic dataset could be more useful than another one in the context of a CDS system based on clinically relevant image features. The experimental results on publicly available benchmark datasets from various color medical imaging modalities including endoscopic, dermoscopic and fundus imaging, indicate that selecting synthetic images of high utility similarity using IUS can result in relative improvements of up to 54.6% in terms of classification performance. The generality of IUS for synthetic data assessment is demonstrated also for grayscale X-ray and ultrasound imaging modalities. IUS implementation is available at https://github.com/innoisys/ius.

合成医学图像数据可以通过创建大规模、隐私保护的训练集，释放基于深度学习（DL）的临床决策支持（CDS）系统的潜力。尽管该领域取得了显著进展，但仍有一个基本未解决的研究问题：“我们如何定量评估合成生成图像集与真实图像集的相似性？”

Universal medical image segmentation aims to unify heterogeneous datasets or annotation protocols within a single adaptable framework. However, existing prompt-based universal models often overlook background context, neglect hierarchical task dependencies, and struggle to generalize to unseen annotation granularities. These challenges are particularly pronounced in OCT-based retinal layer segment...

中文摘要翻译待生成

Coronary artery stenosis is a leading cause of cardiovascular disease, diagnosed by analyzing the coronary arteries from multiple angiography views. Although numerous deep-learning models have been proposed for stenosis detection from a single angiography view, their performance heavily relies on expensive view-level annotations, which are often not readily available in hospital systems. Moreover, these models fail to capture the temporal dynamics and dependencies among multiple views, which are crucial for clinical diagnosis. To address this, we propose SegmentMIL, a transformer-basedmulti-viewmultiple-instance learning framework for patient-level stenosis classification. Trained on a real-world clinical dataset, using patient-level supervision and without any view-level annotations, SegmentMIL jointly predicts the presence of stenosis and localizes the affected anatomical region, distinguishing between the right and left coronary arteries and their respective segments. SegmentMIL obtains high performance on internal and external evaluations and outperforms both view-level models and classical MIL baselines, underscoring its potential as a clinically viable and scalable solution for coronary stenosis diagnosis. Our code is available at https://github.com/NikolaCenic/mil-stenosis.

中文摘要翻译待生成

Assessing the quality of automatic image segmentation is crucial in clinical practice, but often very challenging due to the limited availability of ground truth annotations. Reverse Classification Accuracy (RCA) is an approach that estimates the quality of new predictions on unseen samples by training a segmenter on those predictions, and then evaluating it against existing annotated images. In this work we introduce ConfIC-RCA (Conformal In-Context RCA), a novel method for automatically estimating segmentation quality with statistical guarantees in the absence of ground-truth annotations, which consists of two main innovations. First, In-Context RCA, which leverages recent in-context learning models for image segmentation and incorporates retrieval-augmentation techniques to select the most relevant reference images. This approach enables efficient quality estimation with minimal reference data while avoiding the need of training additional models. Second, we introduce Conformal RCA, which extends both the original RCA framework and In-Context RCA to go beyond point estimation. Using tools from split conformal prediction, Conformal RCA produces prediction intervals for segmentation quality providing statistical guarantees that the true score lies within the estimated interval with a user-specified probability. Validated across 10 different medical imaging tasks in various organs and modalities, our methods demonstrate robust performance and computational efficiency, offering a promising solution for automated quality control in clinical workflows, where fast and reliable segmentation assessment is essential. The code is available at https://github.com/mcosarinsky/Conformal-In-Context-RCA.

中文摘要翻译待生成

Proton Density Fat Fraction (PDFF) is the gold standard for non-invasive fatty liver diagnosis, but its reliance on Magnetic Resonance Imaging (MRI) limits broad clinical applicability. Motivated by the accessibility of B-mode Ultrasound (US) in fatty liver assessment, we propose a novel framework for liver segmentation and PDFF prediction from US images. To enhance generalization ability despite limited paired US-PDFF data, our framework integrates a cross-task self-supervised pretext task that extracts semantic features to guide echo intensity capture, benefiting both liver segmentation and PDFF prediction. To address the noise and artifacts inherent in US images, our framework leverages T2-weighted imaging (T2WI) exclusively during training to establish a feature bridge between US and PDFF, thereby enhancing PDFF prediction. Once trained, the model relies solely on US for inference, making it a practical and cost-effective alternative to MRI-based PDFF estimation. Additionally, our framework introduces an uncertainty-augmented adversarial loss function to refine liver boundary delineation, further improving segmentation and PDFF prediction accuracy. Experimental results demonstrate that our method outperforms state-of-the-art methods in liver segmentation and PDFF prediction; and in a specific application study, our predicted PDFF achieves accuracy comparable to real PDFF for hepatic steatosis classification, highlighting its clinical potential. The full source code and detailed documentation are publicly available at https://github.com/D0ngZhang/SSTB.

质子密度脂肪分数（PDFF）是无创脂肪肝诊断的金标准，但其依赖于磁共振成像（MRI）限制了广泛的临床应用。受B型超声（US）在脂肪肝评估中的可及性启发，我们提出了一种新颖的框架，用于从超声图像中进行肝脏分割和PDFF预测。为了提高泛化能力，尽管...

Diffusion MRI mostly involves quantification of how water diffuses in tissue, and its relationship with tissue microstructure. The technique promises great impact for soft tissue studies, since it provides a method of studying tissue microstructure based on millimetre-scale measurements. While multiple analytical models have been proposed to describe how water diffuses in tissue, specifically by s...

中文摘要翻译待生成

Functional neuroimaging with ultrafast ultrasound is an emerging neuroimaging tool for studying neural activities in the rodent brain. Existing methods, however, are challenged by the compromise between functional imaging sensitivity (i.e., sensitivity in detecting neural responses) and spatial resolution. For example, functional ultrasound (fUS) uses native red blood cells (RBCs) as imaging targe...

中文摘要翻译待生成

Hyperspectral imaging delivers high-resolution spectral-spatial information to support molecular tissue characterization, but its clinical utility is far from being fully realized. Existing segmentation techniques are constrained by fixed or suboptimal band selection strategies and insufficient frequency-domain modeling, which limit their ability to fully exploit discriminative spectral cues and s...

中文摘要翻译待生成

To be adopted in safety-critical domains like medical image analysis, AI systems must provide human-interpretable decisions. Variational Information Pursuit (VIP) offers an interpretable-by-design framework by sequentially querying input images for human-understandable concepts, using their presence or absence to make predictions. However, existing V-IP methods overlook sample-specific uncertainty...

中文摘要翻译待生成

This paper addresses the challenge of multi-disease diagnosis by integrating causal reasoning into the diagnostic framework. In clinical practice, multiple conditions often co-occur, making multi-disease diagnosis more relevant than isolated single-disease cases. However, most deep learning methods focus on single-disease detection and fail to capture the complexity of diagnosing concurrent conditions. Even in multi-label settings, existing approaches mainly rely on correlation-based inference, capturing statistical associations rather than true causal relationships. This can lead to spurious feature-disease associations, where features linked to one disease are mistakenly attributed to another due to frequent co-occurrence, ultimately undermines diagnostic accuracy and interpretability. To address this challenge, we propose a novel framework that incorporates causal intervention into multi-label medical image diagnosis, enabling the model to identify true causal signals rather than misleading correlations arising from co-occurring diseases. Specifically, we model latent disease-related confounders and apply backdoor adjustment to disentangle genuine causal effects from spurious associations. This is achieved by implicitly learning shared feature representations that serve as confounding variables, which are then used to refine image-derived features during prediction. The resulting causal adjustment allows the model to focus on disease-specific cues, improving accuracy and interpretability. Extensive experiments on four diverse medical imaging datasets: ODIR (color fundus photography), LID-FFA (fundus fluorescein angiography), Endo (colonoscopy), and Chestpert (X-ray) demonstrate that our method consistently outperforms existing approaches. Furthermore, our model also effectively separates the diagnosis of co-occurring diseases, high-lighting the potential of causal reasoning to enhance the reliability and clinical applicability of AI-assisted diagnosis. The source code is publicly available at https://github.com/davelailai/BankCausal.git.

本文通过将因果推理融入诊断框架，解决了多疾病诊断的挑战。在临床实践中，多种疾病常常同时发生，使得多疾病诊断比孤立单疾病病例更具相关性。然而，大多数深度学习方法专注于单疾病检测，未能捕捉到诊断并发疾病的复杂性...

Neuron segmentation in complex mouse brain images improves neuron reconstruction and supports studies of brain structure and function, while the existing deep learning-based methods do not sufficiently exploit prior information, including neuronal morphology and imaging mechanism. We propose NUNet-LLM, the first LLM-integrated framework for neuron segmentation and reconstruction. NUNet-LLM consist...

中文摘要翻译待生成

Accurate Cobb angle measurement is essential for scoliosis assessment but remains labor-intensive and observer-dependent. We introduce DGPDT (Detection-Guided Prompt-driven Transformer), a unified transformer-based framework that integrates vertebra detection and foundation-model segmentation for generalizable spinal analysis. A Roboflow Detection Transformer (RF-DETR) with a DINOv2 backbone localizes vertebrae, followed by post-processing to ensure anatomical continuity. The resulting bounding boxes serve as automatic prompts for a fine-tuned Segment Anything Model 2.1 (SAM 2.1), which generates high-resolution vertebral masks. Cobb angles are then computed from vertebral masks, enabling estimation of both main and compensatory curves. Evaluations on the in-house (TVGH-SpineXR) and external (SpineWeb-16) datasets demonstrate encouraging performance on both internal and external datasets, achieving mean Dice coefficients of 0.944 and 0.781, respectively, and mean absolute Cobb angle errors of approximately 2–3° in-domain and 4.93° under cross-domain evaluation. Despite being trained solely on TVGH-SpineXR, DGPDT maintains accuracy comparable to models trained directly on the benchmark dataset. By coupling detection-guided prompting with transformer-based segmentation, DGPDT achieves a clinically acceptable mean absolute error (<5°), suggesting good reproducibility and potential applicability beyond the training dataset.

中文摘要翻译待生成

Aggregating features of tens of thousands of patches into Whole Slide Images (WSIs) representations via aggregators is a crucial step in computational pathology. However, existing aggregation strategies overlook the morphological variability of tissue regions in WSIs stemming from differences in clinical procedures and tumor characteristics, leading to two critical limitations: 1) attention collap...

中文摘要翻译待生成

Medical Referring Image Segmentation (MRIS) involves segmenting target regions in medical images based on natural language descriptions. While achieving promising results, recent approaches usually involve complex design of multimodal fusion or multi-stage decoders. In this work, we propose NTP-MRISeg, a novel framework that reformulates MRIS as an autoregressive next-token prediction task over a ...

中文摘要翻译待生成

Most functional magnetic resonance imaging studies rely on estimates of hierarchically organized functional brain networks whose segregation and integration reflect the cognitive and behavioral changes in humans. However, most existing methods for estimating the community structure of networks from both individual and group-level analysis methods do not account for the variability between subjects...

中文摘要翻译待生成

Low light conditions in endoscopic imaging would lead to poor visibility, reduced contrast, and increased noise, which may hinder accurate diagnosis and surgical guidance. Against this low-light endoscopic image enhancement (LLEIE) task, inspired by the remarkable performance of pretrained CLIP in downstream vision tasks, in this paper, we carefully investigate the pretrained priors of CLIP and em...

中文摘要翻译待生成