Volume 45, Issue 7

Spatial Transcriptomics (ST) technology detects gene expression from tissue biopsies, playing an emerging role in cancer diagnosis and precision medicine. However, the high cost of ST technology limits its broader application. Recently, deep learning approaches have provided insight into predicting gene expression based on H&E-stained histopathology images. Nevertheless, the relationship between morphological features and gene expression is highly complex. To address these challenges, we propose DiffBulk, a novel two-stage framework that leverages conditional diffusion models to learn expressive image representations enriched with gene expression information. In the first stage, we introduce a gene-to-image conditional diffusion model equipped with a permutationinvariant open-embedding gene encoder, which enables unified training across diverse gene panels. In the second stage, diffusion-derived features are fused with representations from a pathology foundation model, effectively bridging the domain gap and improving downstream gene expression prediction. We evaluate DiffBulk on high-quality Xenium ST data curated from the HEST dataset and the CrunchDAO challenge, constructing tile-level pseudo-bulk datasets for training and evaluation. Extensive experiments demonstrate that DiffBulk consistently outperforms state-of-the-art baselines across all metrics for gene expression prediction. These findings highlight the potential of diffusion-based gene-image representation learning and suggest promising directions for future research.

空间转录组学（ST）技术从组织活检中检测基因表达，在癌症诊断和精准医学中发挥着新兴作用。然而，ST技术的高成本限制了其更广泛的应用。最近，深度学习方法为基于H&E染色组织病理学图像预测基因表达提供了思路。然而，m…

Age-related macular degeneration with abnormal blood vessel growth (neovascular AMD) is the leading cause of vision loss in elderly populations. While anti-VEGF injections are the standard treatment, they present financial burdens for patients and vary in effectiveness. Predicting treatment efficacy is therefore crucial for patient care. Current prediction methods fail to fully integrate informati...

中文摘要翻译待生成

Conventional registration approaches frequently underperform when applied to sparse feature alignment (e.g., retinal vessels and filamentous collagen fibers in second-harmonic generation (SHG) and bright-field (BF) images), as these tasks demand simultaneous handling of global affine registration and local deformation correction. End-to-end learning-based approaches struggle with minimal effective...

中文摘要翻译待生成

Universal medical image registration through a single model handling various registration tasks has attracted increasing interest. However, existing deep learning-based methods face two major challenges in adapting to universal registration tasks: 1) they lack generalizable feature representation capabilities for cross-task registration; 2) they rely solely on model architectures with fixed parame...

中文摘要翻译待生成

Domain adaptation methods aim to bridge the gap between datasets by enabling knowledge transfer across domains, reducing the need for additional expert annotations. However, many approaches struggle with reliability in the target domain, an issue particularly critical in medical image segmentation, where accuracy and anatomical validity are essential. This challenge is further exacerbated in spati...

中文摘要翻译待生成

Objective, task-based measures of image quality (IQ) have been widely advocated for assessing and optimizing medical imaging technologies. Besides signal detection theory-based measures, information-theoretic quantities have been proposed to quantify task-based IQ. For example, task-specific information (TSI), defined as the mutual information between an image and a task variable, represents an op...

中文摘要翻译待生成

Cardiac Magnetic Resonance (CMR) imaging is widely used to personalize heart models for cardiac digital twin analysis because of its ability to visualize soft tissues and capture dynamic functions. However, CMR images have an anisotropic nature, characterized by large inter-slice distances and misalignments from cardiac motion. These limitations result in data loss and measurement inaccuracies, hi...

中文摘要翻译待生成

Gleason grading, the clinical gold standard for prostate cancer assessment, is based on subjective evaluation of glandular architecture, resulting in interobserver variability and limited scalability. This highlights the need for automated grading systems. However, their development is hindered by the scarcity of annotated pathology data. Self-supervised learning (SSL) presents a promising solutio...

中文摘要翻译待生成

Automated whole-body lesion segmentation in 18F-FDG PET/CT images marks a pivotal breakthrough in oncological diagnostics, substantially improving the accuracy and efficiency of tumor burden assessment. Manual segmentation is often plagued by significant interobserver variability, underscoring the necessity for automated solutions. The synergistic combination of PET’s exceptional sensitivity for d...

中文摘要翻译待生成

Sparse-View CT (SVCT) reconstruction improves temporal resolution and reduces radiation dose, yet its clinical use is hindered by artifacts due to view reduction and domain shifts from scanner, protocol, or anatomical variations, leading to performance degradation in out-of-distribution (OOD) scenarios. We propose a Cross-Distribution Diffusion Priors-Driven Iterative Reconstruction (CDPIR) framew...

中文摘要翻译待生成

Sampling jitter, i.e., random deviations in the time instants when samples are taken, causes frequency-dependent noise that reduces the signal-to-noise ratio (SNR). This paper generalizes the concept of jitter to magnetoencephalography (MEG) sensor arrays that spatially sample the quasistatic magnetic field due to brain activity. It is shown that spatial jitter, i.e., random deviations in the MEG ...

中文摘要翻译待生成

3D brain MRI studies often examine subtle morphometric differences between cohorts that are hard to detect visually. Given the high cost of MRI acquisition, these studies could greatly benefit from image syntheses, particularly counterfactual image generation, as has been the case for applications in computer vision. However, counterfactual models struggle to produce anatomically plausible MRIs du...

中文摘要翻译待生成

In recent times, geometric foundation models have demonstrated remarkable performance in depth estimation tasks, benefiting from exposure to large-scale data that enables the learning of intricate geometric structures and spatial dependencies. However, their large parameter sizes and high computational complexity pose significant challenges in meeting the efficiency requirements of downstream surg...

中文摘要翻译待生成

The purpose of this work is to demonstrate a robust and clinically validated method for correcting sound speed aberrations in medical ultrasound. We propose a correction method that calculates the focus delays directly from the observed two-way distributed average sound speed. The method beamforms multiple coherence images and selects the sound speed that maximizes the coherence for each image pix...

中文摘要翻译待生成

Clinically, bi-parametric MRI (bp-MRI), including T2-weighted imaging, diffusion-weighted imaging, and apparent diffusion coefficient map, offers essential prior localization of biopsy and focal therapy for suspicious clinically significant prostate cancer (csPCa), and accurate csPCa delineation from bp-MRI is crucial for better outcomes. However, due to the complexity and high variability in appe...

中文摘要翻译待生成

Three-dimensional ultrasound localization microscopy (ULM) enables comprehensive visualization of the vasculature, thereby improving diagnostic reliability. Nevertheless, its clinical translation remains challenging, as the exponential growth in voxel count for full 3D reconstruction imposes heavy computational demands and extensive post-processing time. In this row-column array (RCA)-based 3D in ...

中文摘要翻译待生成

Glaucoma is a leading cause of irreversible blindness worldwide, with asymptomatic early stages often delaying diagnosis and treatment. Early and accurate diagnosis requires integrating complementary information from multiple ocular imaging modalities. However, most existing studies rely on single- or dual-modality imaging, such as fundus and optical coherence tomography (OCT), for coarse binary classification, thereby restricting the exploitation of complementary information and hindering both early diagnosis and stage-specific treatment. To address these limitations, we propose glaucoma lesion evaluation and analysis with multimodal imaging (GLEAM), the first publicly available tri-modal glaucoma dataset comprising scanning laser ophthalmoscopy fundus images, circumpapillary OCT images, and visual field pattern deviation maps, annotated with four disease stages, enabling effective exploitation of multimodal complementary information and facilitating accurate diagnosis and treatment across disease stages. To effectively integrate cross-modal information, we propose hierarchical attentive masked modeling (HAMM) for multimodal glaucoma classification. Our framework employs hierarchical attentive encoders and light decoders to focus cross-modal representation learning on the encoder. The attention module, named multimodal-channel graph attention (MCGA), boosts glaucoma classification performance by emulating two key clinical reasoning steps: first, it uses a multi-head modality gating mechanism to replicate ophthalmologists’ confidence scoring of fundus, OCT, and VF modalities; then, MCGA leverages a relational graph attention network to cross-examine structural-functional consistencies of weighted modalities. The experiments on GLEAM demonstrate that tri-modal fusion significantly outperforms single-modal and dual-modal configurations. Moreover, our proposed HAMM achieves superior performance compared with state-of-the-art multimodal learning methods. The dataset and code are publicly available via https://github.com/microewing/HAMM.

青光眼是全球不可逆失明的主要原因，其无症状早期阶段常常延误诊断和治疗。早期准确诊断需要整合来自多种眼部成像模式的互补信息。然而，大多数现有研究依赖于单模态或双模态成像，如眼底和光学相干断层扫描（OCT），用于粗略的二元分类...

Intraoperative registration of real-time ultra-sound (iUS) to preoperative Magnetic Resonance Imaging (MRI) remains an unsolved problem due to severe modality-specific differences in appearance, resolution, and field-of-view. To address this, we propose a novel 3D cross-modal keypoint descriptor for MRI–iUS matching and registration. Our approach employs a patient-specific matching-by-synthesis approach, generating synthetic iUS volumes from preoperative MRI. This enables supervised contrastive training to learn a shared descriptor space. A probabilistic keypoint detection strategy is then employed to identify anatomically salient and modality-consistent locations. During training, a curriculum-based triplet loss with dynamic hard negative mining is used to learn descriptors that are i) robust to iUS artifacts such as speckle noise and limited coverage, and ii) rotation-invariant. At inference, the method detects keypoints in MR and real iUS images and identifies sparse matches, which are then used to perform rigid registration. Our approach is evaluated using 3D MRI-iUS pairs from the ReMIND dataset. Experiments show that our approach outperforms state-of-the-art keypoint matching methods across 11 patients, with an average precision of 69.8%. For image registration, our method achieves a competitive mean Target Registration Error of 2.39 mm on the ReMIND2Reg benchmark. Compared to existing iUS-MR registration approaches, our framework is interpretable, requires no manual initialization, and shows robustness to iUS field-of-view variation. Code, data and model weights are available at https://github.com/morozovdd/CrossKEY .

术中实时超声（iUS）与术前磁共振成像（MRI）的配准由于外观、分辨率和视野上的严重模态特异性差异仍然是一个未解决的问题。为了解决这个问题，我们提出了一种新的用于MRI-iUS匹配和配准的三维跨模态关键点描述符。我们的方法采用了一种特定患者的合成匹配方法...

Mild cognitive impairment (MCI) is the prodromal stage of dementia involving complex interactions between the brain and peripheral organs. Emerging evidence indicates that heart dysfunction and gut microbiota dysbiosis can contribute to MCI pathogenesis. Yet, these discoveries of cross-organ interactions have not been applied to assist MCI diagnosis. In this work, we propose a novel diagnostic fra...

中文摘要翻译待生成

Pneumonia is an acute respiratory infection, posing a serious threat to health and lives. Lung ultrasound (LUS), as a non-invasive and rapid imaging technique, can monitor real-time changes in lung, providing valuable assistance in clinical diagnosis. However, most LUS studies are limited to frame-level analysis and ignore respiratory cycle changes, leading to diagnostic errors. To address these p...

中文摘要翻译待生成

Vision-Language Models (VLMs) in visual question answering (VQA) offer a unique opportunity to enhance intra-operative decision-making, promote intuitive interactions, and significantly advance surgical education. However, the development of VLMs for surgical VQA is challenging due to limited datasets and the risk of overfitting and catastrophic forgetting during full fine-tuning of pretrained weights. While parameter-efficient techniques like Low-Rank Adaptation (LoRA) and Matrix of Rank Adaptation (MoRA) address adaptation challenges, their uniform parameter distribution overlooks the feature hierarchy in deep networks, where earlier layers, that learn general features, require more parameters than later ones. This work introduces PitVQA++ with an Open-ended PitVQA dataset and vector matrix-low-rank adaptation (Vector-MoLoRA), an innovative VLM fine-tuning approach for adapting GPT-2 to pituitary surgery. Open-Ended PitVQA comprises 109,173 frames from 25 procedural videos with 795,270 question-answer sentence pairs, covering key surgical elements such as phase and step recognition, context understanding, tool detection, localization, and interactions recognition. Vector-MoLoRA incorporates the principles of LoRA and MoRA to develop a matrix-low-rank adaptation strategy that employs rank vectors to allocate more parameters to earlier layers, gradually reducing them in the later layers. Our approach, validated on the Open-Ended PitVQA and EndoVis18-VQA datasets, effectively mitigates catastrophic forgetting while significantly enhancing performance over recent baselines. Performance-rejection analysis further highlights Vector-MoLoRA’s enhanced reliability and trust-worthiness in handling uncertain predictions. Our source code and dataset is available at https://github.com/ HRL-Mike/PitVQA-Plus.

视觉语言模型在视觉问答中为增强术中决策、促进直观交互以及显著推进外科教育提供了独特的机会。然而，由于数据集有限，以及在预训练权重完全微调过程中存在过拟合和灾难性遗忘的风险，用于手术视觉问答的视觉语言模型的开发面临挑战。

Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular hemodynamics marker to detect heart failure. In clinical practice, Right Heart Catheterization is considered a gold standard for assessing cardiac hemodynamics while non-invasive methods are often needed to screen high-risk patients from a large population. In this paper, we propose a multimodal learning pipeline to predict PAWP marker. We utilize complementary information from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal features from CMR scans using tensor-based learning. We propose a graph attention network to select important EHR features for prediction, where we model subjects as graph nodes and feature relationships as graph edges using the attention mechanism. We design four feature fusion strategies: early, intermediate, late, and hybrid fusion. With a linear classifier and linear fusion strategies, our pipeline is interpretable. We validate our pipeline on a large dataset of 2, 641 subjects from our ASPIRE registry. The comparative study against state-of-the-art methods confirms the superiority of our pipeline. The decision curve analysis further validates that our pipeline can be applied to screen a large population. The code is available at https://github.com/prasunc/ hemodynamics.

肺动脉楔压（PAWP）是检测心力衰竭的重要心血管血流动力学标志物。在临床实践中，右心导管检查被认为是评估心脏血流动力学的金标准，但通常需要无创方法从大量人群中筛查高风险患者。本文提出了一种多模态学习流程来预测PA...

Aggregating features of tens of thousands of patches into Whole Slide Images (WSIs) representations via aggregators is a crucial step in computational pathology. However, existing aggregation strategies overlook the morphological variability of tissue regions in WSIs stemming from differences in clinical procedures and tumor characteristics, leading to two critical limitations: 1) attention collap...

中文摘要翻译待生成

Fiber tract segmentation is crucial for clinical applications such as brain function interpretation and surgical planning. Existing methods typically adopt either a cortical-parcellation-based or fiber clustering approach, but fail to simultaneously integrate heterogeneous information (e.g., streamline shape, point position, anatomical priors). In this work, we propose Fiber HGNN, a novel heteroge...

中文摘要翻译待生成

Most functional magnetic resonance imaging studies rely on estimates of hierarchically organized functional brain networks whose segregation and integration reflect the cognitive and behavioral changes in humans. However, most existing methods for estimating the community structure of networks from both individual and group-level analysis methods do not account for the variability between subjects...

中文摘要翻译待生成