Volume 36, Issue 1

Surgical workflow recognition has numerous potential medical applications, such as the automatic indexing of surgical video databases and the optimization of real-time operating room scheduling, among others. As a result, surgical phase recognition has been studied in the context of several kinds of surgeries, such as cataract, neurological, and laparoscopic surgeries. In the literature, two types of features are typically used to perform this task: visual features and tool usage signals. However, the used visual features are mostly handcrafted. Furthermore, the tool usage signals are usually collected via a manual annotation process or by using additional equipment. In this paper, we propose a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information. In previous studies, it has been shown that the tool usage signals can provide valuable information in performing the phase recognition task. Thus, we present a novel CNN architecture, called EndoNet, that is designed to carry out the phase recognition and tool presence detection tasks in a multi-task manner. To the best of our knowledge, this is the first work proposing to use a CNN for multiple recognition tasks on laparoscopic videos. Experimental comparisons to other methods show that EndoNet yields state-of-the-art results for both tasks.

手术工作流程识别具有众多潜在的医学应用，例如手术视频数据库的自动索引和实时手术室调度的优化等。因此，在多种手术（如白内障手术、神经外科手术和腹腔镜手术）的背景下，手术阶段识别已被研究。文献中通常使用两种特征来执行此任务：视觉特征和工具使用信号。然而，所使用的视觉特征大多是手工设计的。此外，工具使用信号通常通过手动注释过程或使用额外设备来收集。在本文中，我们提出了一种新的阶段识别方法，该方法使用卷积神经网络（CNN）从胆囊切除术视频中自动学习特征，并且仅依赖视觉信息。以往的研究表明，工具使用信号可以为执行阶段识别任务提供有价值的信息。因此，我们提出了一种新的CNN架构，称为EndoNet，旨在以多任务方式执行阶段识别和工具存在检测任务。据我们所知，这是首次提出使用CNN进行腹腔镜视频多识别任务的工作。与其他方法的实验比较表明，EndoNet在两项任务上均达到了最先进的结果。

Robust clutter filtering is essential for ultrasound small vessel imaging. Eigen-based clutter filtering techniques have recently shown great improvement in clutter rejection over conventional clutter filters in small animals. However, for in vivo human imaging, eigen-based clutter filtering can be challenging due to the complex spatially-varying tissue and noise characteristics. To address this c...

鲁棒的杂波滤波对于超声小血管成像至关重要。基于特征值的杂波滤波技术最近在小型动物中显示出比传统杂波滤波器更好的杂波抑制效果。然而，在人体活体成像中，由于复杂的空间变化组织和噪声特性，基于特征值的杂波滤波可能具有挑战性。为了解决这个问题...

Accurate segmentation of cervical cells in Pap smear images is an important step in automatic pre-cancer identification in the uterine cervix. One of the major segmentation challenges is overlapping of cytoplasm, which has not been well-addressed in previous studies. To tackle the overlapping issue, this paper proposes a learning-based method with robust shape priors to segment individual cell in ...

在Pap涂片图像中准确分割宫颈细胞是自动识别子宫颈癌前病变的重要步骤。主要分割挑战之一是细胞质重叠，这在先前研究中尚未得到良好解决。为解决重叠问题，本文提出一种基于学习的方法，利用鲁棒形状先验来分割单个细胞...

Spectral computed tomography (CT) produces an energy-discriminative attenuation map of an object, extending a conventional image volume with a spectral dimension. In spectral CT, an image can be sparsely represented in each of multiple energy channels, and are highly correlated among energy channels. According to this characteristics, we propose a tensor-based dictionary learning method for spectr...

光谱计算机断层成像（CT）产生物体的能量分辨衰减图，将传统图像体积扩展到光谱维度。在光谱CT中，图像可以在多个能量通道中的每一个中稀疏表示，并且能量通道之间高度相关。根据这一特性，我们提出了一种基于张量字典学习的方法用于光谱CT重建...

The 3D distribution of the cortical and trabecular bone mass in the proximal femur is a critical component in determining fracture resistance that is not taken into account in clinical routine Dual-energy X-ray Absorptiometry (DXA) examination. In this paper, a statistical shape and appearance model together with a 3D-2D registration approach are used to model the femoral shape and bone density di...

股骨近端皮质骨和小梁骨的三维分布是决定抗骨折能力的关键因素，但在临床常规双能X线吸收测量法（DXA）检查中未被考虑。本文采用统计形状和外观模型以及3D-2D配准方法来建模股骨形状和骨密度分...

Ultrasound super-localization microscopy techniques presented in the last few years enable non-invasive imaging of vascular structures at the capillary level by tracking the flow of ultrasound contrast agents (gas microbubbles). However, these techniques are currently limited by low temporal resolution and long acquisition times. Super-resolution optical fluctuation imaging (SOFI) is a fluorescenc...

过去几年中出现的超声超定位显微成像技术通过追踪超声造影剂（气体微泡）的流动，能够无创成像毛细血管水平的血管结构。然而，这些技术目前受限于低时间分辨率和长采集时间。超分辨率光学波动成像（SOFI）是一种荧光...

While current state of the art MR-PET scanners enable simultaneous MR and PET measurements, the acquired data sets are still usually reconstructed separately. We propose a new multi-modality reconstruction framework using second order Total Generalized Variation (TGV) as a dedicated multi-channel regularization functional that jointly reconstructs images from both modalities. In this way, informat...

虽然当前最先进的MR-PET扫描仪能够同时进行MR和PET测量，但获取的数据集通常仍分别重建。我们提出了一种新的多模态重建框架，使用二阶总广义变分（TGV）作为专用的多通道正则化函数，联合重建来自两种模态的图像。通过这种方式，信息...

Optoacoustic tomography is a fast developing imaging modality, combining the high contrast available from optical excitation of tissue with the high resolution and penetration depth of ultrasound detection. Light is subject to both absorption and scattering when traveling through tissue; adequate knowledge of tissue optical properties and hence the spatial fluence distribution is required to creat...

光声断层成像是一种快速发展的成像模态，它结合了组织光学激发的高对比度与超声检测的高分辨率和穿透深度。光在组织中传播时同时受到吸收和散射；需要充分了解组织光学特性，从而了解空间光通量分布，以创建组织光学特性的定量图像。

Magnetic particle imaging (MPI) is an emerging medical imaging modality which is based on the non-linear response of magnetic nanoparticles to an applied magnetic field. It is an important feature of MPI that even fast dynamic processes can be captured for 3D volumes. The high temporal resolution in turn leads to large amounts of data which have to be handled efficiently. But as the system matrix ...

磁粒子成像（MPI）是一种新兴的医学成像模态，基于磁性纳米颗粒对外加磁场的非线性响应。MPI的一个重要特点是能够捕捉三维体积的快速动态过程。高时间分辨率反过来导致必须高效处理的大量数据。但由于系统矩阵...

To accurately segment pathological and healthy lungs for reliable computer-aided disease diagnostics, a stack of chest CT scans is modeled as a sample of a spatially inhomogeneous joint 3D Markov-Gibbs random field (MGRF) of voxel-wise lung and chest CT image signals (intensities). The proposed learnable MGRF integrates two visual appearance sub-models with an adaptive lung shape submodel. The fir...

为了精确分割病理性和健康肺部以实现可靠的计算机辅助疾病诊断，将一组胸部CT扫描建模为空间不均匀的联合3D马尔可夫-吉布斯随机场（MGRF）的一个样本，该随机场基于体素级的肺部和胸部CT图像信号（强度）。提出的可学习MGRF集成了两个视觉外观子模型和一个自适应肺形状子模型。第一个...

Leakage in retinal angiography currently is a key feature for confirming the activities of lesions in the management of a wide range of retinal diseases, such as diabetic maculopathy and paediatric malarial retinopathy. This paper proposes a new saliency-based method for the detection of leakage in fluorescein angiography. A superpixel approach is firstly employed to divide the image into meaningf...

视网膜血管造影中的渗漏目前是确认多种视网膜疾病（如糖尿病性黄斑病变和儿童疟疾性视网膜病变）病灶活动性的关键特征。本文提出了一种新的基于显著性检测荧光素血管造影中渗漏的方法。首先采用超像素方法将图像分割成有意义的区域……

Accurate localization of anatomical landmarks is an important step in medical imaging, as it provides useful prior information for subsequent image analysis and acquisition methods. It is particularly useful for initialization of automatic image analysis tools (e.g. segmentation and registration) and detection of scan planes for automated image acquisition. Landmark localization has been commonly ...

准确的解剖标志定位是医学成像中的重要步骤，因为它为后续图像分析和采集方法提供了有用的先验信息。它对于自动图像分析工具（例如分割和配准）的初始化以及自动图像采集的扫描平面检测特别有用。标志定位通常...

The proper analysis of biological microscopy images is an important and complex task. Therefore, it requires verification of all steps involved in the process, including image segmentation and tracking algorithms. It is generally better to verify algorithms with computer-generated ground truth datasets, which, compared to manually annotated data, nowadays have reached high quality and can be produ...

正确分析生物显微镜图像是一项重要且复杂的任务。因此，需要验证过程中涉及的所有步骤，包括图像分割和跟踪算法。通常，使用计算机生成的真实数据集来验证算法更好，与手动标注的数据相比，如今这些数据集已达到高质量，并且可以生产...

X-ray luminescence computed tomography (XLCT), which aims to achieve molecular and functional imaging by X-rays, has recently been proposed as a new imaging modality. Combining the principles of X-ray excitation of luminescence-based probes and optical signal detection, XLCT naturally fuses functional and anatomical images and provides complementary information for a wide range of applications in ...

X射线发光断层成像（XLCT）旨在通过X射线实现分子和功能成像，近期被提出作为一种新的成像模态。结合X射线激发发光探针和光信号检测的原理，XLCT自然融合了功能和解剖图像，并为广泛的应用提供互补信息……

Pancreatic neuroendocrine tumors are abnormal growths of hormone-producing cells in the pancreas. Unlike the brain which is protected by the skull, the pancreas can be significantly deformed by its surrounding organs. Consequently, the tumor shape differences observable from images at different time points arise from both tumor growth and pancreatic motion, and tumor growth model personalization m...

胰腺神经内分泌肿瘤是胰腺中产生激素的细胞的异常生长。与受颅骨保护的大脑不同，胰腺可能被其周围器官显著变形。因此，从不同时间点的图像观察到的肿瘤形状差异既源于肿瘤生长也源于胰腺运动，并且肿瘤生长模型的个性化...

We present in vivo images of anisotropic electrical conductivity tensor distributions inside canine brains using diffusion tensor magnetic resonance electrical impedance tomography (DT-MREIT). The conductivity tensor is represented as a product of an ion mobility tensor and a scale factor of ion concentrations. Incorporating directional mobility information from water diffusion tensors, we develop...

我们展示了利用扩散张量磁共振电阻抗成像（DT-MREIT）获得的犬脑内各向异性电导率张量分布的活体图像。电导率张量表示为离子迁移率张量与离子浓度比例因子的乘积。通过结合水扩散张量的方向迁移率信息，我们开发了...

We describe an efficient gradient computation for solving inverse problems arising in magnetic resonance elastography (MRE). The algorithm can be considered as a generalized `adjoint method' based on a Lagrangian formulation. One requirement for the classic adjoint method is assurance of the self-adjoint property of the stiffness matrix in the elasticity problem. In this paper, we show this proper...

我们描述了一种有效的梯度计算方法，用于解决磁共振弹性成像（MRE）中的逆问题。该算法可视为基于拉格朗日公式的广义'伴随方法'。经典伴随方法的一个要求是确保弹性问题中刚度矩阵的自伴性质。在本文中，我们展示了这一特性...

Direct reconstruction of parametric images from raw photon counts has been shown to improve the quantitative analysis of dynamic positron emission tomography (PET) data. However it suffers from subject motion which is inevitable during the typical acquisition time of 1–2 hours. In this work we propose a framework to jointly estimate subject head motion and reconstruct the motion-corrected parametr...

从原始光子计数直接重建参数图像已被证明可以改善动态正电子发射断层扫描（PET）数据的定量分析。然而，它受到对象运动的影响，这在典型的1-2小时采集时间内是不可避免的。在这项工作中，我们提出了一个联合估计对象头部运动并重建运动校正参数的框架...

Denoising of magnetic resonance (MR) images enhances diagnostic accuracy, the quality of image manipulations such as registration and segmentation, and parameter estimation. The first objective of this paper is to introduce a new, high-performance, nonlocal filter for noise reduction in MR image sets consisting of progressively-weighted, that is, multispectral, images. This filter is a multispectr...

磁共振图像的去噪提高了诊断准确性、图像处理（如配准和分割）的质量以及参数估计。本文的第一个目标是介绍一种新的高性能非局部滤波器，用于对由渐进加权（即多光谱）图像组成的MR图像集进行降噪。该滤波器是一种多光谱...

3D Statistical Shape Models (3D-SSM) are widely used for medical image segmentation. However, during segmentation, they typically perform a very limited unidirectional search for suitable landmark positions in the image, relying on weak learners or use-case specific appearance models that solely take local image information into account. As a consequence, segmentation errors arise, and results in ...

3D统计形状模型（3D-SSM）广泛用于医学图像分割。然而，在分割过程中，它们通常执行非常有限的单向搜索来寻找图像中合适的地标位置，依赖于仅考虑局部图像信息的弱学习器或特定用例的外观模型。因此，会产生分割误差，并导致...

An increasing number of X-ray CT procedures are being conducted with drastically reduced dosage, due at least in part to advances in statistical reconstruction methods that can deal more effectively with noise than can traditional techniques. As data become photon-limited, more detailed models are necessary to deal with count rates that drop to the levels of system electronic noise. We present two...

由于统计重建方法比传统技术能更有效地处理噪声，越来越多的X射线CT检查在剂量大幅降低的情况下进行。当数据受到光子限制时，需要更详细的模型来处理降至系统电子噪声水平的计数率。我们提出了两种...

In Breast Conserving Therapy, surgeons measure the thickness of healthy tissue surrounding an excised tumor (surgical margin) via post-operative histological or visual assessment tests that, for lack of enough standardization and reliability, have recurrence rates in the order of 33%. Spectroscopic interrogation of these margins is possible during surgery, but algorithms are needed for parametric ...

在保乳治疗中，外科医生通过术后组织学或视觉评估测试测量切除肿瘤周围健康组织的厚度（手术切缘），由于缺乏足够的标准化和可靠性，复发率高达33%。手术期间可以对切缘进行光谱分析，但需要算法进行参数化...

We introduce a novel Bayesian nonparametric model that uses the concept of disease trajectories for disease subtype identification. Although our model is general, we demonstrate that by treating fractions of tissue patterns derived from medical images as compositional data, our model can be applied to study distinct progression trends between population subgroups. Specifically, we apply our algori...

我们引入了一种新颖的贝叶斯非参数模型，利用疾病轨迹的概念进行疾病亚型识别。尽管我们的模型是通用的，但通过将从医学图像中提取的组织模式分数视为成分数据，我们证明了该模型可以用于研究人群亚组之间不同的进展趋势。具体来说，我们将我们的算法应用于...

We propose a joint information approach for automatic analysis of 2D echocardiography (echo) data. The approach combines a priori images, their segmentations and patient diagnostic information within a unified framework to determine various clinical parameters, such as cardiac chamber volumes, and cardiac disease labels. The main idea behind the approach is to employ joint Independent Component An...

我们提出了一种联合信息方法，用于自动分析二维超声心动图（回声）数据。该方法在一个统一框架内结合先验图像、其分割结果和患者诊断信息，以确定各种临床参数，如心脏腔室容积和心脏疾病标签。该方法背后的主要思想是利用联合独立成分分析...

Simple and robust techniques are lacking to assess performance of flow quantification using dynamic imaging. We therefore developed a method to qualify flow quantification technologies using a physical compartment exchange phantom and image analysis tool. We validate and demonstrate utility of this method using dynamic PET and SPECT. Dynamic image sequences were acquired on two PET/CT and a cardia...

目前缺乏简单且稳健的技术来评估使用动态成像的血流定量性能。因此，我们开发了一种方法，利用物理隔室交换体模和图像分析工具来鉴定血流定量技术。我们使用动态PET和SPECT验证并展示了该方法的实用性。动态图像序列是在两台PET/CT和一台心脏...上获取的。