Volume 35, Issue 5

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

图像识别取得了显著进展，这主要得益于大规模标注数据集和深度卷积神经网络（CNN）的可用性。CNN能够从足够的训练数据中学习数据驱动的、高度代表性的层次化图像特征。然而，在医学成像领域获得像ImageNet那样全面标注的数据集仍然是一个挑战...

Training a deep convolutional neural network (CNN) from scratch is difficult because it requires a large amount of labeled training data and a great deal of expertise to ensure proper convergence. A promising alternative is to fine-tune a CNN that has been pre-trained using, for instance, a large set of labeled natural images. However, the substantial differences between natural and medical images may advise against such knowledge transfer. In this paper, we seek to answer the following central question in the context of medical image analysis: Can the use of pre-trained deep CNNs with sufficient fine-tuning eliminate the need for training a deep CNN from scratch? To address this question, we considered four distinct medical imaging applications in three specialties (radiology, cardiology, and gastroenterology) involving classification, detection, and segmentation from three different imaging modalities, and investigated how the performance of deep CNNs trained from scratch compared with the pre-trained CNNs fine-tuned in a layer-wise manner. Our experiments consistently demonstrated that 1) the use of a pre-trained CNN with adequate fine-tuning outperformed or, in the worst case, performed as well as a CNN trained from scratch; 2) fine-tuned CNNs were more robust to the size of training sets than CNNs trained from scratch; 3) neither shallow tuning nor deep tuning was the optimal choice for a particular application; and 4) our layer-wise fine-tuning scheme could offer a practical way to reach the best performance for the application at hand based on the amount of available data.

从头训练深度卷积神经网络（CNN）是困难的，因为它需要大量标记的训练数据和丰富的专业知识来确保正确收敛。一个有前景的替代方案是微调一个已经使用大量标记自然图像预训练的CNN。然而，自然图像与医学图像之间的显著差异...

Among brain tumors, gliomas are the most common and aggressive, leading to a very short life expectancy in their highest grade. Thus, treatment planning is a key stage to improve the quality of life of oncological patients. Magnetic resonance imaging (MRI) is a widely used imaging technique to assess these tumors, but the large amount of data produced by MRI prevents manual segmentation in a reasonable time, limiting the use of precise quantitative measurements in the clinical practice. So, automatic and reliable segmentation methods are required; however, the large spatial and structural variability among brain tumors make automatic segmentation a challenging problem. In this paper, we propose an automatic segmentation method based on Convolutional Neural Networks (CNN), exploring small 3 $\times$ 3 kernels. The use of small kernels allows designing a deeper architecture, besides having a positive effect against overfitting, given the fewer number of weights in the network. We also investigated the use of intensity normalization as a pre-processing step, which though not common in CNN-based segmentation methods, proved together with data augmentation to be very effective for brain tumor segmentation in MRI images. Our proposal was validated in the Brain Tumor Segmentation Challenge 2013 database (BRATS 2013), obtaining simultaneously the first position for the complete, core, and enhancing regions in Dice Similarity Coefficient metric (0.88, 0.83, 0.77) for the Challenge data set. Also, it obtained the overall first position by the online evaluation platform. We also participated in the on-site BRATS 2015 Challenge using the same model, obtaining the second place, with Dice Similarity Coefficient metric of 0.78, 0.65, and 0.75 for the complete, core, and enhancing regions, respectively.

在脑肿瘤中，胶质瘤是最常见且最具侵袭性的，其最高级别会导致预期寿命极短。因此，治疗计划是改善肿瘤患者生活质量的关键阶段。磁共振成像（MRI）是一种广泛用于评估这些肿瘤的成像技术，但MRI产生的大量数据使得手动分割在合理时间内难以完成。

Automated tissue characterization is one of the most crucial components of a computer aided diagnosis (CAD) system for interstitial lung diseases (ILDs). Although much research has been conducted in this field, the problem remains challenging. Deep learning techniques have recently achieved impressive results in a variety of computer vision problems, raising expectations that they might be applied in other domains, such as medical image analysis. In this paper, we propose and evaluate a convolutional neural network (CNN), designed for the classification of ILD patterns. The proposed network consists of 5 convolutional layers with 2 $\,\times\,$ 2 kernels and LeakyReLU activations, followed by average pooling with size equal to the size of the final feature maps and three dense layers. The last dense layer has 7 outputs, equivalent to the classes considered: healthy, ground glass opacity (GGO), micronodules, consolidation, reticulation, honeycombing and a combination of GGO/reticulation. To train and evaluate the CNN, we used a dataset of 14696 image patches, derived by 120 CT scans from different scanners and hospitals. To the best of our knowledge, this is the first deep CNN designed for the specific problem. A comparative analysis proved the effectiveness of the proposed CNN against previous methods in a challenging dataset. The classification performance ( $\sim 85.5\%$ ) demonstrated the potential of CNNs in analyzing lung patterns. Future work includes, extending the CNN to three-dimensional data provided by CT volume scans and integrating the proposed method into a CAD system that aims to provide differential diagnosis for ILDs as a supportive tool for radiologists.

自动组织表征是计算机辅助诊断（CAD）系统用于间质性肺疾病（ILD）的最关键组成部分之一。尽管该领域已经进行了大量研究，但问题仍然具有挑战性。深度学习技术最近在多种计算机视觉问题中取得了令人印象深刻的成果，这提高了人们对其可能应用于此领域的期望。

Detection and classification of cell nuclei in histopathology images of cancerous tissue stained with the standard hematoxylin and eosin stain is a challenging task due to cellular heterogeneity. Deep learning approaches have been shown to produce encouraging results on histopathology images in various studies. In this paper, we propose a Spatially Constrained Convolutional Neural Network (SC-CNN)...

使用标准苏木精和伊红染色的癌组织病理学图像中细胞核的检测和分类由于细胞异质性而具有挑战性。深度学习已在多项研究中显示出对组织病理学图像产生令人鼓舞的结果。在本文中，我们提出了一种空间约束卷积神经网络（SC-CNN）...

We propose a novel Computer-Aided Detection (CAD) system for pulmonary nodules using multi-view convolutional networks (ConvNets), for which discriminative features are automatically learnt from the training data. The network is fed with nodule candidates obtained by combining three candidate detectors specifically designed for solid, subsolid, and large nodules. For each candidate, a set of 2-D p...

我们提出了一种新颖的计算机辅助检测（CAD）系统，用于肺结节的检测，该系统使用多视角卷积网络（ConvNets），自动从训练数据中学习判别特征。网络输入是通过结合三个专门为实性、亚实性和大结节设计的候选检测器获得的结节候选。对于每个候选，一组二维...

Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses ...

在MR脑图像中进行自动分割对于涉及所有年龄段图像的大规模研究中的定量分析非常重要。本文提出了一种使用卷积神经网络将MR脑图像自动分割成多个组织类别的方法。为了确保该方法获得准确的分割细节以及空间一致性，网络使用了...

Cerebral microbleeds (CMBs) are small haemorrhages nearby blood vessels. They have been recognized as important diagnostic biomarkers for many cerebrovascular diseases and cognitive dysfunctions. In current clinical routine, CMBs are manually labelled by radiologists but this procedure is laborious, time-consuming, and error prone. In this paper, we propose a novel automatic method to detect CMBs from magnetic resonance (MR) images by exploiting the 3D convolutional neural network (CNN). Compared with previous methods that employed either low-level hand-crafted descriptors or 2D CNNs, our method can take full advantage of spatial contextual information in MR volumes to extract more representative high-level features for CMBs, and hence achieve a much better detection accuracy. To further improve the detection performance while reducing the computational cost, we propose a cascaded framework under 3D CNNs for the task of CMB detection. We first exploit a 3D fully convolutional network (FCN) strategy to retrieve the candidates with high probabilities of being CMBs, and then apply a well-trained 3D CNN discrimination model to distinguish CMBs from hard mimics. Compared with traditional sliding window strategy, the proposed 3D FCN strategy can remove massive redundant computations and dramatically speed up the detection process. We constructed a large dataset with 320 volumetric MR scans and performed extensive experiments to validate the proposed method, which achieved a high sensitivity of 93.16% with an average number of 2.74 false positives per subject, outperforming previous methods using low-level descriptors or 2D CNNs by a significant margin. The proposed method, in principle, can be adapted to other biomarker detection tasks from volumetric medical data.

脑微出血是血管附近的小出血。它们已被认为是许多脑血管疾病和认知功能障碍的重要诊断生物标志物。在当前的临床常规中，脑微出血由放射科医生手动标记，但这一过程费力、耗时且容易出错。在本文中，我们提出了一种新颖的自动检测脑微出血的方法……

Automated computer-aided detection (CADe) has been an important tool in clinical practice and research. State-of-the-art methods often show high sensitivities at the cost of high false-positives (FP) per patient rates. We design a two-tiered coarse-to-fine cascade framework that first operates a candidate generation system at sensitivities $\sim 100\%$ of but at high FP levels. By leveraging exist...

自动计算机辅助检测（CADe）在临床实践和研究中已成为重要工具。最先进的方法通常以每名患者高假阳性（FP）率为代价实现高灵敏度。我们设计了一个两阶段的粗到细级联框架，首先运行一个候选生成系统，其灵敏度接近100%，但FP水平较高。通过利用现有...

The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotat...

缺乏公开可用的真实标注数据已被确定为将深度学习的最新发展应用于生物医学成像领域的主要挑战。尽管众包已实现对真实世界图像大规模数据库的标注，但其在生物医学领域的应用需要更深入的理解，从而对实际标注进行更精确的定义……

In this paper, we present a Convolutional Neural Network (CNN) regression approach to address the two major limitations of existing intensity-based 2-D/3-D registration technology: 1) slow computation and 2) small capture range. Different from optimization-based methods, which iteratively optimize the transformation parameters over a scalar-valued metric function representing the quality of the re...

在本文中，我们提出了一种卷积神经网络（CNN）回归方法，以解决现有基于强度的2D/3D配准技术的两个主要限制：1）计算速度慢和2）捕捉范围小。与基于优化的方法不同，后者通过迭代优化变换参数来最大化或最小化表示配准质量的标量度量函数……

We propose a novel segmentation approach based on deep 3D convolutional encoder networks with shortcut connections and apply it to the segmentation of multiple sclerosis (MS) lesions in magnetic resonance images. Our model is a neural network that consists of two interconnected pathways, a convolutional pathway, which learns increasingly more abstract and higher-level image features, and a deconvo...

我们提出了一种基于具有捷径连接的深度3D卷积编码器网络的新颖分割方法，并将其应用于磁共振图像中多发性硬化（MS）病灶的分割。我们的模型是一个由两个相互连接的路径组成的神经网络：一个卷积路径，学习越来越抽象和更高层次的图像特征；以及一个反卷积路径，该路径基于卷积编码器提取的多尺度特征生成精确的分割图。捷径连接将两个路径对称连接。我们在MS病灶分割任务上评估了我们的方法，并展示了相对于几种最新方法的显著改进。

Mammographic risk scoring has commonly been automated by extracting a set of handcrafted features from mammograms, and relating the responses directly or indirectly to breast cancer risk. We present a method that learns a feature hierarchy from unlabeled data. When the learned features are used as the input to a simple classifier, two different tasks can be addressed: i) breast density segmentatio...

乳腺X线摄影风险评分通常通过从乳腺X线照片中提取一组手工特征，并将响应直接或间接与乳腺癌风险相关联来实现自动化。我们提出了一种从未标记数据中学习特征层次结构的方法。当学习到的特征被用作简单分类器的输入时，可以解决两个不同的任务：i）乳腺密度分割...

Convolutional neural networks (CNNs) are deep learning network architectures that have pushed forward the state-of-the-art in a range of computer vision applications and are increasingly popular in medical image analysis. However, training of CNNs is time-consuming and challenging. In medical image analysis tasks, the majority of training examples are easy to classify and therefore contribute little to the CNN learning process. In this paper, we propose a method to improve and speed-up the CNN training for medical image analysis tasks by dynamically selecting misclassified negative samples during training. Training samples are heuristically sampled based on classification by the current status of the CNN. Weights are assigned to the training samples and informative samples are more likely to be included in the next CNN training iteration. We evaluated and compared our proposed method by training a CNN with (SeS) and without (NSeS) the selective sampling method. We focus on the detection of hemorrhages in color fundus images. A decreased training time from 170 epochs to 60 epochs with an increased performance-on par with two human experts-was achieved with areas under the receiver operating characteristics curve of 0.894 and 0.972 on two data sets. The SeS CNN statistically outperformed the NSeS CNN on an independent test set.

卷积神经网络（CNN）是深度学习网络架构，推动了计算机视觉应用中一系列最先进技术，并在医学图像分析中越来越受欢迎。然而，CNN的训练耗时且具有挑战性。在医学图像分析任务中，大多数训练样本易于分类，因此贡献很...

Numerous scientific fields rely on elaborate but partly suboptimal data processing pipelines. An example is diffusion magnetic resonance imaging (diffusion MRI), a non-invasive microstructure assessment method with a prominent application in neuroimaging. Advanced diffusion models providing accurate microstructural characterization so far have required long acquisition times and thus have been ina...

许多科学领域依赖于复杂但部分次优的数据处理流程。扩散磁共振成像（扩散MRI）就是一个例子，它是一种非侵入性的微观结构评估方法，在神经影像学中有重要应用。先进的扩散模型能够提供准确的微结构表征，但迄今为止需要较长的采集时间，因此一直无法广泛应用。

In general image recognition problems, discriminative information often lies in local image patches. For example, most human identity information exists in the image patches containing human faces. The same situation stays in medical images as well. “Bodypart identity” of a transversal slice-which bodypart the slice comes from-is often indicated by local image information, e.g., a cardiac slice an...

在一般的图像识别问题中，判别性信息通常存在于局部图像块中。例如，大多数人类身份信息存在于包含人脸图像块中。在医学图像中同样如此。横断切片的“身体部位身份”——即该切片来自哪个身体部位——通常由局部图像信息指示，例如，心脏切片…

The choice of features greatly influences the performance of a tissue classification system. Despite this, many systems are built with standard, predefined filter banks that are not optimized for that particular application. Representation learning methods such as restricted Boltzmann machines may outperform these standard filter banks because they learn a feature description directly from the tra...

特征的选择极大地影响组织分类系统的性能。尽管如此，许多系统仍使用标准的、预定义的滤波器组，而这些滤波器组并未针对特定应用进行优化。表示学习方法（如受限玻尔兹曼机）可能优于这些标准滤波器组，因为它们直接从训练数据中学习特征描述...

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be...

稳健且快速的解剖对象检测与分割解决方案支持从诊断、患者分层、治疗计划、干预到随访的整个临床工作流程。当前用于解析体积医学图像数据的最先进技术通常基于利用大型标注图像数据库的机器学习方法。两个主要挑战需要...

The papers in this special section focus on the technology and applications supported by deep learning. Deep learning is a growing trend in general data analysis and has been termed one of the 10 breakthrough technologies of 2013. Deep learning is an improvement of artificial neural networks, consisting of more layers that permit higher levels of abstraction and improved predictions from data. To ...

本特刊聚焦于深度学习支持的技术与应用。深度学习在一般数据分析中日益成为趋势，并被称为2013年十大突破性技术之一。深度学习是人工神经网络的改进，包含更多层，允许更高级别的抽象和从数据中改进预测。至...

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

要求潜在作者提交未发表的新稿件，以便纳入本次征稿通知中所述的即将举行的活动。

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

诚邀潜在作者提交新的、未发表的稿件，以便纳入本次征稿通知所述的即将举行的活动。

Presents the table of contents for this issue of the publication.

介绍本期出版物的目录。

Describes the above-named upcoming conference event. May include topics to be covered or calls for papers.

描述上述即将召开的会议活动。可能包括涵盖的主题或征稿通知。

Describes the above-named upcoming conference event. May include topics to be covered or calls for papers.

描述上述即将举行的会议活动。可能包括涵盖的主题或征文通知。

Provides a listing of the editors, board members, and current staff for this issue of the publication.

提供本期出版物的编辑、编委会成员和现任工作人员名单。