Lesions characterized by computed tomography (CT) scans, are arguably often elliptical objects. Therefore, radiologists often use two perpendicular diameters to measure the geometry of lesions¹, which closely represent the major and minor axises of an ellipse. Figure 1 shows an example of a lesion in a CT scan and its corresponding two-axis annotation.

Figure 1.

To aid radiologists in efficiently reviewing these CT scans, lots of object detection algorithms originally developed for analyzing nature images, e.g. Region Proposal Network (RPN), have been adopted to detect lesions in medical images. However, most of these algorithms only predict the bounding box of a lesion without fully measuring the elliptical geometry of the lesion, since objects in nature images are mostly annotated with bounding boxes. Extensions on RPN have been introduced to also predict the rotation angle of the object, but it has its own limitations. Figure 2 shows two illustrative examples.

Figure 2. Two illustrative examples of how the rotation angle may affect the overlap between two ellipses differently. The ellipse of solid line is the ground truth and the ellipse of dash line is proposed by the detection model, where both ellipses only differ in the rotation angle. (a) the major axis is significantly longer than the minor axis, (b) the major axis is slightly longer than the minor axis.

We present Gaussian Proposal Network (GPN) that learns to measure the elliptical geometry of lesions by modelling their bounding ellipses as 2D Gaussian distributions on the image plane. Figure 3 shows an illustrative comparison between GPN and RPN. Instead of predicting the rotation angle, GPN minimizes the Kullback-Leibler (KL) divergence between the proposed Gaussian distribution and the ground truth Gaussian distribution to jointly localize the lesion and predict its elliptical geometry. 

Figure 3.

We tested our algorithm on the large scale medical imaging dataset DeepLesion recently released from NIH, which has around 32,000 CT scans of lesions with various sizes and shapes. GPN proposes lesion bounding ellipses of much higher overlap with the ground truth than RPN. In terms of lesion localization score (FROC), GPN outperforms RPN by more than 10%. Figure 4 shows a few examples. 

Figure 4. Selected examples of top 3 proposed bounding ellipses from GPN (cyan) and RPN (magenta) compared to the ground truth (orange) on the test set of DeepLesion.

For more details, please refer to our paper (https://arxiv.org/abs/1902.09658). We also open sourced our algorithm at github

(https://github.com/baidu-research/GPN) for reproducing our results and further research on this topic.

Authors: Yi Li

The technical term of this annotation is called response evaluation criteria in solid tumors (RECIST) diameters. Each RECIST diameter annotation consists of two axies: the first one measures the longest diameter of the lesion and the second one measures the longest diameter perpendicular to the first axis.