Semisupervised heterogeneous ensemble for ship target discrimination in synthetic aperture radar images
-
Abstract: Ship detection using synthetic aperture radar (SAR) plays an important role in marine applications. The existing methods are capable of quickly obtaining many candidate targets, but numerous non-ship objects may be wrongly detected in complex backgrounds. These non-ship false alarms can be excluded by training discriminators, and the desired accuracy is obtained with enough verified samples. However, the reliable verification of targets in large-scene SAR images still inevitably requires manual interpretation, which is difficult and time consuming. To address this issue, a semisupervised heterogeneous ensemble ship target discrimination method based on a tri-training scheme is proposed to take advantage of the plentiful candidate targets. Specifically, various features commonly used in SAR image target discrimination are extracted, and several acknowledged classification models and their classic variants are investigated. Multiple discriminators are constructed by dividing these features into different groups and pairing them with each model. Then, the performance of all the discriminators is tested, and better discriminators are selected for implementing the semisupervised training process. These strategies enhance the diversity and reliability of the discriminators, and their heterogeneous ensemble makes more correct judgments on candidate targets, which facilitates further positive training. Experimental results demonstrate that the proposed method outperforms traditional tri-training.
-
Figure 6. Ship target discrimination results of subimage No. 3. a. Results by TT; b. results by D-TT; c. results by PS-TT; d. results by the proposed method. Correct detections are marked with red boxes, false alarms are marked with green boxes and are numbered, the missed ships are marked with blue boxes and are numbered.
Figure 7. Ship target discrimination results of subimage No. 4. a. Results by TT; b. results by D-TT; c. results by PS-TT; d. results by the proposed method. Correct detections are marked with red boxes, false alarms are marked with green boxes and are numbered, the missed ships are marked with blue boxes and are numbered.
Figure 8. Ship target discrimination results of subimage No. 5. a. Results by TT; b. results by D-TT; c. results by PS-TT; d. results by the proposed method. Correct detections are marked with red boxes, false alarms are marked with green boxes and are numbered, the missed ships are marked with blue boxes and are numbered.
Table 1. Adopted synthetic aperture radar image features for discrimination
Feature name Explanation Feature Symbol Old-Lincoln features standard deviation The standard deviation of all the pixels in a target-sized box. F1 fractal dimension The Hausdorff dimension of the spatial distribution of strong scatterers in the region of the target-sized box. F2 weighted-rank fill ratio The power of strong scatterers and normalizing by the total power of all pixels within the target-size box. F3 mass The number of pixels in the target-shaped blob. F4 diameter The length of the diagonal of the smallest rectangle that encloses the target-shaped blob. F5 square-normalized rotational inertia The second mechanical moment of the target-shaped blob around its center, normalized by the inertia of an equal mass square. F6 maximum CFAR statistic The maximum value in the CFAR image is contained within the target-shaped blob. F7 mean CFAR statistic The average value of the CFAR image is taken over the target-shaped blob. F8 percent bright CFAR statistic The percentage of pixels within the target-shaped blob that exceeds a certain CFAR value. The CFAR value is set as AvCS in this paper. F9 specific-entropy The number of pixels that exceed the threshold that is set to quantity corresponding to the 98th percentile of the surrounding clutter and normalize this value by the total number of pixels in a target-sized box. F10 contiguousness Segment each image (target-size box and CFAR image) into three separate images (shadow, background, and target) based on the amplitude of individual pixels, then computing numbers from each of these six regions of interest. F11−F16 New-Lincoln features threshold The optimal threshold for an image chip is just greater than the clutter background pixel value and smaller than the target pixel value (active pixel). F17 activation The fraction of pixels that are activated in the optimally thresholded image. F18 dispersion The weighted average distance from the centroid of a high-intensity pixel on the object, where the weights are assigned in proportion to the mass at each pixel location. F19 inflection The rate of change of the mass dispersion statistic at the optimal threshold. F20 acceleration It measures the acceleration associated with the rate of change of the mass dispersion statistic at the optimal threshold. F21 Gao features average signal-to-noise-ratio The average contrast of the target or the false alarms to the background in a candidate chip. F22 peak signal-to-noise-ratio The peak contrast of the target or the false alarms to the background in a candidate chip. F23 percentage of bright pixels The percentage of the brightest pixels with contrast higher than p% of PSNR in all the “active” pixels and p is set to 50 according to the reference. F24 Bhanu features projection Project the potential target pixels on a horizontal line (or a vertical line, the major diagonal line, the minor diagonal line) and compute the maximum distance. F25−F28 distance The minimum (or maximum, average) distance from each potential target pixel to the centroid. F29−F31 moment The horizontal (or vertical, diagonal) second-order distance from each potential target pixel to the centroid. F32−F34 Note: CFAR, constant false alarm rate; PSNR, peak signal-to-noise-ratio. Table 2. Details of three synthetic aperture radar images
Regions Acquire
timeIncidence
angle/(°)Latitude Longitude 1 2016–10–30 30.72–45.98 29.91°–31.82°N 120.36°–123.29°E 2 2020–03–01 30.79–46.08 28.07°–29.98°N 120.77°–123.66°E 3 2020–03–24 30.86–46.17 26.20°–28.12°N 119.13°–121.98°E Table 3. Details of the subimages for experimental
No. Regions Ship targets False alarms Purpose 1 Region 1 68 369 OLSs 2 Region 3 162 130 OLSs 3 Region 1 27 test 4 Region 2 98 test 5 Region 2 70 test Table 4. Performance of different models pairing with different feature groups
No. Feature groups Gaussian SVM/% Linear DA/% Quadratic LR/% Weighted KNN/% Complex DT/% Average/% 1 1_VV 96.80 97.50 96.80 96.80 97.50 97.08 2 1_VH 97.70 97.40 76.40 97.20 96.20 92.98 3 1_VV & VH 95.50 96.00 94.90 95.80 96.20 95.68 4 2_VV 76.60 78.40 82.80 77.40 81.00 79.24 5 2_VH 77.20 78.10 79.40 78.00 78.80 78.30 6 2_VV & VH 80.80 78.30 81.90 77.90 78.90 79.56 7 3_VV 88.40 87.30 95.60 96.10 95.60 92.60 8 3_VH 93.70 92.60 94.10 95.90 92.40 93.74 9 3_VV & VH 90.20 92.30 92.60 90.40 91.90 91.48 10 4_VV 97.20 97.20 96.40 97.10 97.70 97.12 11 4_VH 97.20 96.80 96.10 96.10 97.20 96.68 12 4_VV & VH 95.70 96.00 90.00 96.20 97.90 95.16 Average 90.58 90.66 89.75 91.24 91.78 Note: 1_VV means the first feature group was obtained under VV polarization, and 1_VV & VH means the assembly of the first feature group was obtained under both VV and VH polarization. The best performance is highlighted in bold. SVM, support vector machine; DA, discriminant analysis; LR, logistic regression; KNN, knearest neighbor; DT, decision tree. Table 5. The performance of five deep learning detectors on the testing set of LS-SSDD-v1.0
Model Backbone Epoch mAP/% Cascade R-CNN ResNet50-vd-SSLDv2-FPN 12 83.25 Faster R-CNN ResNet50-vd-SSLDv2-FPN 12 81.57 DCN ResNet50-vd-FPN 12 82.58 DETR ResNet50 300 58.87 PP-YOLO v2 ResNet50_vd 300 49.70 Note: In different backbone, vd means employing ResNet with version D (He et al., 2019), SSLD means employing Simple Semi-supervised Label Distillation (Cui et al., 2021), and FPN means employing Feature Pyramid Networks (Lin et al., 2017). DCN, Deformable ConvNets v2; DETR, Deformable Transformers. The best performance is highlighted in bold. Table 6. The results of the constant false alarm rate detector and Cascade R-CNN over three subimages
Methods Missed
shipsFalse
alarmsPoD/% FAR/% FOM/% K-CFAR_VV 4 183 97.95 93.85 50.53 K-CFAR_VH 1 257 99.49 131.79 42.92 Cascade R-CNN_VV 2 114 98.97 58.46 62.46 Cascade R-CNN_VH 5 64 97.44 32.82 73.36 Note: PoD indicates the probability of detection, FAR indicates the false alarm rate, and FOM indicates the figure of merit. Table 7. The discrimination results of the initial discriminators and refined discriminators
Discriminators Missed
shipsFalse
alarmsPoD/% FAR/% FOM/% Initial complex Tree 11 5 94.36 2.56 92.00 linear DA 10 12 94.87 6.15 89.37 weighted KNN 11 31 94.36 15.90 81.42 Gaussian SVM 19 7 90.26 3.59 87.13 quadratic LR 20 10 89.74 5.13 85.37 Refined complex Tree 9 5 97.44 4.62 93.14 linear DA 12 3 98.46 6.15 92.75 weighted KNN 8 4 97.95 4.10 94.09 Gaussian SVM 4 6 96.92 2.05 94.97 quadratic LR 6 3 98.46 3.08 95.52 Note: PoD indicates the probability of detection, FAR indicates the false alarm rate, and FOM indicates the figure of merit. DA, discriminant analysis; KNN, knearest neighbor; SVM, support vector machine; LR, logistic regression. Table 8. The discrimination results of the different semisupervised methods
Methods Missed ships False alarms PoD/% FAR/% FOM/% TT 10 10 94.87 5.13 90.24 D-TT 10 5 94.87 2.56 92.50 PS-TT 5 8 97.44 4.10 93.63 Proposed method 2 3 98.97 1.54 97.47 Note: PoD indicates the probability of detection, FAR indicates the false alarm rate, and FOM indicates the figure of merit. -
[1] Ai Jiaqiu, Pei Zhilin, Yao Baidong, et al. 2021. AIS data aided Rayleigh CFAR ship detection algorithm of multiple-target environment in SAR images. IEEE Transactions on Aerospace and Electronic Systems, [2] Aiello M, Vezzoli R, Gianinetto M. 2019. Object-based image analysis approach for vessel detection on optical and radar images. Journal of Applied Remote Sensing, 13(1): 014502 [3] Albukhanajer W A, Jin Yaochu, Briffa J A. 2017. Classifier ensembles for image identification using multi-objective Pareto features. Neurocomputing, 238: 316–327. doi: 10.1016/j.neucom.2017.01.067 [4] Amozegar M, Khorasani K. 2016. An ensemble of dynamic neural network identifiers for fault detection and isolation of gas turbine engines. Neural Networks, 76: 106–121. doi: 10.1016/j.neunet.2016.01.003 [5] Ao Wei, Xu Feng, Li Yongchen, et al. 2018. Detection and discrimination of ship targets in complex background from spaceborne ALOS-2 SAR images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(2): 536–550. doi: 10.1109/JSTARS.2017.2787573 [6] Baudat G, Anouar F. 2000. Generalized discriminant analysis using a kernel approach. Neural Computation, 12(10): 2385–2404. doi: 10.1162/089976600300014980 [7] Belkin M, Niyogi P. 2004. Semi-supervised learning on Riemannian manifolds. Machine Learning, 56(1): 209–239 [8] Bhanu B, Lin Yingqiang. 2003. Genetic algorithm based feature selection for target detection in SAR images. Image and Vision Computing, 21(7): 591–608. doi: 10.1016/S0262-8856(03)00057-X [9] Blum A, Mitchell T. 1998. Combining labeled and unlabeled data with co-training. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory. Madison, WI: ACM, 92–100 [10] Cai Zhaowei, Vasconcelos N. 2021. Cascade R-CNN: High quality object detection and instance segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5): 1483–1498. doi: 10.1109/TPAMI.2019.2956516 [11] Camps-Valls G, Marsheva T V B, Zhou Dengyong. 2007. Semi-supervised graph-based hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 45(10): 3044–3054. doi: 10.1109/TGRS.2007.895416 [12] Chang Yanglang, Anagaw A, Chang Lena, et al. 2019. Ship detection based on YOLOv2 for SAR imagery. Remote Sensing, 11(7): 786. doi: 10.3390/rs11070786 [13] Chen Shiyuan, Li Xiaojiang, Chi Shaoquan, et al. 2020. Ship target discrimination in SAR images based on BOW model with multiple features and spatial pyramid matching. IEEE Access, 8: 166071–166082. doi: 10.1109/ACCESS.2020.3022642 [14] Cui Cheng, Guo Ruoyu, Du Yuning, et al. 2021. Beyond self-supervision: A simple yet effective network distillation alternative to improve backbones. arXiv preprint, arXiv: 2103.05959 [15] Dasgupta S, Littman M L, McAllester D. 2001. PAC generalization bounds for co-training. In: Proceedings of the 14th International Conference on Neural Information Processing Systems. Vancouver, British Columbia: MIT Press, 375–382 [16] Deng Jia, Dong Wei, Socher R, et al. 2009. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL: IEEE, 248–255 [17] Di Martino G, Iodice A, Riccio D, et al. 2014. Filtering of azimuth ambiguity in stripmap synthetic aperture radar images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(9): 3967–3978. doi: 10.1109/JSTARS.2014.2320155 [18] Du Lan, Dai Hui, Wang Yan, et al. 2020. Target discrimination based on weakly supervised learning for high-resolution SAR images in complex scenes. IEEE Transactions on Geoscience and Remote Sensing, 58(1): 461–472. doi: 10.1109/TGRS.2019.2937175 [19] Falqueto L E, Sá J A S, Paes R L, et al. 2019. Oil rig recognition using convolutional neural network on Sentinel-1 SAR images. IEEE Geoscience and Remote Sensing Letters, 16(8): 1329–1333. doi: 10.1109/LGRS.2019.2894845 [20] Gao Gui. 2011. An improved scheme for target discrimination in high-resolution SAR images. IEEE Transactions on Geoscience and Remote Sensing, 49(1): 277–294. doi: 10.1109/TGRS.2010.2052623 [21] Gao Fei, Shi Wei, Wang Jun, et al. 2019. Enhanced feature extraction for ship detection from multi-resolution and multi-scene synthetic aperture radar (SAR) images. Remote Sensing, 11(22): 2694. doi: 10.3390/rs11222694 [22] Haider N S, Singh B K, Periyasamy R, et al. 2019. Respiratory sound based classification of chronic obstructive pulmonary disease: A risk stratification approach in machine learning paradigm. Journal of Medical Systems, 43(8): 255. doi: 10.1007/s10916-019-1388-0 [23] He Jinglu, Wang Yinghua, Liu Hongwei, et al. 2018. A novel automatic PolSAR ship detection method based on superpixel-level local information measurement. IEEE Geoscience and Remote Sensing Letters, 15(3): 384–388. doi: 10.1109/LGRS.2017.2789204 [24] He Tong, Zhang Zhi, Zhang Hang, et al. 2019. Bag of tricks for image classification with convolutional neural networks. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA: IEEE, 558–567 [25] Hua Wenqiang, Wang Shuang, Liu Hongying, et al. 2017. Semisupervised PolSAR image classification based on improved cotraining. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(11): 4971–4986. doi: 10.1109/JSTARS.2017.2728067 [26] Huang Xin, Wang Xinxin, Lv Wenyu, et al. 2021. PP-YOLOv2: A practical object detector. arXiv preprint, arXiv: 2104.10419 [27] Hwang J I, Jung H S. 2018. Automatic ship detection using the artificial neural network and support vector machine from X-band SAR satellite images. Remote Sensing, 10(11): 1799. doi: 10.3390/rs10111799 [28] Interferometric wide swath. 2020. https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/acquisition-modes/interferometric-wide-swath[2020-9-29] [29] Kang Miao, Ji Kefeng, Leng Xiangguang, et al. 2017. Contextual region-based convolutional neural network with multilayer fusion for SAR ship detection. Remote Sensing, 9(8): 860. doi: 10.3390/rs9080860 [30] Kreithen D E, Halversen S D, Owirka G J. 1993. Discriminating targets from clutter. The Lincoln Laboratory Journal, 6(1): 25–52 [31] Lang Haitao, Tao Yunhong, Niu Lihui, et al. 2020. A new scattering similarity based metric for ship detection in polarimetric synthetic aperture radar image. Acta Oceanologica Sinica, 39(5): 145–150. doi: 10.1007/s13131-020-1563-7 [32] Lang Haitao, Zhang Jie, Zhang Xi, et al. 2016. Ship classification in SAR image by joint feature and classifier selection. IEEE Geoscience and Remote Sensing Letters, 13(2): 212–216. doi: 10.1109/LGRS.2015.2506570 [33] Li Yongxu, Lai Xudong, Zhang Xi, et al. 2019. Comparative study of sea clutter distribution and ship detectors’ performance for Sentinel-1 synthetic aperture radar image. Journal of Applied Remote Sensing, 13(4): 044506 [34] Lin T Y, Dollár P, Girshick R, et al. 2017. Feature pyramid networks for object detection. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 936–944 [35] Liu Hongying, Zhu Dexiang, Yang Shuyuan, et al. 2016. Semisupervised feature extraction with neighborhood constraints for polarimetric SAR classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(7): 3001–3015. doi: 10.1109/JSTARS.2016.2532922 [36] Ma Liyong, Tang Lidan, Xie Wei, et al. 2018. Ship detection in SAR using extreme learning machine. In: International Conference on Machine Learning and Intelligent Communications. Heidelberg: Springer, 558–568 [37] Nigam K, McCallum A K, Thrun S, et al. 2000. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2): 103–134 [38] PaddlePaddle Authors. 2021. PaddlePaddle/Paddledetection: object detection and instance segmentation toolkit based on PaddlePaddle. https://github.com/PaddlePaddle/PaddleDetection[2021-10-20] [39] Pelich R, Longépé N, Mercier G, et al. 2015. AIS-based evaluation of target detectors and SAR sensors characteristics for maritime surveillance. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 8(8): 3892–3901. doi: 10.1109/JSTARS.2014.2319195 [40] Ren Shaoqing, He Kaiming, Girshick R, et al. 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 1137–1149. doi: 10.1109/TPAMI.2016.2577031 [41] Seijo-Pardo B, Porto-Díaz I, Bolón-Canedo V, et al. 2017. Ensemble feature selection: homogeneous and heterogeneous approaches. Knowledge-Based Systems, 118: 124–139. doi: 10.1016/j.knosys.2016.11.017 [42] Sentinel-1 Observation Scenario. 2020. https://sentinel.esa.int/web/sentinel/missions/sentinel-1/observation-scenario[2020-09-29] [43] Tello M, López-Martínez C, Mallorquí J J, et al. 2009. Advances in unsupervised ship detection with multiscale techniques. In: 2009 IEEE International Geoscience and Remote Sensing Symposium. Cape Town: IEEE, IV-979–IV-982 [44] Verbout S M, Weaver A L, Novak L M. 1998. New image features for discriminating targets from clutter. In: Proceedings Volume 3395, Radar Sensor Technology III. Orlando, FL: SPIE, 120–137 [45] Vespe M, Greidanus H. 2012. SAR image quality assessment and indicators for vessel and oil spill detection. IEEE Transactions on Geoscience and Remote Sensing, 50(11): 4726–4734. doi: 10.1109/TGRS.2012.2190293 [46] Wang Shuang, Guo Yanhe, Hua Wenqiang, et al. 2020. Semi-supervised PolSAR image classification based on improved tri-training with a minimum spanning tree. IEEE Transactions on Geoscience and Remote Sensing, 58(12): 8583–8597. doi: 10.1109/TGRS.2020.2988982 [47] Ward K D, Tough R J A, Watts S. 2006. Sea Clutter: Scattering the K Distribution and Radar Performance. London: The Institution of Engineering and Technology [48] Zhang Tianwen, Zhang Xiaoling, Ke Xiao, et al. 2020. LS-SSDD-v1.0: A deep learning dataset dedicated to small ship detection from large-scale Sentinel-1 SAR images. Remote Sensing, 2020,12(18): 2997 [49] Zhou Zhihua, Li Ming. 2005. Tri-training: Exploiting unlabeled data using three classifiers. IEEE Transactions on knowledge and Data Engineering, 17(11): 1529–1541. doi: 10.1109/TKDE.2005.186 [50] Zhu Xizhou, Hu Han, Lin S, et al. 2019. Deformable ConvNets V2: More deformable, better results. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA: IEEE, 9300–9308 [51] Zhu Xizhou, Su Weijie, Lu Lewei, et al. 2021. Deformable DETR: Deformable transformers for end-to-end object detection. arXiv: 2010.04159