Automatic Segmentation of Land Cover in Satellite Images
Keywords:
Satellite images, Semantic segmentation, Transfer learning, Semi-supervised learningAbstract
Semantic segmentation problems such as landcover segmentation rely on large amounts of annotated images to excel. Without such data for target regions, transfer learning methods are widely used to incorporate knowledge from other areas and domains to improve performance. In this study, we analyze the performance of landcover segmentation models trained on low-resolution images with insufficient data for the targeted region or zoom level. In order to boost performance on target data, we experiment with models trained with unsupervised, semi-supervised, and supervised transfer learning approaches, including satellite images from public datasets and other unlabeled sources.
According to experimental results, transfer learning improves segmentation performance by 3.4% MIoU (mean intersection over union) in rural regions and 12.9% MIoU in urban regions. We observed that transfer learning is more effective when two datasets share a comparable zoom level and are labeled with identical rules; otherwise, semi-supervised learning is more effective using unlabeled data. Pseudo labeling based unsupervised domain adaptation method improved building detection performance in urban cities. In addition, experiments showed that HRNet outperformed building segmentation approaches in multi-class segmentation.
References
J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, W. Liu, and B. Xiao, “Deep high-resolution representation learning for visual recognition,” CoRR, vol. abs/1908.07919, 2019. [Online]. Available: http://arxiv.org/abs/1908.07919
G. Druck, B. Settles, and A. McCallum, “Active learning by labeling features,” in Conference on Empirical Methods in Natural Language Processing, Aug. 2009, pp. 81–90.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI2015. Springer, 2015.
Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, 2018, pp. 3–11.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected crfs,” arXiv preprint arXiv:1412.7062, 2014.
L.-C. Chen, G. Papandreou, I. Kokkinos, and K. Murphy, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on PAMI, vol. 40, no. 4, pp. 834–848, 2017.
L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587, 2017.
L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder- decoder with atrous separable convolution for semantic image segmentation,” in ECCV, 2018, pp. 801–818.
B. Neupane, T. Horanont, and J. Aryal, “Deep learning-based semantic segmentation of urban features in satellite images: A review and meta analysis,” Remote Sensing, vol. 13, no. 4, p. 808, 2021.
Z. Guo, H. Shengoku, G. Wu, Q. Chen, W. Yuan, X. Shi, X. Shao, Y. Xu, and R. Shibasaki, “Semantic segmentation for urban planning maps based on u-net,” in IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2018, pp. 6187–6190.
W. Li, C. He, J. Fang, J. Zheng, H. Fu, and L. Yu, “Semantic segmentation-based building footprint extraction using very high- resolution satellite images and multi-source gis data,” Remote Sensing, vol. 11, no. 4, p. 403, 2019.
Y. Yi, Z. Zhang, W. Zhang, and C. Zhang, “Semantic segmentation of urban buildings from vhr remote sensing imagery using a deep convolutional neural network,” Remote sensing, vol. 11, no. 15, 2019.
S. Du, S. Du, B. Liu, and X. Zhang, “Incorporating deeplabv3+ and object-based image analysis for semantic segmentation of very high resolution remote sensing images,” International Journal of Digital Earth, vol. 14, no. 3, pp. 357–378, 2021.
D. Hong, Z. Han, J. Yao, L. Gao, B. Zhang, A. Plaza, and J. Chanussot, “Spectralformer: Rethinking hyperspectral image classification with transformers,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–15, 2021.
Z. Xu, W. Zhang, Z. Yang, and J. Li, “Efficient transformer for remote sensing image segmentation,” Remote Sensing, vol. 13, p. 3585, 2021.
L. Scheibenreif, J. Hanna, M. Mommert, and D. Borth, “Self-supervised vision transformers for land-cover segmentation and classification,” in CVPR, 2022, pp. 1422–1431
G. Baier, A. Deschemps, M. Schmitt, and N. Yokoya, “Building a parallel universe—image synthesis from land cover maps and auxiliary raster data,” CoRR, 2020.
E. Maggiori, Y. Tarabalka, G. Charpiat, and P. Alliez, “The inria aerial image labeling benchmark,” in IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, 2017.
I. Demir, K. Koperski, D. Lindenbaum, G. Pang, J. Huang, S. Basu, F. Hughes, D. Tuia, and R. Raskar, “Deepglobe 2018: A challenge to parse the earth through satellite images,” in Proceedings of the IEEE CVPR Workshops, 2018, pp. 172–181.
M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, vol. 312, pp. 135–153, 2018.
J. Zhang, W. Li, P. Ogunbona, and D. Xu, “Recent advances in transfer learning for cross-dataset visual recognition: A problem-oriented perspective,” ACM Computing Surveys, vol. 52, pp. 1–38, 2019.
Y. Ouali, C. Hudelot, and M. Tami, “An overview of deep semi- supervised learning,” arXiv preprint arXiv:2006.05278, 2020.
G. Wilson and D. J. Cook, “A survey of unsupervised deep domain adaptation,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 11, no. 5, pp. 1–46, 2020.
G. Hinton et al., “How to do backpropagation in a brain,” in Invited talk at the NIPS’2007 deep learning workshop, vol. 656, 2007, pp. 1–16.
Y. Jin, X. Wang, M. Long, and J. Wang, “Minimum class confusion for versatile domain adaptation,” in European Conference on Computer Vision. Springer, 2020, pp. 464–480.
M. Long, H. Zhu, J. Wang, and M. I. Jordan, “Deep transfer learning with joint adaptation networks,” in ICML, 2017.
Y. Li, N. Wang, J. Shi, J. Liu, and X. Hou, “Revisiting batch normalization for practical domain adaptation,” arXiv:1603.04779, 2016.
E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, “Randaugment: Practical automated data augmentation with a reduced search space,” in CVPRW, 2020, pp. 702–703.
Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le, “Self-training with noisy student improves imagenet classification,” in CVPR, 2020.
Lee, Dong-Hyun. “Pseudo-Label : The Simple and Efficient Semi- Supervised Learning Method for Deep Neural Networks.” (2013).
M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in ICML. PMLR, 2019, pp. 6105–6114.
Openstreetmap contributors, “Zoom levelsopenstreetmap,” 2022, [Online; accessed 6-June-2022]. [Online]. Available: https://wiki.openstreetmap.org/wiki/Zoom levels
G. Baier, A. Deschemps, M. Schmitt, and N. Yokoya, “Geonrw,” 2020.
Lee, Dong-Hyun. ”Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.” In Workshop on challenges in representation learning, ICML, vol. 3, no. 2, p. 896. 2013.
P. Yakubovskiy, “Segmentation models pytorch,” 2020. [Online]. Available: https://github.com/qubvel/segmentation models.pytorch
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 American Scientific Research Journal for Engineering, Technology, and Sciences
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who submit papers with this journal agree to the following terms.
