Dataset creation methodology for CNN land use/cover classification: Thailand’s rural area study case
Article Sidebar
Main Article Content
Abstract
Land cover is a powerful tool and takes advantage of Convolutional Neural Network (CNN) in remote sensing image recognition. However, the existing datasets are pretty small or are not relied to the studied class where the land cover is performed. In this paper, a methodology is proposed and detailed to create dataset images to be used for land cover through CNN. This method consists in 4 steps. Firstly, large remote sensing images are collected. Then, a large amount of tiles are created using an adequate sampling method. Using a coarse model tiles are automatically labeled. Finally, dataset is cleaned from mislabeled images in order to be used in a CNN model. Rural area in Thailand is used as study case for a 4 class dataset: buildings, forest, roads and wasteland. In a first step, satellite images are cropped using overlapping process to create dataset tiles. Then, coarse model based on pixel RGB bands value is developed and by applying ratio on these RGB filters, tiles can be classified. Results show that building and wasteland class can be created with a very high precision of at least 98% demonstrating the robustness of the proposed method to quickly perform a dataset image. Forest presents a good precision with a value of 90%. On the opposite, roads class presents a low precision of 68% and therefore, this datasets needs to be manually cleaned by the users. Finally, effects of cropping and overlapping size are investigated and results show that using a different cropping size requires a new calibration of the methodology.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Journal of TCI is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence, unless otherwise stated. Please read our Policies page for more information...
References
S. A. Abou Rafee et al., “Large-Scale Hydrological Modelling of the Upper Paraná River Basin,” Water, vol. 11, no. 5, pp. 882, 2019.
A. R. Choudhury, B. Parajuli, and P. Kumar, “QuadRoad: An Ensemble of CNNs for Road Segmentation,” Procedia Comput. Sci., vol. 176, pp. 138-147, 2020.
Y. Bai, M. Feng, H. Jiang, J. Wang, Y. Zhu, and Y. Liu, “Assessing consistency of five global land cover datasets in China,” Remote Sens., vol. 6, no. 9, pp. 8739-8759, 2014.
S. Basu, S. Ganguly, S. Mukhopadhyay, R. DiBiano, M. Karki, and R. Nemani, “Deepsat: a learning framework for satellite imagery,” in 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems, Nov. 2015, pp. 1-10.
G. Castilla and G. J. Hay, “Uncertainties in land use data,” Hydrol. Earth Syst. Sci., vol. 11, no. 6, pp. 1857-1868, 2007.
K. Chen, K. Fu, M. Yan, X. Gao, X. Sun, and X. Wei, "Semantic Segmentation of Aerial Images With Shuffling Convolutional Neural Networks," in IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 173-177, 2018.
G. Cheng, J. Han, and X. Lu, "Remote Sensing Image Scene Classification: Benchmark and State of the Art," in Proceedings of the IEEE, vol. 105, no. 10, pp. 1865-1883, 2017.
C. Zhang, P. Yue, D. Tapete, B. Shangguan, M. Wang, and Z. Wu, “A multi-level context-guided classification method with object-based convolutional neural network for land cover classification using very high resolution remote sensing images,” Int. J. Appl. Earth Obs. Geoinf., vol. 88, 2020.
C. Yoo, D. Han, J. Im, and B. Bechtel, “Comparison between convolutional neural networks and random forest for local climate zone classification in mega urban areas using Landsat images,” ISPRS J. Photogramm. Remote Sens., vol. 157, pp. 155-170, 2019.
P. Chermprayong, N. Hongkarnjanakul, D. Rouquette, C. Schwob, and L. Mezeix, “Convolutional Neural Network for Thailand’s Eastern Economic Corridor (EEC) land cover classification using overlapping process on satellite images.” Remote Sens. Appl.: Soc. Environ., vol. 23, 2021.
D. Konstantinidis, V. Argyriou, T. Stathaki, and N. Grammalidis, “A modular CNN-based building detector for remote sensing images. Comput. Netw., vol. 168, 2020.
Britannica “Encyclopædia Britannica.” BRITANNICA.com. https://www.britannica.com (accessed Jul. 28, 2011).
Food and Agriculture Organization of the United Nations, “National Agro-Economic Zoning for Major Crops in Thailand (NAEZ),” FAO, Rome, Italy, 2017.
F. S.Y. Watanabe et al., “Inland water's trophic status classification based on machine learning and remote sensing data,” Remote Sens. Appl.: Soc. Environ., vol. 19, 2020.
J. A. Foley et al., “Global Consequences of Land Use,” Science, vol. 309, no. 5734, pp. 570-574, 2005.
R. Fuchs, M. Herold, P. H. Verburg, and J. G. P. W. Clevers, “A high-resolution and harmonized model approach for reconstructing and analysing historic land changes in Europe,” Biogeosciences, vol. 10, pp. 1543-1559, 2013.
K. He, X. Zhang, S. Ren, and J. Sun, “Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 2015, doi: 10.1109/TPAMI.2015.2389824
P. Helber, B. Bischke, A. Dengel, and D. Borth, "EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 7, pp. 2217-2226, 2019, doi: 10.1109/JSTARS.2019.2918242.
M. Höjer et al.,“Scenarios in selected tools for environmental systems analysis,” J. Clean. Prod., vol. 16, no. 18, pp. 1958-1970, 2008.
R. Hollmann et al., “The ESA Climate Change Initiative: Satellite Data Records for Essential Climate Variables,” Bulletin of the American Meteorological Society, vol. 94, no. 10, pp. 1541-1552, 2013.
C. Homer et al., “Completion of the 2011 National Land Cover Database for the Conterminous United States-Representing a Decade of Land Cover Change Information,” Photogramm. Eng. Remote Sens., vol. 81, no. 5, pp. 345-354, 2015.
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2261-2269.
J.-D. Sylvain, G. Drolet, and N. Brown, “Mapping dead forest cover using a deep convolutional neural network and digital aerial photography,” ISPRS J. Photogramm. Remote Sens., vol. 156, pp. 14-26, 2019.
K. Trincsi, T-T-H. Pham, and S. Turner, “Mapping mountain diversity: Ethnic minorities and land use land cover change in Vietnam's borderlands,” Land Use Policy, vol. 41, pp. 484-497, 2014.
K. D. Ngo, A. M. Lechner, and T. T. Vu, “Land cover mapping of the Mekong Delta to support natural resource management with multi-temporal Sentinel-1A synthetic aperture radar imagery,” Remote Sens. Appl.: Soc. Environ., vol. 17, 2020.
Kristo and C. C. Chua, “Cost effective window arrangement for spatial pyramid matching,” Journal Vis. Commun. Image Represent., vol. 29, pp. 79-88, 2015.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25, pp. 1097-1105, 2012.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
H. Li et al., “RSI-CB: A Large-Scale Remote Sensing Image Classification Benchmark Using Crowdsourced Data,” Sensors, vol. 20, no. 6, pp. 1594, 2020.
W. Li, C. Chen, M. Zhang, H. Li, and Q. Du, "Data Augmentation for Hyperspectral Image Classification With Deep CNN," in IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 4, pp. 593-597, 2019.
L. Ma, Y. Liu, X. Zhang, Y. Ye, G. Yin, and B. A. Johnson, “Deep learning in remote sensing applications: a meta-analysis and review,” ISPRS J. Photogramm. Remote Sens., vol. 152, pp. 166-177, 2019.
M. Panahi, N. Sadhasivam, H. R. Pourghasemi, F. Rezaie, and S. Lee, “Spatial prediction of groundwater potential mapping based on convolutional neural network (CNN) and support vector regression (SVR),” J. Hydrol., vol. 588, 2020.
D. Marmanis, J. D. Wegner, S. Galliani, K. Schindler, M. Datcu, and U. Stilla, “Semantic segmentation of aerial images with an ensemble of CNNS,” ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., vol. III-3, pp. 473–480, 2016.
J. A. Martins, V. S. Brand, M. N. Capucim, C. B. Machado, D. G. A. Piccilli, and L. D. Martins, “The impact of rainfall and land cover changes on the flow of a medium-sized river in the South of Brazil,” Energy Procedia, vol. 95, pp. 272-278, 2016.
M. Schaefer and N. X. Thinh, “Evaluation of Land Cover Change and Agricultural Protection Sites: A GIS and Remote Sensing Approach for Ho Chi Minh City, Vietnam,” Heliyon, vol. 5, no. 5, 2019.
W. B. Meyer and B. L. Turner II, Changes in land use and land cover: A global perspective. 1st ed. Cambridge University Press, 1994.
M. Wang, H. Zhang, W. Sun, S. Li, F. Wang, and G. Yang, “A Coarse-to-Fine Deep Learning Based Land Use Change Detection Method for High-Resolution Remote Sensing Images,” Remote Sens., vol. 12, no. 12, 2020.
V. Mnih and G. E. Hinton, “Learning to detect roads in high-resolution aerial images,” in Proceedings of the 11th European Conference on Computer Vision (ECCV)., 5-11 Sep. 2010.
M. V. B. de Morais, V. V. U. Guerrero, L. D. Martins, and J. A. Martins, “Dynamical Downscaling of Future Climate Change Scenarios in Urban Heat Island and Its Neighborhood in a Brazilian Subtropical Area,” in 2nd International Electronic Conference on Atmospheric Sciences, in Proceedings, vol. 1, no. 15, Jul. 2017, pp. 1-13.
R. H. Moss et al., “The next generation of scenarios for climate change research and assessment,” Nature, vol. 463, pp. 747-756, 2010.
M. U. Müller, N. Ekhtiari, R. M. Almeida, and C. Rieke, “Super-resolution of multispectral satellite images using convolutional neural networks,” ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci., vol. 1-2020, pp. 33-40, 2020.
N. D. A. Halim et al., “Spatial assessment of land use impact on air quality in mega urban regions, Malaysia,” Sustain. Cities Soc., vol. 63, 2020.
R. Patarasuk, “Road network connectivity and land-cover dynamics in Lop Buri province, Thailand,” J. Transp. Geogr., vol. 28, pp. 111-123, 2013.
R. Patarasuk and M. W. Binford, “Longitudinal analysis of the road network development and land-cover change in Lop Buri province, Thailand, 1989-2006,” Appl. Geogr., vol. 32, no. 2, pp. 228-239, 2012.
R. Shang, J. He, J. Wang, K. Xu, L. Jiao, and R. Stolkin, “Dense connection and depthwise separable convolution based CNN for polarimetric SAR image classification,” Knowl. Based Syst., vol. 194, 2020.
P. R. Emparanza, N. Hongkarnjanakul, D. Rouquette, C. Schwob, and L. Mezeix, “Land cover classification in Thailand's Eastern Economic Corridor (EEC) using convolutional neural network on satellite images.” Remote Sensing Applications: Society and Environment, vol. 20, 2020.
O. Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” Int. J. Comput. Vis., no. 115, pp. 211–252, Apr. 2015, doi: 10.1007/s11263-015-0816-y.
S. Baamonde, M. Cabana, N. Sillero, M. G. Penedo, H. Naveira, and J. Novo, “Fully automatic multi-temporal land cover classification using Sentinel-2 image data,” Procedia Comput. Sci., vol. 159, pp. 650-657, 2019, doi: 10.1016/j.procs.2019.09.220.
G. Sheng, W. Yang, T. Xu, and H. Sun, “High-resolution satellite scene classification using a sparse coding based multiple feature combination,” Int. J. Remote Sens., vol. 33, no. 8, pp. 2395-2412. Apr. 2012, doi: 10.1080/01431161.2011.608740.
D. P. Shrestha, M. Suriyaprasit, and S. Prachansri, “Assessing soil erosion in inaccessible mountainous areas in the tropics: The use of land cover and topographic parameters in a case study in Thailand,” Catena, vol. 121, pp. 40-52, Oct. 2014, doi: 10.1016/j.catena.2014.04.016.
S. Ji, Z. Chi, A. Xu, Y. Shi, and Y. Duan, “3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images,” Remote Sens., vol. 10, pp. 75, Jan. 2018, doi: 10.3390/rs10010075.
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in ICLR 2015, May. 2015, pp. 1-14.
“Statistica.” STATISTA.com. https://www.statista.com/ (accessed Dec. 17, 2021).
G. Sumbul, M. Charfuelan, B. Demir, and V. Markl, “BigEarthNet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding,” IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 2019, pp. 5901-5904, doi: 10.1109/IGARSS.2019.8900532.
S. D. Tarigan, “Land Cover Change and its Impact on Flooding Frequency of Batanghari Watershed, Jambi Province, Indonesia,” Procedia Environ. Sci., vol. 33, pp. 386-392, 2016, doi: 10.1016/j.proenv.2016.03.089
C. Szegedy et al., “Going Deeper with Convolutions,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 1-9, doi: 10.1109/CVPR.2015.7298594.
T. Talema and B. T. Hailu, “Mapping rice crop using sentinels (1 SAR and 2 MSI) images in tropical area: A case study in Fogera wereda, Ethiopia,” Remote Sens. Appl.: Soc. Environ., vol. 18, Apr. 2020, doi: 10.1016/j.rsase.2020.100290.
D. T. Nguyen, I. Iskandar, and S. Ho, “Land cover change and the CO2 stock in the Palembang City, Indonesia: A study using remote sensing, GIS technique and LUMENs,” Egypt. J. Remote. Sens. Space Sci., vol. 19, no. 2, pp. 313-321. Dec. 2016, doi: 10.1016/j.ejrs.2016.08.004.
G. C. Veerabhadrappa, S. C., T. K. Jaya Ram, and A. Haswanth, “Unsupervised Learning for Satellite Image Classification,” IOSR J. VLSI Signal Processing, vol. 4, no. 2, pp. 01-04, Jan. 2014, doi: 10.9790/4200-04240104.
P. H. Verburg, K. Neumann, and L. Nol, “Challenges in using land use and land cover data for global change studies,” Glob. Change Biol., vol. 17, pp. 974-989, 2011, doi: 10.1111/j.1365-2486.2010.02307.x.
The World Bank. “Indicators.” DATA.WORLDBANK.org. https://data.worldbank.org/indicator. (accessed Dec. 14, 2021).
G.-S. Xia, W. Yang, J. Delon, Y. Gousseau, H. Sun, and H. Maitre, “Structural high-resolution satellite image indexing,”. In ISPRS TC VII Symposium-100 Years ISPRS, Vienna, Austria, Jul. 2010, vol. 38, pp. 298-303.
G.-S. Xia et al., “AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification,” in IEEE Trans. Geosci. Remote Sens., vol. 55, no. 7, pp. 3965-3981, July 2017, doi: 10.1109/TGRS.2017.2685945.
Y. Yang and S. Newsam, “Bag-of-visual-words and spatial extensions for land-use classification,” in 18th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL GIS 2010), San Jose, CA, USA, Nov. 2010, pp. 270-279.
Y. Shendryk, Y. Rist, C. Ticehurst, and P. Thorburn, “Deep learning for multi-modal classification of cloud, shadow and land cover scenes in Planet Scope and Sentinel-2 imagery,” ISPRS J. Photogramm. Remote Sens., vol. 157, pp. 124-136, Nov. 2019, doi: 10.1016/j.isprsjprs.2019.08.018.
Z. Xu, K. Guan, N. Casler, B. Peng, and S. Wang, “3D convolutional neural network method for land cover classification using LiDAR and multi-temporal Landsat imagery,” ISPRS J. Photogramm. Remote Sens., vol. 144, pp. 423-434, Oct. 2018, doi: 10.1016/j.isprsjprs.2018.08.005.
L. Zhao, P. Tang, and L.-Z. Huo, “Feature significance-based multibag-of-visual-words model for remote sensing image scene classification,” J. Appl. Remote Sens., vol. 10, no. 3, pp. 035004. Jun. 2016, doi: 10.1117/1.JRS.10.035004.
W. Zhou, S. Newsam, C. Li, and Z. Shao, “PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval,”. ISPRS J. Photogramm. Remote Sens., vol. 145, Part A, pp. 197-209. Nov. 2018, doi: 10.1016/j.isprsjprs.2018.01.004.
B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning Transferable Architectures for Scalable Image Recognition,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 8697-8710, doi: 10.1109/CVPR.2018.00907.
Q. Zou, L. Ni, T. Zhang, and Q. Wang, “Deep Learning Based Feature Selection for Remote Sensing Scene Classification,” in IEEE Geosci. Remote Sens. Lett., vol. 12, no. 11, pp. 2321-2325, Nov. 2015, doi: 10.1109/LGRS.2015.2475299.