Wildfire aerial thermal image segmentation using unsupervised methods: a multilayer level set approach
Tiago Garcia
A * , Ricardo Ribeiro A and Alexandre Bernardino A
+ Author Affiliations
- Author Affiliations
A Institute for Systems and Robotics, Instituto Superior Tecnico, Lisbon, Portugal.
* Correspondence to: tiagojosegarcia@tecnico.ulisboa.pt
International Journal of Wildland Fire 32(3) 435-447 https://doi.org/10.1071/WF22136
Submitted: 1 July 2022 Accepted: 17 February 2023 Published: 17 March 2023
© 2023 The Author(s) (or their employer(s)). Published by CSIRO Publishing on behalf of IAWF. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)
Abstract
Background and aims: Infrared thermal images of a propagating wildfire taken by manned or unmanned aerial vehicles can help firefighting authorities with combat planning. Segmenting these images into regions of different fire temperatures is a necessary step to measure the fire perimeter and determine the location of the fire front.
Methods: This work proposes a multilayer segmentation method based on level sets, which have the property of handling topology, making them suitable to segment images that contain scattered fire areas. The experimental results were compared using hand-drawn labels over a set of images provided by the Portuguese Air Force as ground truth. These labels were carefully drawn by the authors to ensure that they complied with the requirements indicated by the Portuguese National Authority for Emergency and Civil Protection. The proposed method was optimised to ensure contour smoothness and reliability, as well as reduce computation time.
Key results: The proposed method can surpass other common unsupervised methods in terms of intersection over union, although it has not yet been able to perform real-time segmentation.
Conclusions: Although falling out of use in relation to supervised and deep learning methods, unsupervised segmentation can still be very useful when annotated datasets are unavailable.
Keywords: airborne sensors, firefront tracking, image segmentation, level set segmentation, thermal images, thermal mapping, unsupervised segmentation, wildfire monitoring.
References
Bailon-Ruiz R, Lacroix S (2020) Wildfire remote sensing with UAVs: A review from the autonomy point of view. In ‘2020 International Conference on Unmanned Aircraft Systems (ICUAS)’, September 2020. pp. 412–420. (IEEE)Banerjee S, Bhattacharya M (2010) Segmentation of medical images using Selective Binary and Gaussian Filtering regularized level set (SBGFRLS) method. In ‘2010 3rd International Conference on Biomedical Engineering and Informatics’, October 2010. Vol. 2. pp. 541–545. (IEEE)
Bowman DMJS, Kolden CA, Abatzoglou JT, Johnston FH, van der Werf GR, Flannigan M (2020) Vegetation fires in the Anthropocene. Nature Reviews Earth & Environment 1, 500–515.
| Vegetation fires in the Anthropocene.Crossref | GoogleScholarGoogle Scholar |
Briggs WL, Henson VE, McCormick SF (2000) ‘A multigrid tutorial.’ (Society for Industrial and Applied Mathematics)
Butz RJ (2009) Traditional fire management: historical fire regimes and land use change in pastoral East Africa. International Journal of Wildland Fire 18, 442–450.
| Traditional fire management: historical fire regimes and land use change in pastoral East Africa.Crossref | GoogleScholarGoogle Scholar |
Caselles V, Catté F, Coll T, Dibos F (1993) A geometric model for active contours in image processing. Numerische Mathematik 66, 1–31.
| A geometric model for active contours in image processing.Crossref | GoogleScholarGoogle Scholar |
Caselles V, Kimmel R, Sapiro G (1997) Geodesic active contours. International Journal of Computer Vision 22, 61–79.
| Geodesic active contours.Crossref | GoogleScholarGoogle Scholar |
Chan TF, Vese LA (2001) Active contours without edges. IEEE Transactions on Image Processing 10, 266–277.
| Active contours without edges.Crossref | GoogleScholarGoogle Scholar |
Chung G, Vese LA (2005) Energy minimization based segmentation and denoising using a multilayer level set approach. In ‘Energy Minimization Methods in Computer Vision and Pattern Recognition’. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 3757 LNCS. (Eds A Rangarajan, B Vemuri, AL Yuille) pp. 439–455. (Springer Berlin Heidelberg)
Chung G, Vese LA (2009) Image segmentation using a multilayer level-set approach. Computing and Visualization in Science 12, 267–285.
| Image segmentation using a multilayer level-set approach.Crossref | GoogleScholarGoogle Scholar |
Comaniciu D, Meer P (1999) Mean shift analysis and applications. In ‘Proceedings of the seventh IEEE international conference on computer vision’, September 1999. Vol. 2. pp. 1197–1203. (IEEE)
Cruz MG, Sullivan AL, Gould JS, Sims NC, Bannister AJ, Hollis JJ, Hurley RJ (2012) Anatomy of a catastrophic wildfire: the Black Saturday Kilmore East fire in Victoria, Australia. Forest Ecology and Management 284, 269–285.
| Anatomy of a catastrophic wildfire: the Black Saturday Kilmore East fire in Victoria, Australia.Crossref | GoogleScholarGoogle Scholar |
Dupuy J-l, Fargeon H, Martin-StPaul N, Pimont F, Ruffault J, Guijarro M, Hernando C, Madrigal J, Fernandes P (2020) Climate change impact on future wildfire danger and activity in southern Europe: a review. Annals of Forest Science 77, 35
| Climate change impact on future wildfire danger and activity in southern Europe: a review.Crossref | GoogleScholarGoogle Scholar |
Fukunaga K, Hostetler L (1975) The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory 21, 32–40.
| The estimation of the gradient of a density function, with applications in pattern recognition.Crossref | GoogleScholarGoogle Scholar |
Grau V, Mewes AUJ, Alcañiz M, Kikinis R, Warfield SK (2004) Improved watershed transform for medical image segmentation using prior information. IEEE Transactions on Medical Imaging 23, 447–458.
| Improved watershed transform for medical image segmentation using prior information.Crossref | GoogleScholarGoogle Scholar |
Harkat H, Nascimento JMP, Bernardino A, Ahmed HFT (2023) Fire images classification based on a handcraft approach. Expert Systems with Applications 212, 118594
| Fire images classification based on a handcraft approach.Crossref | GoogleScholarGoogle Scholar |
He L, Osher S (2007) Solving the Chan–Vese Model by a Multiphase Level Set Algorithm Based on the Topological Derivative. In ‘Scale Space and Variational Methods in Computer Vision’. (Eds F Sgallari, A Murli, N Paragios) pp. 777–788. (Springer: Berlin Heidelberg)
Huang Y, Wu JW (2010) Infrared thermal image segmentations employing the multilayer level set method for non-destructive evaluation of layered structures. NDT & E International 43, 34–44.
| Infrared thermal image segmentations employing the multilayer level set method for non-destructive evaluation of layered structures.Crossref | GoogleScholarGoogle Scholar |
Huang Y, Lee M-G, Lin S-Y, Xiaoyu Y-I (2013) Segmenting thermal images of pervious concrete pavement temperature with employing the multilayer level set approach. In ‘Proceedings of the International Conference on Sustainable Design, Engineering, and Construction 2012, November 7–9, 2012, Fort Worth, Texas, United States’. (Eds WKO Chong, J Gong, J Chang, MK Siddiqui) ISBN (print): 9780784412688. pp. 757–764.
| Crossref |
Jones MW, Abatzoglou JT, Veraverbeke S, Andela N, Lasslop G, Forkel M, et al. (2022) Global and regional trends and drivers of fire under climate change. Reviews of Geophysics 60, e2020RG000726
| Global and regional trends and drivers of fire under climate change.Crossref | GoogleScholarGoogle Scholar |
Li C, Xu C, Gui C, Fox MD (2010) Distance regularized level set evolution and its application to image segmentation. IEEE Transactions on Image Processing 19, 3243–3254.
| Distance regularized level set evolution and its application to image segmentation.Crossref | GoogleScholarGoogle Scholar |
Liu D, Yu J (2009) Otsu method and K-means. In ‘2009 Ninth International Conference on Hybrid Intelligent Systems’, August 2009. Vol. 1. pp. 344–349. (IEEE)
Lourenço L, Nunes A, Bento-Gonçalves A, Vieira A (2012) Soil erosion after wildfires in Portugal: What happens when heavy rainfall events occur. In ‘Research on Soil Erosion’. (Eds D Godone, S Stanchi) pp. 65–88. (InTech)
Malladi R, Sethian JA (1996) Level Set and Fast Marching Methods in Image Processing and Computer Vision. In ‘Proceedings of 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland’. Vol. 1, Issue 4. pp. 489–492. (IEEE)
Moreno JC, Surya Prasath VB, Proença H, Palaniappan K (2014) Fast and globally convex multiphase active contours for brain MRI segmentation. Computer Vision and Image Understanding 125, 237–250.
| Fast and globally convex multiphase active contours for brain MRI segmentation.Crossref | GoogleScholarGoogle Scholar |
Mumford D, Shah J (1989) Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics 42, 577–685.
| Optimal approximations by piecewise smooth functions and associated variational problems.Crossref | GoogleScholarGoogle Scholar |
Ng HP, Ong SH, Foong KWC, Goh PS, Nowinski WL (2006) Medical image segmentation using k-means clustering and improved watershed algorithm. In ‘Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation’, 2006. pp. 61–65. (IEEE)
Oscher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. Journal of Computational Physics 79, 12–49.
| Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations.Crossref | GoogleScholarGoogle Scholar |
Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics 9, 62–66.
| A threshold selection method from gray-level histograms.Crossref | GoogleScholarGoogle Scholar |
Parente J, Pereira MG, Amraoui M, Tedim F (2018) Negligent and intentional fires in Portugal: Spatial distribution characterization. Science of the Total Environment 624, 424–437.
| Negligent and intentional fires in Portugal: Spatial distribution characterization.Crossref | GoogleScholarGoogle Scholar |
Perrolas G, Niknejad M, Ribeiro R, Bernardino A (2022) Scalable Fire and Smoke Segmentation from Aerial Images Using Convolutional Neural Networks and Quad-Tree Search. Sensors 22, 1701
| Scalable Fire and Smoke Segmentation from Aerial Images Using Convolutional Neural Networks and Quad-Tree Search.Crossref | GoogleScholarGoogle Scholar |
Pinto P, Silva ÁP, Viegas DX, Almeida M, Raposo J, Ribeiro LM (2022) Influence of Convectively Driven Flows in the Course of a Large Fire in Portugal: The Case of Pedrógão Grande. Atmosphere 13, 414
| Influence of Convectively Driven Flows in the Course of a Large Fire in Portugal: The Case of Pedrógão Grande.Crossref | GoogleScholarGoogle Scholar |
Santana B, Cherif EK, Bernardino A, Ribeiro R (2022) Real-Time Georeferencing of Fire Front Aerial Images Using Iterative Ray-Tracing and the Bearings-Range Extended Kalman Filter. Sensors 22, 1150
| Real-Time Georeferencing of Fire Front Aerial Images Using Iterative Ray-Tracing and the Bearings-Range Extended Kalman Filter.Crossref | GoogleScholarGoogle Scholar |
Sargento F, Ribeiro R, Cherif EK, Bernardino A (2022) Real-time Georeferencing of Fire Front Aerial Images using Structure from motion and Iterative Closest Point. In ‘Workshop on Image Analysis for Forest Environmental Monitoring’, ICPR, 2022.
Sinaga KP, Yang MS (2020) Unsupervised K-means clustering algorithm. IEEE Access 8, 80716–80727.
| Unsupervised K-means clustering algorithm.Crossref | GoogleScholarGoogle Scholar |
Turco M, Jerez S, Augusto S, Tarín-Carrasco P, Ratola N, Jiménez-Guerrero P, Trigo RM (2019) Climate drivers of the 2017 devastating fires in Portugal. Scientific Reports 9, 13886
| Climate drivers of the 2017 devastating fires in Portugal.Crossref | GoogleScholarGoogle Scholar |
Tymstra C, Stocks BJ, Cai X, Flannigan MD (2020) Wildfire management in Canada: Review, challenges and opportunities. Progress in Disaster Science 5, 100045
| Wildfire management in Canada: Review, challenges and opportunities.Crossref | GoogleScholarGoogle Scholar |
Valero MM, Rios O, Pastor E, Planas E (2018) Automated location of active fire perimeters in aerial infrared imaging using unsupervised edge detectors. International Journal of Wildland Fire 27, 241–256.
| Automated location of active fire perimeters in aerial infrared imaging using unsupervised edge detectors.Crossref | GoogleScholarGoogle Scholar |
Verde JC, Zêzere JL (2010) Assessment and validation of wildfire susceptibility and hazard in Portugal. Natural Hazards and Earth System Sciences 10, 485–497.
| Assessment and validation of wildfire susceptibility and hazard in Portugal.Crossref | GoogleScholarGoogle Scholar |
Vese LA, Chan TF (2002) A multiphase level set framework for image segmentation using the Mumford and Shah model. International Journal of Computer Vision 50, 271–293.
| A multiphase level set framework for image segmentation using the Mumford and Shah model.Crossref | GoogleScholarGoogle Scholar |
Viegas DXFC, Raposo JRN, Ribeiro CFM, Reis LCD, Abouali A, Viegas CXP (2021) On the non-monotonic behaviour of fire spread. International Journal of Wildland Fire 30, 702–719.
| On the non-monotonic behaviour of fire spread.Crossref | GoogleScholarGoogle Scholar |
Wen D, Ren A, Ji T, Flores-Parra IM, Yang X, Li M (2020) Segmentation of thermal infrared images of cucumber leaves using K-means clustering for estimating leaf wetness duration. International Journal of Agricultural and Biological Engineering 13, 161–167.
| Segmentation of thermal infrared images of cucumber leaves using K-means clustering for estimating leaf wetness duration.Crossref | GoogleScholarGoogle Scholar |
Yuan C, Zhang Y, Liu Z (2015) A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques. Canadian Journal of Forest Research 45, 783–792.
| A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques.Crossref | GoogleScholarGoogle Scholar |
Yuan C, Liu Z, Zhang Y (2017) Fire detection using infrared images for UAV-based forest fire surveillance. In ‘2017 International Conference on Unmanned Aircraft Systems’, ICUAS, 2017. pp. 567–572. (IEEE)
Zavala LM, de Celis R, Jordán A (2014) How wildfires affect soil properties. A brief review. Cuadernos de Investigación Geográfica 40, 311–331.
| How wildfires affect soil properties. A brief review.Crossref | GoogleScholarGoogle Scholar |
Zhao HK, Chan T, Merriman B, Osher S (1996) A variational level set approach to multiphase motion. Journal of Computational Physics 127, 179–195.
| A variational level set approach to multiphase motion.Crossref | GoogleScholarGoogle Scholar |
