Forest fire smoke detection under complex backgrounds using TRPCA and TSVB
Xiaohu Qiang A , Guoxiong Zhou A C , Aibin Chen B , Xin Zhang A and Wenzhuo Zhang A
+ Author Affiliations
- Author Affiliations
A Research Center of Forestry Information, College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha, 410004, China.
B Wildlife Conservation and Utilisation Laboratory, College of Forestry, Central South University of Forestry and Technology, Changsha, 410004, China.
C Corresponding author. Email: zhougx01@163.com
International Journal of Wildland Fire 30(5) 329-350 https://doi.org/10.1071/WF20086
Submitted: 10 June 2020 Accepted: 25 February 2021 Published: 24 March 2021
Abstract
It is difficult to detect forest fires in complex backgrounds owing to the many interfering factors in forest fire smoke. In this paper, a novel method that combines Time Domain Robust Principal Component Analysis (TRPCA) and a Two-Stream Composed of Visual Geometry Group Network (VGG) and Bi-Long Short-Term Memory (BLSTM) (TSVB) model is proposed for forest fire smoke detection. First, features are extracted from the smoke video from the spatial stream (static) and time stream (dynamic). For the spatial stream, static features are extracted from a single-frame image of the smoke video using the VGG network. For the time stream, continuous-frame binary images of the smoke are obtained using the TRPCA algorithm. Then, the dynamic features of the smoke are extracted by VGG and BLSTM. Finally, the static and dynamic features are fused using a concatenate function to achieve forest fire smoke detection. The experimental results show that compared with the single-feature model, the proposed method effectively improves learning ability and prediction ability, and shows strong robustness against interference factors in a complex background, with accuracy of forest fire smoke detection reaching 90.6%.
Keywords: dynamic features, forest fire smoke detection, static features, TRPCA algorithm, TSVB model.
References
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5, 157–166.| Learning long-term dependencies with gradient descent is difficult.Crossref | GoogleScholarGoogle Scholar | 18267787PubMed |
Candes EJ, Li X, Ma Y, Wright J (2009) Robust principal component analysis? arXiv:0912.3599.
Cao X, Yang L, Guo X (2015) Total variation regularized RPCA for irregularly moving object detection under dynamic background. IEEE Transactions on Cybernetics 46, 1014–1027.
Cetin E (2015) Computer vision based fire detection dataset. Available at http://signal.ee.bilkent.edu.tr/VisiFire/Demo/SmokeClips/ [Verified 20 December 2015]
Chen TH, Yin YH, Huang SF, et al. (2006) The smoke detection for early fire-alarming system based on video processing. In ‘Proceedings of 2006 International Conference on Intelligent Information Hiding and Multimedia’. (Eds) pp. 427–430. (IEEE: Pasadena, CA, USA).
Chen JZ, Wang ZJ, Chen HH, et al (2016) Dynamic smoke detection using cascaded convolutional neural network for surveillance videos. Journal of University of Electronic Science and Technology of China 46, 992–996.
Diba A, Fayyaz M, Sharma V, et al (2017) Temporal 3D ConvNets: new architecture and transfer learning for video classification. arXiv:1711.08200
Du W, Wang Y, Qiao Y (2017) Recurrent spatial-temporal attention network for action recognition in videos. IEEE Transactions on Image Processing 27, 1347–1360.
Eltantawy A, Shehata MS (2015) Moving object detection from moving platforms using Lagrange multiplier. In ‘IEEE International Conference on Image Processing (ICIP)’. pp. 2586–2590.
Filonenko A, Hernandez DC, Jo KH (2018) Fast smoke detection for video surveillance using CUDA. IEEE Transactions on Industrial Informatics 14, 725–733.
| Fast smoke detection for video surveillance using CUDA.Crossref | GoogleScholarGoogle Scholar |
Fujiwara N, Terada K (2004) Extraction of a smoke region using fractal coding. In ‘IEEE International Symposium on Communications and Information Technology (ISCIT 2004)’. pp. 659–662 (IEEE: Pasadena, CA, USA).
Gao Z, Cheong LF, Wang YX (2014) Block-sparse RPCA for salient motion detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 1975–1987.
| Block-sparse RPCA for salient motion detection.Crossref | GoogleScholarGoogle Scholar | 26352629PubMed |
Gubbi J, Marusic S, Palaniswami M (2009) Smoke detection in video using wavelets and support vector machines. Fire Safety Journal 44, 1110–1115.
| Smoke detection in video using wavelets and support vector machines.Crossref | GoogleScholarGoogle Scholar |
Guillemant P, Vicente J (2001) Real-time identification of smoke images by clustering motions on a fractal curve with a temporal embedding method. Optical Engineering 40, 554
| Real-time identification of smoke images by clustering motions on a fractal curve with a temporal embedding method.Crossref | GoogleScholarGoogle Scholar |
Guo X, Tao H, Dong Y (2016) Ground moving target detection algorithm based on joint graph constraint and robust principal component analysis. Dianzi Yu Xinxi Xuebao 38, 2475–2481.
Hayashi T, Watanabe S, Toda T, Hori T, Le Roux J, Takeda K (2017) Duration-controlled LSTM for polyphonic sound event detection. IEEE/ACM Transactions on Audio, Speech, and Language Processing 25, 2059–2070.
| Duration-controlled LSTM for polyphonic sound event detection.Crossref | GoogleScholarGoogle Scholar |
He B, W Wei, B Zhang, Gao L, Song Y (2019) Improved deep convolutional neural network for human action recognition. Application Research of Computers 36, 3107–3111.
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Computation 9, 1735–1780.
| Long short-term memory.Crossref | GoogleScholarGoogle Scholar | 9377276PubMed |
Ji S, Wei X, Ming Y, et al (2013) 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 221–231.
| 3D convolutional neural networks for human action recognition.Crossref | GoogleScholarGoogle Scholar | 22392705PubMed |
Keimyung University (2012) Wildfire smoke video database. (CVPR Lab, Keimyung University) Available at https://cvpr.kmu.ac.kr/ [Verified 9 March 2021].
Kopilovic I, Vagvolgyi B, Szirányi T (2000) Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection. In ‘Proceedings of the 15th International Conference on Pattern Recognition, 2000’. pp. 714–717 (IEEE Computer Society).
Krishnan K, Prabhu N, Babu RV (2016) ARRNET: Action recognition through recurrent neural networks. In ‘Proceedings of 2016 International Conference on Signal Processing and Communications (SPCOM)’. pp. 1–5 (IEEE: Bangalore, India)
Li J, Cheng JH, Shi JY, et al. (2012) Brief introduction of back propagation (BP) neural network algorithm and its improvement. In ‘Advances in computer science and information engineering’. pp 4694–4702. (Springer: Berlin)
Liao Y, Xiong P, Min W, Min W, Lu J (2019) Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks. IEEE Access 7, 38044–38054.
| Dynamic sign language recognition based on video sequence with BLSTM-3D residual networks.Crossref | GoogleScholarGoogle Scholar |
Mittal A, Paragios N (2004) Motion-based background subtraction using adaptive kernel density estimation. In ‘Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004)’. pp. 302–309 (IEEE: Washington DC, USA)
Ng YH, Hausknecht M, Vijayanarasimhan S, et al. (2015) Beyond short snippets: deep networks for video classification. In ‘2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)’. pp. 424–428 (IEEE: Pasadena, CA, USA)
Nian C, Yang Z, Gen L, et al (2016) Overview of moving target detection based on robust principal component analysis. Journal of Image and Graphics 21, 1265–1275.
Ordóñez FJ, Roggen D (2016) Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 115–140.
| Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition.Crossref | GoogleScholarGoogle Scholar |
Reisen F, Duran SM, Flannigan M, et al (2015) Wildfire smoke and public health risk. International Journal of Wildland Fire 24, 1029–1044.
Sak H, Senior A, Beaufays F (2014) Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. In ‘Proceedings of INTERSPEECH’. pp. 338–342. (Singapore)
Simonyan K, Zisserman A (2014a) Very deep convolutional networks for large-scale image recognition. Computer Science arXiv preprint arXiv,1409.1556
Simonyan K, Zisserman A (2014b) Two-Stream convolutional networks for action recognition in videos. Advances in Neural Information Processing Systems 1, 568–576.
Sobral A, Bouwmans T, Zahzah EH (2015) Double-constrained RPCA based on saliency maps for foreground detection in automated maritime surveillance. In ‘12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)’. pp. 1–6 (IEEE: Pasadena, CA, USA)
Srivastava N, Mansimov E, Salakhutdinov R (2015) Unsupervised learning of video representations using LSTMs. In ‘Proceedings of the 32nd International Conference on Machine Learning’. pp. 843–852 (International Machine Learning Society (IMLS))
Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In ‘IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999’. pp. 246–252 (IEEE: Pasadena, CA, USA)
Tian H, Li W, Wang L, et al. (2012) A novel video-based smoke detection method using image separation. In ‘IEEE International Conference on Multimedia & Expo’. pp. 532–537 (IEEE: Pasadena, CA, USA)
Toreyin BU, Dedeoglu Y, Cetin AE (2006) Contour based smoke detection in video using wavelets. In ‘European Signal Processing Conference’. pp. 1–5 (IEEE: Pasadena, CA, USA)
Tung TX, Kim JM (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Safety Journal 46, 276–282.
| An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems.Crossref | GoogleScholarGoogle Scholar |
University of Science and Technology of China (2004) State Key Lab of Fire Science. Available at http://staff.ustc.edu.cn/~yfn/vsd.html [Verified 20 December 2015]
University of Salerno (2015) Smoke detection dataset. Available at http://mivia.unisa.it/ [Verified 20 December 2015].
Veeriah V, Zhuang N, Qi G (2015) Differential recurrent neural networks for action recognition. In ‘Proceedings of IEEE International Conference on Computer Vision’. pp. 4041–4049 (IEEE Press: Piscataway, NJ)
Verstockt S, Van Hoecke S, Beji T, et al (2013) A multi-modal video analysis approach for car park fire detection. Fire Safety Journal 57, 44–57.
| A multi-modal video analysis approach for car park fire detection.Crossref | GoogleScholarGoogle Scholar |
Vicente J, Guillemant P (2002) An image processing technique for automatically detecting forest fire. International Journal of Thermal Sciences 41, 1113–1120.
| An image processing technique for automatically detecting forest fire.Crossref | GoogleScholarGoogle Scholar |
Yin ZJ, Wan BY, Yuan FN, et al (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access: Practical Innovations, Open Solutions 5, 18429–18438.
| A deep normalization and convolutional neural network for image smoke detection.Crossref | GoogleScholarGoogle Scholar |
Yosinski J, Clune J, Bengio Y, et al (2014) How transferable are features in deep neural networks? Advances in Neural Information Processing Systems 27, 3320–3328.
Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Safety Journal 46, 132–139.
| Video-based smoke detection with histogram sequence of LBP and LBPV pyramids.Crossref | GoogleScholarGoogle Scholar |
Zhao Y, Lu W, Zheng Y, et al. (2012) An early smoke detection system based on increment of optical flow residual. In ‘IEEE International Conference on Machine Learning & Cybernetics’. pp. 1474–1479 (IEEE: Xi’an, China)
Zhou Z, Li X, Wright J, et al. (2010) Stable principal component pursuit. In ‘2010 IEEE International Symposium on Information Theory’. pp. 1518–1522 (IEEE: Pasadena, CA, USA)
Zhou X, Yang C, Yu W (2013) Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 597–610.
| Moving object detection by detecting contiguous outliers in the low-rank representation.Crossref | GoogleScholarGoogle Scholar | 22689075PubMed |
