Insect detection from imagery using YOLOv3-based adaptive feature fusion convolution network
Abderraouf Amrani
A B , Ferdous Sohel A B * , Dean Diepeveen B C , David Murray A and Michael G. K. Jones B
+ Author Affiliations
- Author Affiliations
A Information Technology, Murdoch University, Murdoch, WA 6150, Australia.
B Centre for Crop and Food Innovation, Food Futures Institute, Murdoch University, Murdoch, WA 6150, Australia.
C Department of Primary Industries and Regional Development, Western Australia, South Perth, WA 6151, Australia.
Crop & Pasture Science - https://doi.org/10.1071/CP21710
Submitted: 9 October 2021 Accepted: 29 April 2022 Published online: 7 June 2022
© 2022 The Author(s) (or their employer(s)). Published by CSIRO Publishing. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)
Abstract
Context: Insects are a major threat to crop production. They can infect, damage, and reduce agricultural yields. Accurate and fast detection of insects will help insect control. From a computer algorithm point of view, insect detection from imagery is a tiny object detection problem. Handling detection of tiny objects in large datasets is challenging due to small resolution of the insects in an image, and other nuisances such as occlusion, noise, and lack of features.
Aims: Our aim was to achieve a high-performance agricultural insect detector using an enhanced artificial intelligence machine learning technique.
Methods: We used a YOLOv3 network-based framework, which is a high performing and computationally fast object detector. We further improved the original feature pyramidal network of YOLOv3 by integrating an adaptive feature fusion module. For training the network, we first applied data augmentation techniques to regularise the dataset. Then, we trained the network using the adaptive features and optimised the hyper-parameters. Finally, we tested the proposed network on a subset dataset of the multi-class insect pest dataset Pest24, which contains 25 878 images.
Key results: We achieved an accuracy of 72.10%, which is superior to existing techniques, while achieving a fast detection rate of 63.8 images per second.
Conclusions: We compared the results with several object detection models regarding detection accuracy and processing speed. The proposed method achieved superior performance both in terms of accuracy and computational speed.
Implications: The proposed method demonstrates that machine learning networks can provide a foundation for developing real-time systems that can help better pest control to reduce crop damage.
Keywords: adaptive feature fusion, crop protection, deep learning, insect detection, object detection, pest management, small object detection, YOLO.
References
Australian Department of Agriculture, Water and the Environment (2021) Plant pests and diseases @ONLINE. Available at https://www.awe.gov.au/biosecurity-trade/pests-diseases-weeds/plantBradshaw CJA, Leroy B, Bellard C, Roiz D, Albert C, Fournier A, Barbet-Massin M, Salles J-M, Simard F, Courchamp F (2016) Massive yet grossly underestimated global costs of invasive insects. Nature Communications 7, 12986
| Massive yet grossly underestimated global costs of invasive insects.Crossref | GoogleScholarGoogle Scholar |
Dangles O, Mesías V, Crespo-Perez V, Silvain J-F (2009) Crop damage increases with pest species diversity: evidence from potato tuber moths in the tropical Andes. Journal of Applied Ecology 46, 1115–1121.
| Crop damage increases with pest species diversity: evidence from potato tuber moths in the tropical Andes.Crossref | GoogleScholarGoogle Scholar |
Gao T, Packer B, Koller D (2011) A segmentation-aware object detection model with occlusion handling. In ‘Proceedings of the IEEE conference on computer vision and pattern recognition’. pp. 1361–1368. (IEEE)
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In ‘Proceedings of the IEEE conference on computer vision and pattern recognition’. pp. 580–587. (IEEE)
Guarnieri A, Maini S, Molari G, Rondelli V (2011) Automatic trap for moth detection in integrated pest management. Bulletin of Insectology 64, 247–251.
Hasan ASMM, Sohel F, Diepeveen D, Laga H, Jones MGK (2021) A survey of deep learning techniques for weed detection from images. Computers and Electronics in Agriculture 184, 106067
| A survey of deep learning techniques for weed detection from images.Crossref | GoogleScholarGoogle Scholar |
He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In ‘Proceedings of the IEEE international conference on computer vision’. pp. 2961–2969. (IEEE)
Hinterstoisser S, Cagniart C, Ilic S, Sturm P, Navab N, Fua P, Lepetit V (2012) Gradient response maps for real-time detection of textureless objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 876–888.
| Gradient response maps for real-time detection of textureless objects.Crossref | GoogleScholarGoogle Scholar | 22442120PubMed |
Hogenhout SA, Oshima K, Ammar E-D, Kakizawa S, Kingdom HN, Namba S (2008) Phytoplasmas: bacteria that manipulate plants and insects. Molecular Plant Pathology 9, 403–423.
| Phytoplasmas: bacteria that manipulate plants and insects.Crossref | GoogleScholarGoogle Scholar | 18705857PubMed |
Kang S-H, Cho J-H, Lee S-H (2014) Identification of butterfly based on their shapes when viewed from different angles using an artificial neural network. Journal of Asia-Pacific Entomology 17, 143–149.
| Identification of butterfly based on their shapes when viewed from different angles using an artificial neural network.Crossref | GoogleScholarGoogle Scholar |
Kaya Y, Kayci L, Tekin R (2013) A computer vision system for the automatic identification of butterfly species via gabor-filter-based texture features and extreme learning machine: GF + ELM. TEM Journal 2, 13–20.
Kaya Y, Kayci L, Uyar M (2015) Automatic identification of butterfly species based on local binary patterns and artificial neural network. Applied Soft Computing 28, 132–137.
| Automatic identification of butterfly species based on local binary patterns and artificial neural network.Crossref | GoogleScholarGoogle Scholar |
Lai K, Bo L, Ren X, Fox D (2011) A large-scale hierarchical multi-view RGB-D object dataset. In ‘Proceedings of the IEEE international conference on robotics and automation’. pp. 1817–1824. (IEEE)
Li K, Zhu J, Li N (2021) Insect detection and counting based on YOLOv3 model. In ‘Proceedings of the 2021 IEEE 4th international conference on electronics technology (ICET)’. pp. 1229–1233. (IEEE)
Liu J, Wang X (2021) Plant diseases and pests detection based on deep learning: a review. Plant Methods 17, 22
| Plant diseases and pests detection based on deep learning: a review.Crossref | GoogleScholarGoogle Scholar | 33627131PubMed |
Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In ‘Proceedings of the IEEE international conference on computer vision’. pp. 2980–2988. (IEEE)
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In ‘European conference on computer vision’. pp. 21–37. (Springer)
Liu B, Hu Z, Zhao Y, Bai Y, Wang Y (2019a) Recognition of pyralidae insects using intelligent monitoring autonomous robot vehicle in natural farm scene. arXiv:1903.10827
| Recognition of pyralidae insects using intelligent monitoring autonomous robot vehicle in natural farm scene.Crossref | GoogleScholarGoogle Scholar |
Liu S, Huang D, Wang Y (2019b) Learning spatial fusion for single-shot object detection. arXiv:1911.09516
| Learning spatial fusion for single-shot object detection.Crossref | GoogleScholarGoogle Scholar |
Liu Y, Sun P, Wergeles N, Shang Y (2021) A survey and performance evaluation of deep learning methods for small object detection. Expert Systems with Applications 172, 114602
| A survey and performance evaluation of deep learning methods for small object detection.Crossref | GoogleScholarGoogle Scholar |
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110.
| Distinctive image features from scale-invariant keypoints.Crossref | GoogleScholarGoogle Scholar |
Pimentel D (2009) Pesticides and pest control. In ‘Integrated pest management: innovation-development process’. (Eds R Peshin, AK Dhawan) pp. 83–87. (Springer)
Plantinga H, Dyer CR (1990) Visibility, occlusion, and the aspect graph. International Journal of Computer Vision 5, 137–160.
| Visibility, occlusion, and the aspect graph.Crossref | GoogleScholarGoogle Scholar |
Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. arXiv:1804.02767
| YOLOv3: an incremental improvement.Crossref | GoogleScholarGoogle Scholar |
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In ‘Proceedings of the IEEE conference on computer vision and pattern recognition’. pp. 779–788. (IEEE)
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In ‘Proceedings of the advances in neural information processing systems 28’. pp. 91–99. (Curran Associates)
Savary S, Willocquet L, Pethybridge SJ, Esker P, McRoberts N, Nelson A (2019) The global burden of pathogens and pests on major food crops. Nature Ecology & Evolution 3, 430–439.
| The global burden of pathogens and pests on major food crops.Crossref | GoogleScholarGoogle Scholar |
Shammi S, Sohel F, Diepeveen D, Zander S, Jones MG (2022) A survey of image-based computational learning techniques for frost detection in plants. Information Processing in Agriculture
| A survey of image-based computational learning techniques for frost detection in plants.Crossref | GoogleScholarGoogle Scholar |
Shen Y, Zhou H, Li J, Jian F, Jayas DS (2018) Detection of stored-grain insects using deep learning. Computers and Electronics in Agriculture 145, 319–325.
| Detection of stored-grain insects using deep learning.Crossref | GoogleScholarGoogle Scholar |
Silveira M, Monteiro A (2009) Automatic recognition and measurement of butterfly eyespot patterns. Biosystems 95, 130–136.
| Automatic recognition and measurement of butterfly eyespot patterns.Crossref | GoogleScholarGoogle Scholar | 18955106PubMed |
Strauss SY, Zangerl AR (2002) Plant-insect interactions in terrestrial ecosystems. In ‘Plant-animal interactions: an evolutionary approach’. (Eds CM Herrera, O Pellmyr) pp. 77–106. (Blackwell Publishing)
Tang Z, Chen Z, Qi F, Zhang L, Chen S (2021) Pest-YOLO: deep image mining and multi-feature fusion for real-time agriculture pest detection. In ‘Proceedings of the 2021 IEEE international conference on data mining (ICDM)’. pp. 1348–1353. (IEEE)
Thenmozhi K, Srinivasulu Reddy U (2019) Crop pest classification based on deep convolutional neural network and transfer learning. Computers and Electronics in Agriculture 164, 104906
| Crop pest classification based on deep convolutional neural network and transfer learning.Crossref | GoogleScholarGoogle Scholar |
Toshev A, Taskar B, Daniilidis K (2010) Object detection via boundary structure segmentation. In ‘Proceedings of the 2010 IEEE computer society conference on computer vision and pattern recognition’. pp. 950–957. (IEEE)
Wang J, Lin C, Ji L, Liang A (2012) A new automatic identification system of insect images at the order level. Knowledge-Based Systems 33, 102–110.
| A new automatic identification system of insect images at the order level.Crossref | GoogleScholarGoogle Scholar |
Wang G, Wang K, Lin L (2019) Adaptively connected neural networks. In ‘Proceedings of the IEEE/CVF conference on computer vision and pattern recognition’. pp. 1781–1790. (IEEE)
Wang Q-J, Zhang S-Y, Dong S-F, Zhang G-C, Yang J, Li R, Wang H-Q (2020) Pest24: a large-scale very small object data set of agricultural pests for multi-target detection. Computers and Electronics in Agriculture 175, 105585
| Pest24: a large-scale very small object data set of agricultural pests for multi-target detection.Crossref | GoogleScholarGoogle Scholar |
Wang R, Liu L, Xie C, Yang P, Li R, Zhou M (2021) AgriPest: a large-scale domain-specific benchmark dataset for practical agricultural pest detection in the wild. Sensors 21, 1601
| AgriPest: a large-scale domain-specific benchmark dataset for practical agricultural pest detection in the wild.Crossref | GoogleScholarGoogle Scholar | 33668820PubMed |
Wen C, Wu D, Hu H, Pan W (2015a) Pose estimation-dependent identification method for field moth images using deep learning architecture. Biosystems Engineering 136, 117–128.
| Pose estimation-dependent identification method for field moth images using deep learning architecture.Crossref | GoogleScholarGoogle Scholar |
Wu X, Zhan C, Lai YK, Cheng MM, Yang J (2019) Ip102: A large-scale benchmark dataset for insect pest recognition. In ‘Proceedings of the IEEE/CVF conference on computer vision and pattern recognition’. pp. 8787–8796. (IEEE)
Xia D, Chen P, Wang B, Zhang J, Xie C (2018) Insect detection and classification based on an improved convolutional neural network. Sensors 18, 4169
| Insect detection and classification based on an improved convolutional neural network.Crossref | GoogleScholarGoogle Scholar |
Zhang Z, He T, Zhang H, Zhang Z, Xie J, Li M (2019) Bag of freebies for training object detection neural networks. arXiv:1902.04103
| Bag of freebies for training object detection neural networks.Crossref | GoogleScholarGoogle Scholar |
Zhao Z-Q, Zheng P, Xu S-T, Wu X (2019) Object detection with deep learning: a review. IEEE Transactions on Neural Networks and Learning Systems 30, 3212–3232.
| Object detection with deep learning: a review.Crossref | GoogleScholarGoogle Scholar |
Zheng WS, Gong S, Xiang T (2012) Quantifying and transferring contextual information in object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 762–777.
| Quantifying and transferring contextual information in object detection.Crossref | GoogleScholarGoogle Scholar | 21844619PubMed |
