Real-time tracking and counting of grape clusters in the field based on channel pruning with YOLOv5s

Lei Shen, Jinya Su, Runtian He, Lijie Song, Rong Huang, Yulin Fang, Yuyang Song, Baofeng Su* (Corresponding Author)

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)

Abstract

Accurate fruit counting helps grape wine industry make better logistics and decisions before harvest, and therefore produce higher quality wine. In view of poor real-time performance of the existing fruit tracking and counting methods, and a lack of effective counting methods for cluster-like fruits due to their huge shape variabilities. In this study, an end-to-end lightweight counting pipeline is developed to automate the processing of video data for real-time tracking and counting of grape clusters in field conditions. First, based on channel pruning algorithm, a more lightweight YOLOv5s cluster detection model is obtained, where number of model parameters, model size and floating-point operations (FLOPs) are reduced by 79 %, 76 %, and 58 %, respectively, and the pruned model size is only 3.4 MB. Secondly, the soft non-maximum suppression is introduced in prediction stage to improve detection performance for clusters with overlapping grapes. Test results show that mAP reaches 82.3 % and average inference time is 6.1 ms per image, which effectively reduces model parameters and complexity while ensuring detection accuracy. Finally, online multiple object tracking of clusters is implemented by integrating the detection results and SORT algorithm, where two counting modes are set by introducing counting lines. Test results on 8 videos indicated that the average counting accuracy of the proposed method reached 84.9 %, correlation coefficient with manual counting reached 0.9905, and speed of video processing reached up to 50.4 frames per second (FPS), meeting field real-time requirements. This study provides a timely technical reference for the development of orchard robots to achieve real-time automated yield estimation and accurate crop management decisions.

Original languageEnglish
Article number107662
Number of pages14
JournalComputers and Electronics in Agriculture
Volume206
Early online date1 Feb 2023
DOIs
Publication statusPublished - 1 Mar 2023

Bibliographical note

This work was supported by the Key R&D Projects of Shaanxi Province, China (Grant No. 2021NY-041) and the Key R&D Program of Ningxia Hui Autonomous Region, China (Grant No. 2021BEF02017). The authors would like to thank the anonymous reviewers for their helpful comments and suggestions.

CRediT authorship contribution statement
Lei Shen: Conceptualization, Methodology, Software, Formal analysis, Resources, Visualization, Writing – original draft. Jinya Su: Formal analysis, Writing – review & editing. Runtian He: Formal analysis, Investigation. Lijie Song: Formal analysis, Investigation. Rong Huang: Resources, Investigation. Yulin Fang: Resources. Yuyang Song: Resources, Supervision. Baofeng Su: Project administration, Supervision, Funding acquisition, Writing – review & editing.

Data Availability Statement

Data will be made available on request.

Supplementary data to this article can be found online at https://doi.org/10.1016/j.compag.2023.107662.

Keywords

  • Channel pruning
  • Grape
  • Multiple object tracking
  • Real-time counting
  • YOLOv5s

Fingerprint

Dive into the research topics of 'Real-time tracking and counting of grape clusters in the field based on channel pruning with YOLOv5s'. Together they form a unique fingerprint.

Cite this