多模型融合框架用于复杂果园环境下稳健芒果检测(原标题:A Multi-Model Fusion Framework for Robust Mango Detection in Complex Orchard Environments)

1. Introduction

With the global population expected to exceed 9.7 billion by 2050, agricultural production must increase significantly to meet growing food demand [1]. In this context, precision agriculture has emerged as a key paradigm, leveraging data-driven technologies to improve productivity and resource efficiency [2]. Fruit production, particularly for high-value crops such as mango, plays a critical role in supporting agricultural economies and farmer incomes in tropical and subtropical regions [3,4]. Accurate fruit detection and localization are fundamental to applications such as yield estimation, growth monitoring, and robotic harvesting. Vision-based sensing systems deployed on unmanned ground vehicles (UGVs) or unmanned aerial vehicles (UAVs) provide an effective solution for large-scale orchard monitoring [5,6]. Field-deployed vision systems on unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) operate under challenging imaging and computational conditions. Uneven terrain, platform vibration, motion blur, changing viewpoints, dust, and rapidly varying illumination can reduce image quality and detection stability. In addition, onboard platforms generally have limited computing resources, memory, energy supply, and thermal capacity. UAV-based systems are further constrained by payload, battery endurance, flight altitude, and communication bandwidth. These practical factors require detection methods to balance accuracy, environmental robustness, and computational efficiency according to the intended application. Accordingly, the present study prioritizes detection accuracy and robustness in complex orchard scenes, while also quantitatively evaluating the computational overhead of the proposed framework. However, despite advances in deep learning-based object detection, achieving reliable fruit detection in real-world orchard environments remains a challenging problem [7].

Unlike structured industrial environments, natural orchards exhibit significant variability and environmental uncertainty, which introduces multiple challenges for visual perception. Severe occlusion and dense fruit overlapping are common, where fruits, leaves, and branches obscure each other, leading to incomplete visual information and increased missed detections [8,9,10]. Traditional non-maximum suppression (NMS) methods may further merge adjacent fruits, causing counting errors in clustered scenarios [10]. In addition, visual camouflage significantly reduces the discriminative power of color features, as many mango varieties share similar color characteristics with surrounding foliage [11,12]. Illumination variations such as backlighting, shadows, and reflections further exacerbate this issue by altering the appearance of fruits across different conditions [13,14,15,16]. Moreover, scale variation caused by changing camera distances results in the coexistence of large and small targets in a single image, making it difficult for detection models to maintain both sensitivity and localization accuracy [17,18,19]. These factors collectively impose stringent requirements on the robustness and generalization capability of detection methods in agricultural scenarios.

Early fruit detection approaches relied on handcrafted features combined with traditional machine learning methods. While effective in controlled environments, these approaches are highly sensitive to illumination changes and complex backgrounds, limiting their applicability in real-world orchards [20]. With the development of deep learning, convolutional neural networks have significantly improved detection performance. Two-stage detectors such as Faster R-CNN achieve high accuracy but are computationally expensive, making them less suitable for real-time agricultural applications. Consequently, single-stage detectors, particularly the YOLO (You Only Look Once) series, have become widely adopted due to their favorable balance between accuracy and efficiency [8]. Recent studies have focused on improving YOLO-based models by introducing attention mechanisms, feature fusion modules, and lightweight architectures, which enhance performance under specific conditions such as occlusion or small targets [21,22,23,24]. However, most existing approaches still rely on single-model architectures, which are inherently limited in handling the diverse and long-tail variations present in complex orchard environments.

To address these limitations, ensemble learning and inference-level enhancement strategies have been explored. Techniques such as test-time augmentation (TTA), weighted box fusion (WBF), and Soft-NMS have been shown to improve detection robustness by aggregating predictions from multiple views or models [25,26]. Nevertheless, these techniques are often applied independently, lacking a unified framework that systematically integrates multi-model diversity and decision-level fusion for agricultural scenarios. Therefore, there is a need for a more comprehensive detection paradigm that can effectively exploit complementary information across models while maintaining robustness under complex environmental conditions.

In this study, we propose a multi-model fusion framework for robust mango detection in complex orchard environments. TTA, WBF, Noisy-OR, and Soft-NMS are established techniques. The methodological contribution of this study lies in their task-oriented integration into a unified multi-model and multi-view inference framework, together with an aggregation strategy that avoids repeatedly counting correlated TTA predictions from the same detector. The proposed approach leverages multiple detection models with different capacities and integrates their predictions through a unified inference pipeline to improve robustness and reliability. The main contributions of this work are summarized as follows: (1) a multi-model fusion strategy is developed to exploit complementary strengths across models, improving detection robustness under complex agricultural conditions; (2) a unified inference pipeline is designed by integrating multi-scale augmentation and decision-level fusion techniques, enhancing stability under occlusion, illumination variation, and dense object overlap; and (3) experiments on two complementary datasets demonstrate the effectiveness of the proposed framework compared with single-model baselines under within-dataset evaluation protocols.

2. Materials and Methods

2.1. Overview of the Proposed Framework

As illustrated in Figure 1, the proposed mango detection framework consists of five stages: multi-scale test-time augmentation (TTA) inference, inverse transformation mapping and within-model TTA matching and fusion, cross-model matching and WBF-based coordinate fusion, model-level consensus rescoring based on Noisy-OR, and Gaussian Soft-NMS post-processing.

Figure 1. Overall architecture and workflow of the proposed framework. The arrows indicate the sequential data flow between modules, while the different colors are used to visually distinguish the five processing stages and do not encode additional quantitative information.

(1)
Multi-scale TTA inference.

The input orchard image is transformed using multi-scale resizing and horizontal flipping to generate six TTA views. These views are independently processed by three base detectors, namely YOLOv8n, YOLOv8s, and YOLOv8m, to obtain predictions under different test-time transformations.

(2)
Inverse transformation mapping and within-model TTA matching and fusion.

The predicted bounding boxes from the different TTA views are first mapped back to the coordinate system of the original image through the corresponding inverse transformations. Within each base detector, predictions of the same class are then matched according to a predefined TTA matching threshold based on IoU. Predictions considered to correspond to the same target are fused to produce a single model-specific candidate detection for subsequent cross-model fusion.

(3)
Cross-model matching and WBF-based coordinate fusion.

The model-specific candidate detections produced by YOLOv8n, YOLOv8s, and YOLOv8m are matched across base detectors according to class labels and a predefined cross-model matching threshold. Candidates considered to correspond to the same target are assigned to the same cross-model candidate cluster. After removing repeated contributions from the same detector, only candidate clusters supported by at least two distinct base detectors are retained. WBF is then applied to fuse the bounding-box coordinates of the retained candidates.

(4)
Model-level consensus rescoring based on Noisy-OR.

For each retained candidate target, the model-specific confidence scores provided by the supporting base detectors are adjusted using the corresponding detector reliability weights and Score Power. The adjusted confidence scores are then combined using Noisy-OR to obtain the final consensus-ranking score. Because the three base detectors belong to the same YOLOv8 family and are trained on the same dataset, the resulting score is interpreted as a heuristic consensus-ranking score rather than as a calibrated posterior probability.

(5)
Gaussian Soft-NMS post-processing.

Finally, Gaussian Soft-NMS is applied to the remaining candidate boxes. Instead of directly removing highly overlapping boxes, Gaussian Soft-NMS continuously decays their confidence scores according to their overlap, thereby suppressing residual redundant detections while reducing the risk of incorrectly suppressing adjacent mangoes in densely clustered scenes.

2.2. Selection of Base Models and Construction of Model Diversity

An effective ensemble detection method relies on prediction diversity and error complementarity among base models. To achieve a balance among computational complexity, feature representation capability, and deployment scalability, three YOLOv8 models with different parameter scales, namely YOLOv8n, YOLOv8s, and YOLOv8m, are selected as the base detectors in this study. These models share consistent input and output interfaces, which facilitates their integration within a unified post-processing framework. Meanwhile, they exhibit complementary characteristics in terms of feature learning capacity and target response behavior.

YOLOv8n is a lightweight model with high inference efficiency, making it suitable for rapid detection of clear targets. However, due to its relatively limited network capacity, it may suffer from missed detections in scenarios involving severe occlusion, uneven illumination, and complex background textures.

YOLOv8s is a medium-scale model that provides a favorable trade-off between inference speed and detection accuracy. It can produce relatively stable intermediate-scale feature representations and maintain good detection performance in most conventional orchard images.

YOLOv8m is a larger-capacity model with deeper feature extraction layers and stronger semantic modeling capability. Therefore, it generally exhibits higher robustness when detecting small-scale fruits, partially occluded targets, and objects under complex background interference.

The combination of models with different capacities enables the detection system to simultaneously exploit the fast response of lightweight models and the complex feature representation capability of larger models. As a result, complementary detection responses can be formed under different target scales, occlusion levels, and background interference conditions, thereby improving the overall stability of the detection system.

2.3. Test-Time Augmentation and Inverse Transformation Mapping

To alleviate the adverse effects of varying imaging distances, target scale differences, and local occlusions on detection performance, test-time augmentation (TTA) is introduced during the inference stage of the base models [27]. Let the original input image be denoted by $I$. The proposed framework contains $M = 3$ base detectors, denoted by $f_{\theta_{m}}$, where $m \in \left{\right. 1 , \ldots , M \left.\right}$ corresponds to YOLOv8n, YOLOv8s, and YOLOv8m, respectively. TTA defines a set of transformations $\mathcal{T} = \left{\right. T_{1} , T_{2} , \ldots , T_{K} \left.\right}$, where $K = 6$ denotes the number of test-time views. The transformations adopted in this study include scale resizing and horizontal flipping. Specifically, the scaling factors are set to 0.9, 1.0, and 1.1, corresponding to 90% down-scaling, the original scale, and 110% up-scaling, respectively. For each scaled image, both the original orientation and the horizontally flipped version are retained. Therefore, each base detector produces predictions from six test-time views.

For the $m$-th base detector under the $j$-th TTA view, the prediction set is denoted by

$$
\mathcal{B}{m , j} = f{\theta_{m}} \left(\right. T_{j} \left(\right. I \left.\right) \left.\right)
$$

(1)

where $m \in \left{\right. 1 , \ldots , M \left.\right}$ indexes the base detector and $j \in \left{\right. 1 , \ldots , K \left.\right}$ indexes the TTA view. The set $\mathcal{B}_{m , j}$ consists of the bounding-box coordinates and confidence scores generated by detector $m$ from the $j$-th transformed image:

$$
\mathcal{B}{m , j} = \left{\right. \left(\right. x{1 , n} , y_{1 , n} , x_{2 , n} , y_{2 , n} , s_{n} \left.\right) \left.\right}{n = 1}^{N{m , j}}
$$

(2)

where ($x_{1 , n} , y_{1 , n} , x_{2 , n} , y_{2 , n} , s_{n} \left.\right)$ denotes the coordinates of the $n$-th predicted bounding box, $s_{n}$ denotes its confidence score, and $N_{m , j}$ denotes the number of predictions generated by detector $m$ under TTA view $j$.

To fuse these predictions, their bounding-box coordinates are first mapped back to the coordinate system of the original image. Let $T_{j}^{- 1}$ denote the inverse transformation corresponding to $T_{j}$. The complete set of raw predictions generated by all base detectors and TTA views is expressed as

$$
\mathcal{P}{r a w} = \cup{m = 1}^{M} \cup_{j = 1}^{K} T_{j}^{- 1} \left(\right. \mathcal{B}_{m , j} \left.\right) , M = 3 , K = 6 .
$$

(3)

Here, $T_{j}^{- 1}$ reverses the scaling and horizontal-flipping operations applied by $T_{j}$, thereby mapping the predicted bounding-box coordinates back to the original image space. Consequently, $\mathcal{P}_{r a w}$ contains the predictions generated by all three base detectors across all six TTA views.

After inverse coordinate mapping, the predictions from the six TTA views are first matched within each base detector according to class labels and a predefined IoU threshold. Specifically, the candidate boxes are sorted in descending order of confidence. The candidate with the highest current confidence is selected as the reference box, and its IoU with the remaining candidates of the same class is calculated. Candidates whose IoU with the reference box is not lower than the predefined TTA matching threshold are considered to correspond to the same target and are grouped together. The grouped candidates are then removed from the remaining candidate set, and the procedure is repeated until all candidate boxes have been assigned. Within each group, at most one candidate from each TTA view is retained. If multiple candidates from the same TTA view satisfy the matching criterion, only the one with the highest confidence score is retained.

For candidate boxes from different TTA views that are assigned to the same group, their bounding-box coordinates are computed as a confidence-weighted average. The confidence scores of the TTA views that actually contribute matched candidates are then averaged arithmetically to obtain a single candidate detection for the corresponding target from that base detector. After this within-model TTA matching and fusion procedure, YOLOv8n, YOLOv8s, and YOLOv8m each produce their respective model-specific candidate detections, which are subsequently used as the inputs to cross-model matching and WBF-based fusion.

This strategy enhances the scale adaptability of the detector without requiring model retraining. Specifically, enlarged views improve the effective pixel representation of distant small targets, and reduced views alleviate recognition instability in close-range large targets caused by local truncation or insufficient receptive fields. Horizontal flipping further provides prediction consistency under directional perturbations, thereby supplying more comprehensive candidate bounding boxes for the subsequent fusion stage. Importantly, the six TTA views of the same base detector are used only to construct model-specific candidate detections and are not treated as independent evidence sources in the subsequent Noisy-OR fusion. Therefore, each base detector contributes at most one model-level confidence score for a given candidate target.

2.4. Spatial Consistency Fusion Based on WBF

The inputs to cross-model fusion are the model-specific candidate detections produced by YOLOv8n, YOLOv8s, and YOLOv8m after the within-model TTA matching and fusion procedure described in Section 2.3. To identify candidate detections from different base detectors that potentially correspond to the same target, all model-specific candidates are first sorted in descending order of confidence. The candidate with the highest current confidence is selected as the reference candidate, and its intersection over union (IoU) with the remaining candidates of the same class is calculated. Candidates whose IoU with the reference candidate is not lower than the predefined cross-model matching threshold are considered to correspond to the same target and are assigned to the same cross-model candidate cluster. The candidates assigned to the current cluster are then removed from the rem