Advances in Binocular Stereo Vision-Driven 3D Perception and Intelligent Analysis Methods for Agriculture

Logical Operator Operator

Search Text

Search Type

add_circle_outline

remove_circle_outline

Article Menu

Font Type:

Arial Georgia Verdana

Font Size:

Aa Aa Aa

Line Spacing:

  

Column Width:

  

Background:

Open Access Review

by
Rui Ye

1,2,
Jialin Wang

1,2,
Zhihao Kong

1,2 and
Mingxiong Ou

1,2,*

1

School of Agricultural Engineering, Jiangsu University, Zhenjiang 212013, China

2

High-Tech Key Laboratory of Agricultural Equipment and Intelligence of Jiangsu Province, Jiangsu University, Zhenjiang 212013, China

Author to whom correspondence should be addressed.

Submission received: 3 July 2026 / Revised: 4 August 2026 / Accepted: 8 August 2026 / Published: 10 August 2026

Abstract

Binocular stereo vision is a low-cost and scalable 3D perception technology that shows strong potential in agricultural phenotyping and smart agriculture. By estimating depth from multi-view RGB images, it enables non-contact, high-precision sensing of crop structure, canopy morphology, growth dynamics, and livestock traits, providing essential support for digital and intelligent agricultural production. With recent advances in deep learning-based stereo matching, multimodal sensor fusion, and 3D reconstruction, its robustness and accuracy in complex field environments have been significantly improved. This paper systematically reviews recent progress in agricultural applications of binocular stereo vision, covering system architectures, traditional and deep learning-based stereo matching methods, point cloud reconstruction techniques, and emerging supervision strategies such as 3D Gaussian splatting. It further summarizes key applications, including high-throughput phenotyping, fruit localization and robotic harvesting, weed detection and precision spraying, autonomous navigation, and livestock body condition assessment, highlighting its role in multi-task agricultural perception systems. Finally, the paper discusses major challenges, including low-texture matching difficulty, occlusions in complex environments, cross-domain generalization, real-time lightweight deployment, and limited dataset availability. Future directions are outlined in foundation model-based visual perception, self- and weakly supervised learning, multimodal fusion, and edge-efficient model design, aiming to support large-scale deployment in smart agriculture.

1. Introduction

Plant phenotypes refer to the set of observable traits resulting from the interaction between genotype and environment. Accurate acquisition of crop phenotypic information is fundamental for understanding genotype–phenotype–environment interactions and serves as an essential basis for modern crop breeding, precision crop management, and intelligent agricultural decision-making [1,2,3]. Traditional phenotyping methods rely heavily on manual measurements, such as plant height estimation with rulers, leaf counting, and destructive biomass sampling [4,5]. As shown in the study by Zou et al., manual counting in mature wheat density estimation is not only inefficient, but also incurs substantially increasing labor costs as cultivation scale expands [6]. In addition, conventional approaches typically depend on discrete sampling at isolated time points, making it difficult to capture continuous growth dynamics throughout the crop life cycle, which often leads to missing information at key developmental stages.

Two-dimensional (2D) imaging technologies have partially alleviated the limitations of manual measurements. Unmanned aerial vehicles (UAVs) and satellite remote sensing platforms enable large-scale and high-throughput data acquisition [7,8]. However, due to the inherent loss of depth information in 2D projections, the full three-dimensional structure of crops cannot be faithfully represented. Issues such as leaf overlap, organ occlusion, and viewpoint distortion remain difficult to resolve [9,10,11]. For crops with complex canopy structures, such as maize, wheat, and rice, 2D images are insufficient to accurately characterize key structural traits, including leaf area index, leaf angle distribution, and canopy volume, thereby limiting their applicability in fine-grained phenotypic analysis [12,13].

In parallel with advances in sensing hardware, intelligent analysis methods have become increasingly important for converting agricultural images and three-dimensional data into biologically and operationally meaningful information. Conventional machine-learning methods rely on hand-crafted features and statistical classifiers, whereas recent deep learning approaches can automatically learn hierarchical representations for image classification, object detection, semantic and instance segmentation, stereo matching, and pose estimation. For three-dimensional agricultural data, point-cloud filtering, registration, segmentation, skeleton extraction, geometric modeling, and trait-estimation algorithms are commonly used to characterize plant and animal structures. Multimodal sensor fusion and temporal modeling further integrate complementary information from RGB images, depth sensors, LiDAR, multispectral cameras, and sequential observations. Together, these intelligent analysis methods support crop phenotyping, fruit localization, weed recognition, autonomous navigation, robotic harvesting, and livestock monitoring, thereby transforming raw sensor measurements into actionable information for smart agriculture.

From a global perspective, binocular stereo vision and intelligent analysis methods have attracted increasing attention in agricultural research and automation. Studies in North America, Europe, and East Asia have progressively moved from controlled laboratory experiments toward field-based deployment and integrated perception systems for high-throughput phenotyping, robotic harvesting, precision spraying, autonomous machinery, and livestock monitoring. Regional research priorities vary according to crop types, production systems, labor availability, equipment infrastructure, and investment in agricultural digitalization. Meanwhile, the availability of low-cost stereo cameras, deep learning models, edge-computing platforms, and multimodal sensing systems has lowered the technical barriers to agricultural 3D perception. Nevertheless, global adoption remains uneven because of limited domain-specific datasets, substantial environmental variability, differences in farm scale and infrastructure, and the absence of standardized evaluation and deployment frameworks. These conditions indicate that future progress requires not only algorithmic innovation, but also internationally diverse datasets, cross-region validation, affordable hardware integration, and closer collaboration among researchers, equipment manufacturers, and agricultural practitioners.

The emergence of three-dimensional (3D) perception technologies has provided a promising pathway to overcome these limitations. Point clouds and mesh-based representations preserve complete spatial geometric information, enabling organ-level morphometric measurements and canopy-scale structural analysis [14,15]. Wu et al. constructed maize plant point clouds using multi-view stereo vision and achieved the automatic extraction of stem–leaf segmentation, plant height, and leaf area, with a coefficient of determination (R 2) of 0.994 between estimated and manually measured plant height [16]. Zhou et al. further developed a temporal point cloud dataset covering four crops, including maize, soybean, and rice, across eleven varieties [17]. By combining skeleton extraction and leaf matching, they enabled cross-temporal organ tracking and revealed dynamic growth patterns across different crops. Compared with 2D methods, 3D data provide richer structural information and support non-destructive, continuous monitoring, significantly improving measurement consistency and repeatability [18,19].

Among existing 3D sensing technologies, binocular stereo vision occupies a unique position. Current crop 3D sensing approaches can be broadly categorized into active and passive methods [20,21,22]. Active methods, including structured light, time-of-flight (Time of Flight) cameras, and LiDAR, obtain depth information by actively emitting signals [23,24]. Although these methods offer high accuracy, they suffer from high hardware cost and limited adaptability to outdoor environments. In field-based crop phenotyping, ToF cameras are prone to generating large invalid measurement regions under strong ambient illumination, while LiDAR, despite its high precision, remains expensive. Passive reconstruction methods mainly include multi-view stereo (Multi-View Stereo), structure from motion (Structure from Motion), and binocular stereo vision, all of which rely solely on ambient light for 3D reconstruction [25,26,27].

Binocular stereo vision has become one of the most promising solutions for agricultural 3D perception due to several advantages [28,29]. First, it is highly cost-effective, requiring only two standard RGB cameras, with hardware costs far lower than LiDAR systems [30]. Zhang et al. compared fruit tree canopy volume estimation between binocular vision and LiDAR in UAV-based precision spraying and reported that the binocular system reduced cost by more than 80% while maintaining volume estimation errors within 10% [31]. Second, binocular stereo vision provides a practical and cost-effective solution for open-field applications, but its performance remains sensitive to strong and spatially variable illumination. Direct sunlight, shadows, overexposure, and specular reflections may reduce photometric consistency between stereo image pairs and consequently degrade matching accuracy. In comparison, LiDAR is generally more robust to variations in ambient illumination, although intense solar radiation may still reduce its signal-to-noise ratio and effective sensing range. Third, it provides high information density by generating dense depth maps, which are essential for fine-grained organ-level phenotyping. Yang et al. combined binocular stereo vision with supervision signals derived from 3D Gaussian splatting to generate dense depth maps for leafy vegetables, achieving an endpoint error (EPE) of only 0.91 pixels in leaf regions [32]. Fourth, the method benefits from a mature algorithmic foundation, and recent advances in deep learning have further significantly improved stereo matching accuracy [33].

However, binocular stereo vision also faces several critical limitations in agricultural applications. The first is the difficulty of stereo matching in low-texture and repetitive-texture regions, as crop leaf surfaces often lack distinctive features while repetitive structures easily lead to mismatches. Wu et al. observed in oil-tea fruit harvesting robots that traditional stereo matching methods produce large holes in leaf regions, whereas improved deep learning-based networks significantly reduce 3D localization errors [16]. Occlusion remains another major challenge. Zou et al. reported that severe occlusion in dense wheat canopies leads to substantial segmentation errors when directly applying clustering methods on reconstructed point clouds [6]. The trade-off between computational complexity and real-time performance is also prominent. Yi et al., in a study on real-time volume estimation of four fruit types, found that lightweight models achieved higher frame rates but underperformed in occlusion handling and small-object detection compared to more complex networks [34]. In addition, high-quality stereo matching datasets for agricultural scenes are extremely limited. Unlike autonomous driving, which benefits from large-scale datasets such as KITTI, existing agricultural datasets are small and lack diversity. To address this issue, Yang et al. constructed a lettuce stereo dataset with 36 scenes and more than 8000 stereo image pairs by leveraging 3D Gaussian splatting for multi-view reconstruction and dense annotation generation [32].

To address these challenges, recent studies have made substantial progress from the perspectives of algorithm design, dataset construction, and system integration. Niu et al. proposed a stereo foundation model fine-tuning strategy driven by 2D Gaussian splatting for maize field scenarios [35]. Using multi-view reconstruction, they synthesized 3625 high-fidelity stereo pairs, and through a partial parameter freezing strategy, fine-tuned only 45.7% of trainable parameters, reducing end-point error in disparity estimation to 1.85 pixels. Wang et al. introduced the Plant Stereo dataset, providing a high-quality benchmark for plant 3D reconstruction [36]. Ajagekar et al. integrated vision foundation models such as Segment Anything with RGB-D cameras to build a phenotyping system for densely planted lettuce, enabling non-destructive and continuous monitoring of plant height, area, and volume at 10-min intervals, with correlation coefficients exceeding 0.94 compared to manual measurements [37].

This review provides a comprehensive narrative overview of advances in binocular stereo vision for crop phenotyping and smart agriculture, focusing on representative studies published between 2015 and 2025. This time window was selected because deep learning achieved a major breakthrough in stereo matching around 2015. A structured literature search was conducted using databases including Web of Science, Scopus, and ScienceDirect, with keywords covering the intersection of binocular stereo vision, stereo matching, plant phenotyping, crops, and precision agriculture. Relevant studies were identified through the iterative screening of titles, abstracts, and full texts, with priority given to methodologically innovative works and those reporting quantitative performance evaluations. While we aimed for broad coverage across application domains, we acknowledge that the review is narrative in nature and does not follow formal systematic review protocols (e.g., PRISMA). Given the interdisciplinary nature of the field and the diversity of agricultural scenarios, research designs, and evaluation metrics, we consider this narrative approach most suitable for synthesizing current progress and identifying future research priorities.

Overall, binocular stereo vision has evolved from early proof-of-concept studies toward large-scale application-oriented research in agriculture, exhibiting trends toward higher accuracy, real-time processing, lightweight deployment, and improved generalization. However, a significant gap remains between algorithmic performance and large-scale field deployment, particularly in terms of robustness in complex environments, cross-domain generalization, and the lack of standardized evaluation frameworks. Therefore, a systematic review of existing studies is essential for further advancing this field.

2. Fundamentals of Stereo Vision and Adaptation to Agricultural Environments

Stereo vision reconstructs three-dimensional spatial structures from two-dimensional images by simulating the human binocular vision mechanism [38,39]. It serves as a key technical approach for object localization and point cloud reconstruction in agricultural scenes. Despite its strong potential in agricultural phenotyping, its practical deployment in complex field environments remains challenging due to high model complexity, limited dataset availability, and the absence of unified evaluation standards. This section systematically analyzes the core components of binocular stereo vision and their suitability for agricultural applications from a system-level perspective [40,41].

2.1. Construction and Computational Pipeline of Binocular Stereo Vision Systems

The construction and computational pipeline of binocular stereo vision systems consist of three tightly coupled stages: camera calibration, epipolar rectification, and outdoor system design [42,43]. These stages jointly determine the final accuracy of depth estimation.

Camera calibration is the foundational step in stereo vision systems, aiming to establish the mapping relationship between pixel coordinates and 3D world coordinates by estimating intrinsic and extrinsic camera parameters [44,45,46]. The checkerboard-based calibration method proposed by Zhang has been widely adopted in agricultural applications due to its high accuracy and robust implementation [31]. Zou et al. a