Quantifying Visual Symptom Severity in Plants Using Deep Learning(MDPI Remote Sens. 2026-08-17)

1. Introduction

Pests and diseases are a severe threat to food security, trade and livelihoods globally. Every year, up to 40 percent of crops are lost due to plant pests and diseases [1]. Therefore, plant pest and disease protection has become an important research area, due to its high correlation with food security, climate change, and environmental sustainability [2]. Reliable, precise and accurate estimates of disease severity are important for predicting yield loss, monitoring and forecasting epidemics, and effective disease resistance [3]. Particularly, the estimation of early symptoms for plant disease severity is one of major milestones to improve agricultural production [4]. Visual assessment of disease severity has long been used in plant pathology. Classical ordinal scales such as the Horsfall–Barratt scale categorize severity based on the proportion of symptomatic areas relative to the total leaf area [5]. In parallel, especially for virus diseases, symptom severity is also often categorized into qualitative classes such as, asymptomatic, mild, moderate, severe based on overall visual symptom expression [6,7]. Although these approaches are simple and widely applicable, they are inherently subjective and sensitive to scale design and evaluator experience. To enable automation of these tasks, numerous approaches have been explored, ranging from rule-based image recognition methods to machine learning-based techniques, such as support vector machines, as well as studies integrating hyperspectral imaging. A comprehensive overview of these approaches is provided in the review by Faye et al. Recently, with advances in deep learning-based image recognition techniques, several studies have sought to automate the quantitative assessment of symptom severity by calculating the proportion of symptomatic areas within images.

Thomas et al. applied a support vector machine (SVM) to images captured under outdoor conditions to detect symptomatic regions and subsequently evaluated disease severity based on the proportion of symptomatic areas [8]. Garg et al. proposed a multi-stage segmentation approach for UAV images, where background regions were first removed, and this was then followed by segmentation of symptomatic areas [9]. Zhao et al. utilized a deep learning-based segmentation model to extract lesion regions on leaves and quantified disease severity based on the proportion of lesion area relative to the total leaf area [10]. Palma et al. demonstrated that, when symptomatic regions are clearly distinguishable in leaf images, segmentation of disease-affected areas can be achieved using a training-free method [11]. In contrast, Bedi et al. addressed leaf images acquired under both indoor and outdoor conditions by first training an autoencoder to extract feature vectors, classifying images into healthy and diseased classes using a feature-based classifier, and finally segmenting symptomatic regions only in images classified as diseased to estimate disease severity based on the symptomatic area [12].

Moreover, Palma et al. demonstrated that, when symptomatic regions are clearly distinguishable in leaf images, disease-affected areas can be segmented and quantified using a training-free approach [11]. However, these approaches are applicable only to disease symptoms that involve clearly visible necrotic lesions. In addition, attempts have been made to classify symptom severity into discrete categories using plant leaf images. Xiang et al. treated disease severity estimation as a classification task by categorizing leaf images into discrete severity levels [4]. Yang et al. proposed a unified classification approach that simultaneously identified plant type, disease, and severity [13]. In addition to individual methodological studies, several comprehensive reviews have summarized plant disease severity assessment from a deep learning perspective. Shi et al. reviewed plant disease severity assessment methods based on convolutional neural networks (CNNs), with a focus on deep learning-driven approaches [14]. Faye et al. provided a broader overview of plant disease severity assessment, covering rule-based image analysis methods, conventional machine learning techniques, hyperspectral imaging-based approaches, and recent advances in deep learning [2]. Furthermore, Demilie reviewed machine learning and deep learning techniques for plant disease detection and diagnosis, including methods for assessing disease severity [15].

However, the implementation of these approaches requires the construction of datasets in which symptom severity is predefined and categorized, followed by supervised model training based on such labeled data. Moreover, many image datasets used in previous disease severity estimation studies were acquired under controlled conditions, such as indoor environments with uniform backgrounds and lighting, thereby limiting their applicability to outdoor settings. Furthermore, most of these studies have focused primarily on severe disease symptoms that are easily recognizable even by non-experts.

Potatoes are the world’s most important root and tuber crop. One seed potato can produce approximately 10 potato tubers, and disease in one seed potato can be propagated exponentially through seed potato production [16]. To promote stable potato production, quality seed potatoes that are healthy and disease-free should be supplied. Experienced laborers are required in the fields for visual inspection and rogueing abnormal plants during seed potato production. Rogueing is an essential practice in seed potato production because it prevents diseased plants and tubers from entering subsequent generations of seed potato multiplication. In contrast, rogueing is generally not performed in commercial potato production fields, as diseased potatoes primarily reduce yield but do not pose a risk to human health. Therefore, efficient detection of abnormal plants is particularly important in seed potato production fields, which were the target application of this study. Our previous studies developed an automated abnormal potato plant detection system using deep learning models [17,18]. The objective of these studies is to reduce the labor required for abnormal plant rogueing tasks conducted by experts. To achieve this goal, it is necessary to detect subtle disease symptoms that can be recognized only through expert experience. Early detection and rogueing of abnormal plants at the stage of subtle symptoms is essential, as failure to do so can lead to the spread of infection to neighboring plants. Therefore, existing visual symptom severity quantification methods that focus solely on clearly severe symptoms cannot be applied to this problem. Even if a custom dataset were to be constructed and existing deep learning-based classification approaches using qualitative classes—such as asymptomatic, mild, moderate, and severe—were adopted, it would still be necessary to prepare labeled training data for each visual symptom severity category. Moreover, since the manifestation of disease symptoms varies among varieties, a large amount of labeled data would have to be prepared for each variety, rendering this approach impractical. To address these challenges, we propose two simple methods for quantifying mild visual symptoms that can be recognized only by experts, using a dataset consisting solely of clearly healthy images and clearly symptomatic images: a class-probability-based method and a classification-accuracy-based method. Although the proposed framework enables quantitative assessment of visual symptom severity at the image-group level, its direct application to individual images remains challenging. This is because models trained with cross-entropy loss are optimized to assign high confidence to the predicted class, often resulting in overconfident and poorly calibrated probabilities for mildly symptomatic images. Such behavior has been widely recognized in deep learning-based classification [19]. Consequently, mild symptoms tend to be assigned extreme probability values close to 0 or 1, which limits their usefulness for visual symptom severity quantification at the individual image level. To address this issue, we investigate the effectiveness of probability calibration and regularization techniques, specifically temperature scaling [19] and label smoothing [20], which has also been reported to improve probability calibration.

The dataset used in this study is described in Section 2. The two proposed methods are described in Section 3. The results of the proposed methods are shown in Section 4. Discussion and conclusions are presented in Section 5 and Section 6, respectively.

2. Materials

We acquired images to prepare the training, validation, and test sets. During the daytime in the Sapporo and Obihiro areas (Hokkaido, Japan), shooting was performed in June–July 2021 and 2022 (Figure 1).

Figure 1. Research field locations. This map was generated from JAXA ALOS AW3D30.

We collected RGB images using portable cameras (GoPro, GoPro, Inc., San Mateo, CA, USA). To enable efficient image capture of multiple ridges simultaneously and to mitigate issues caused by direct sunlight—such as leaf shadows cast onto neighboring leaves and image overexposure due to specular reflection—we developed and employed a custom-designed imaging cart, as shown in Figure 2. The camera was mounted in a nadir-view configuration at a distance of approximately 60 cm above the plant canopy. Images were acquired at a resolution of 4000 × 3000 pixels, corresponding to a spatial resolution of approximately 0.3 mm pixel−1. The imaging cart was operated at a speed of approximately 2 km h−1. Images were acquired between 10:00 and 15:00 under sunny or cloudy weather conditions. Training data for the deep learning model should include a large volume of images of healthy and abnormal plants. However, the viral disease acceptance criteria in Japanese seed potato production can be as low as 0.1–0.3% [21]. Thus, based on these criteria, very few abnormal plants are present in the seed potato production field. Therefore, acquisition of abnormal plant images was an issue. To address this problem, we planted healthy seed potatoes, potato virus Y (PVY)-infected seed potatoes, and seed potatoes immersed in a suspension of Pectobacterium brasiliense (approximately 1 × 10 6 cells/mL), the causal agent of blackleg disease, in Sapporo and Obihiro. PVY belongs to the genus Potyvirus. Three potato cultivars, Toyoshiro, Konahime, and Kitaakari, were included in this study (Table 1). The research fields were located in areas where no potato production fields were present nearby, thereby reducing the risk of disease transmission to other potato fields.

Figure 2. A custom-designed imaging cart for simultaneous imaging of multiple ridges and for mitigating issues caused by direct sunlight.

Table 1. Summary of the research fields and numbers of asymptomatic and diseased plants used in this study.

Images of abnormal plants with confirmed symptoms were then captured [17]. The target disease symptoms in this study included mosaic and wilting. Each captured image was partitioned into non-overlapping grids of 530 × 530 pixels. All grid-based image patches were visually assessed by two experienced potato rogueing specialists and categorized into five classes: asymptomatic, ambiguous (asymptomatic but difficult to judge), mild, moderate, and severe (Figure 3). In this study, the labels assigned by these specialists were treated as the reference standard because visual inspection by certified inspectors is the current standard practice in seed potato production and forms the basis of regulatory decision-making. While most existing studies rely on clearly manifested disease symptoms that can be identified without expert knowledge (Figure 3b), such late-stage detection is inadequate for practical rogueing operations. Since effective disease control necessitates plant removal at the earliest possible stage, this study focuses on subtle symptoms that are difficult to assess without extensive experience. To this end, we independently constructed a dataset consisting of early-stage disease images that require expert-level judgment and used it for model development and evaluation (Figure 3a).

Figure 3. Representative image examples. (a) Each visual symptom severity level for mosaic and wilting symptoms as defined in this study. (b) A leaf image from the PlantVillage dataset.

3. Methods

3.1. Proposed Methods

We previously developed a technique to visualize the features learned by deep learning-based classification models, in which the distance between the input image coordinates in the feature space and the decision boundary is utilized [22]. By applying the same principle, it is also possible to estimate relative visual symptom severity using simpler approaches. In the feature space, ambiguous and mild symptom images tend to be distributed in the vicinity of the decision boundary, while asymptomatic and severe symptom images are distributed at greater distances from the boundary (Figure 4). As a result, the classification probabilities for the asymptomatic and severe classes are high, whereas those for the ambiguous and mild classes are relatively low. Similarly, classification accuracy is higher for the asymptomatic and severe classes and lower for the ambiguous and mild classes. Based on this property, visual symptom severity is determined by training a binary deep learning classification model using only asymptomatic and severe images and utilizing the resulting class probabilities. In addition, when a set of images with the same level of visual symptom severity is available, visual symptom severity is also determined using the classification accuracy obtained for the corresponding image set. Accordingly, in this study, a binary classification model was first trained using only images from the asymptomatic and severe classes. The trained model was then applied to images from all five classes (asymptomatic, ambiguous, mild, moderate, and severe), and both the class probabilities and classification accuracy were computed. Using these measures, the feasibility of assessing visual symptom severity was systematically evaluated.

Figure 4. Schematic illustration of visual symptom severity distribution in the feature space.

For model comparison, five widely used deep learning classification architectures—ResNet-50 [23], VGG-19 [24], Inception V3 [20], DenseNet-121 [25], and Vision Transformer (ViT [26])—were implemented using the Torchvision (v0.14.1) library and systematically evaluated. The purpose of this comparison was not to identify the best-performing architecture, but to examine whether similar trends were observed across different architectures. The models were initialized using ImageNet-pretrained weights and trained on RGB images resized to 224 × 224 pixels. No data augmentation was applied. The input images were converted to tensors and normalized using the ImageNet mean and standard deviation ([0.485, 0.456, 0.406] and [0.229, 0.224, 0.225], respectively). Training was performed using stochastic gradient descent (SGD) with a momentum of 0.9 and a cross-entropy loss function. Learning rates of 0.001, 0.005, 0.01, and 0.05 were evaluated, and the learning rate yielding the highest validation accuracy was selected. A CyclicLR scheduler (mode = “triangular2”) was used. The batch size was 128, and training was performed for 50 epochs. The model achieving the highest validation accuracy on the validation dataset was selected for subsequent evaluation.

Although the proposed framework enables visual symptom severity quantification at the image-group level, its direct application to individual images is limited. Models trained with cross-entropy loss tend to produce overconfident predictions, assigning extreme probabilities even to mildly symptomatic images. This behavior hinders accurate visual symptom severity quantification at the individual image level. In practice, the feature distribution does not follow the ideal separation assumed by the binary classification framework. As illustrated in Figure 5, ambiguous samples are often located close to the asymptomatic cluster, while mild and moderate samples tend to be distributed near the severely affected cluster. This overlap leads to biased probability outputs and contributes to the overconfident predictions observed in intermediate symptom stages.

Figure 5. Schematic illustratio