Swin Transformer+ConvNeXt双流融合的水稻叶部病害分类

1. Introduction

Rice (Oryza sativa) is one of the most important staple crops, providing a primary food source for more than 50% of the global population. However, rice production is continuously affected by various foliar diseases, including Neck Blast, Leaf Scald, Sheath Blight, Narrow Brown Leaf Spot, Leaf Blast, Rice Hispa, Brown Spot, and Bacterial Leaf Blight. These diseases significantly reduce crop productivity and quality, creating serious challenges for global food security, particularly in developing countries where agriculture plays a major role in the economy. Furthermore, favorable environmental conditions can accelerate disease propagation, emphasizing the importance of accurate and timely disease diagnosis for effective crop management [1,2,3].

Traditionally, rice disease identification has relied on expert visual inspection and laboratory-based analysis [4]. Although these approaches can provide reliable diagnosis, they are often time-consuming, labor-intensive, and difficult to access for small-scale farmers in remote agricultural regions. Inaccurate or delayed disease identification may lead to inappropriate pesticide application, increasing production costs and causing potential environmental impacts [5]. Therefore, automated and accessible disease recognition systems have become increasingly important for supporting efficient agricultural practices.

In recent years, deep learning (DL)-based approaches, particularly Convolutional Neural Networks (CNNs), have demonstrated significant potential for plant disease classification using leaf images. CNN-based methods effectively extract local spatial patterns, including color variations, textures, and lesion characteristics, enabling high classification performance under controlled experimental conditions [6]. However, CNN-based models may face challenges in capturing long-range dependencies and global contextual relationships due to their inherently local receptive field operations [2]. These limitations can affect their ability to fully represent complex disease patterns that involve both localized symptoms and broader leaf-level characteristics.

To address the limitations associated with convolution-based feature extraction, Vision Transformers (ViTs) have recently gained attention due to their ability to model global relationships through self-attention mechanisms. Unlike CNNs, transformer-based architectures represent images as sequences of patches, enabling comprehensive analysis of spatial dependencies across the entire image [7]. Consequently, transformer-based methods have demonstrated improved robustness and competitive performance in various plant disease recognition tasks [8]. Furthermore, a deep learning-based ROA-DM framework has been proposed for automated rice leaf disease classification, integrating a Deep Autoencoder (DAE), Deep Max-out Network (DMN), and Remora Optimization Algorithm (ROA) [9]. However, despite their high predictive performance, many existing DL-based agricultural diagnosis systems still face challenges related to model interpretability. These models often operate as black-box systems, providing predictions without sufficient explanations regarding disease-related visual evidence. Such limitations reduce user confidence and hinder practical adoption in agricultural applications. Therefore, Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM) and attention visualization, have become increasingly important for identifying disease-relevant regions and improving model transparency [10].

Beyond accurate and interpretable disease recognition, practical agricultural applications require intelligent decision-support capabilities that can provide understandable and actionable recommendations to farmers. Recently, Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and knowledge-based reasoning. The integration of LLMs with disease recognition systems provides opportunities to generate disease-specific management recommendations, preventive strategies, and treatment guidance, thereby bridging the gap between automated diagnosis and agricultural decision-making [11,12]. Nevertheless, many existing studies remain primarily focused on offline model evaluation, while challenges related to real-time deployment, user interaction, interpretability, and practical usability remain insufficiently addressed. Therefore, an integrated framework combining accurate classification, explainable prediction, and intelligent advisory support is highly desirable for real-world smart farming applications [13].

To address these challenges, this study proposes a dual-backbone complementary representation learning framework for accurate and interpretable rice leaf disease diagnosis. The proposed framework integrates Swin Transformer and ConvNeXt [14] architectures to exploit complementary visual representations. Specifically, Swin Transformer is employed to capture global contextual dependencies and long-range spatial relationships, whereas ConvNeXt is utilized to extract fine-grained local texture features associated with disease symptoms. An attention-based feature fusion module is further applied to adaptively combine these heterogeneous feature representations and enhance discriminative disease feature learning. Additionally, Grad-CAM is incorporated to provide visual explanations by highlighting disease-relevant regions contributing to model predictions. Furthermore, a web-based LLM advisory module is introduced as a post-diagnosis decision-support component to generate disease-specific management recommendations based on the predicted disease category. The LLM module is used exclusively for advisory purposes and does not participate in feature extraction, classification, or performance evaluation.

The main contributions of this study are summarized as follows:

• A dual-backbone complementary representation learning framework is proposed by integrating Swin Transformer and ConvNeXt to capture global contextual information and local disease-specific texture characteristics.

• An attention-based adaptive feature fusion mechanism used to combine complementary feature representations and improve discriminative disease classification.

• A comprehensive evaluation framework is established using multiple state-of-the-art baseline models, cross-validation, and cross-dataset testing for robustness and generalization capability.

• An explainable disease diagnosis mechanism based on Grad-CAM is incorporated to improve model transparency and identify disease-relevant image regions.

• A practical web-based decision-support extension with an LLM-based advisory module is introduced to provide disease-specific agricultural recommendations after diagnosis.

The remainder of this paper is organized as follows. Section 2 presents the related work, including recent advances in rice leaf disease diagnosis, transformer-based architectures, explainable artificial intelligence, and intelligent agricultural decision-support systems. Section 3 describes the proposed methodology, including dataset preparation, preprocessing, dual-backbone Swin Transformer-ConvNeXt architecture, attention-based feature fusion, Grad-CAM-based explainability, and the post-diagnosis LLM advisory module. Section 4 presents experimental results and detailed analysis, including comparisons with baseline models, ablation studies, cross-validation results, cross-dataset evaluation, confusion matrix analysis, ROC-AUC analysis, and explainability assessment. Section 5 discusses the web-based deployment framework and its applicability for smart agricultural practices. Finally, Section 6 concludes the paper and highlights future research directions.

2. Related Work

The growing need for intelligent and scalable agricultural solutions has made automated rice leaf disease detection an active research topic. In recent years, deep learning-based solutions have greatly enhanced the precision and effectiveness of plant disease diagnosis. Nevertheless, the current approaches differ in their capacity to reflect complicated features, be interpretable, and be applicable in the real world. Convolutional Neural Networks (CNNs) have been widely applied to rice leaf disease classification because they are powerful at extracting local spatial features like texture, color, and lesion patterns. For example, Nguyen et al. [15] introduced an LCSNet–FedPerGC framework for rice leaf disease classification, which is a combination of a lightweight CNN and an optimized federated learning strategy. The framework was shown to be highly accurate for classification, efficient for inference and robust with respect to both IID and non-IID data distributions. A similar approach was used to introduce a Concurrency Attention-Based CNN for rice leaf disease recognition under different lighting conditions, which uses a multiple-parallel attention mechanism to capture discriminative features and ignore background interference, with a classification accuracy of 96.2% [16]. Moreover, Twin-CNN structures [17] have shown enhanced performance through the application of dual-stream feature extraction. Despite the promising results obtained using CNN-based models, they mainly use local receptive fields, which restricts their capacity to extract long-range dependencies and global contextual information. As a result, they can perform poorly in real-life situations with complex backgrounds, occlusions, and different illumination. Transformer-based models have been suggested to address the shortcomings of CNNs in detecting plant diseases. Vision Transformers (ViTs) are self-attention models that are used to capture global relations between image patches, enhancing contextual knowledge. As shown by Singha Roy and Kukreja [18], ViT-based models are more robust when it comes to rice leaf disease classification. Likewise, AISOA-SSformer [19] uses a transformer-based segmentation framework with sparse attention to boost disease localization. In addition, Swin Transformer-based approaches [20] have shown strong performance by incorporating hierarchical feature representation and shifted window attention mechanisms. Explainable transformer models [21] further improve transparency by incorporating attention visualization. Nevertheless, the benefits of transformer-based models are not without drawbacks as they typically need massive datasets and high-performance computing, and most current implementations cannot be explained or deployed. More recent studies have concentrated on hybrid architectures that combine CNN and transformer models to utilize both local and global feature representations. As an example, GRG-ViT [22] combines generative adversarial networks (GANs) with Vision Transformers to deal with the issue of class imbalance and enhance generalization. Likewise, hybrid CNN–transformer models [23] exhibit better performance due to the integration of convolutional feature extraction and global attention mechanisms. In addition, ConvNeXt-based architectures [24] are also suggested as CNN backbone modernizations that enable a transition between CNNs and transformers. Hybrid models based on multiple AI models that are green and emphasize computational efficiency and sustainability are also highlighted as green AI-based hybrid models [25]. Nevertheless, A considerable number of hybrid models do not include integrated explainability analysis and do not encompass the use of better classification performance, even though they are more effective than their counterparts. Explainable Artificial Intelligence (XAI) has become a key element of enhancing trust and transparency in AI-based agricultural systems. Grad-CAM and other techniques have been extensively applied to identify disease-related areas on leaf images [26]. Transformer-based explainability techniques [21] further increase interpretability through the use of attention maps. Despite the fact that these methods offer great insight into how models make decisions, they are not usually incorporated into a holistic decision-support system but are used as modules on their own. The combination of Large Language Models (LLMs) with vision-based systems is a recent development that has introduced new opportunities in intelligent agriculture. Advisory frameworks assisted by an LLM [27] can be used to produce disease-specific recommendations, such as treatment plans and prevention. Multimodal vision-language models [28,29] also add more capabilities to the system by integrating image comprehension with natural language reasoning. Also, research emphasizes the need to go beyond dataset-based assessment to real-world usage [30]. Practical AI solutions for farmers are feasible as illustrated by web-based deployment frameworks and real-time mobile systems [31] and [32], respectively. Nevertheless, the current LLM implementations tend to be disconnected with the model of explainable visions, which prevents them from connecting predictions with practical insights. Although recent progress has been made, a number of problems still exist. First, the existing approaches focus mainly on classification performance and do not fully consider the robustness of the system in its real-world applications involving changes in lighting conditions, background complexity and other environmental factors. Second, many hybrid CNN–Transformer models have been proposed to improve feature representation, but most do not yet fully leverage the complementary characteristics of global information and fine-grained local information related to disease textures via adaptive feature learning. Third, there has been little time spent on incorporating explainable AI approaches, which could boost the openness and credibility of disease prognosis. Finally, although there are many research papers that look at automatic disease detection, there are limited studies have explored integrating diagnosis models with farmer-oriented decision-support systems. In view of this, a dual-backbone complementary representation learning network is proposed to tackle the above problems, which involves using both Swin Transformer and ConvNeXt with an attention-based feature fusion module to provide a robust and interpretable rice leaf disease diagnosis. A list of suggested management recommendations is also included using an LLM-based advisory system, which is further augmented with a web-enabled system, and Grad-CAM to offer visual explanations.

Table 1 shows the quantitative comparison of various methods used for detecting diseases on rice leaves. Some of the important characteristics include model structure, explainability, advisibility, deployment, robustness, and scalability.

Table 1. Comparison of recent rice leaf disease detection methods.

As summarized in Table 1, most of the current techniques are mainly employed for classification accuracy and/or feature extraction, lacking a holistic and verifiable diagnostic framework. CNN-based approaches [15,16] well model local visual patterns with limited capability of modeling long-range contextual dependencies. Transformer-based methods [19] have been employed to enhance the modeling of global context, but they generally have limited interpretability and applicability. Current hybrid methods [22,23,25] blend CNNs and Transformers; however, they most commonly integrate features using static or identity mappings and fail to substantially leverage the complementary feature representations using adaptive attention mechanisms. The proposed model, on the contrary, adopts the dual-backbone complementary representation learning scheme and attention-based feature fusion for simultaneous learning of global contextual and local texture representation and generalization of disease characterization. Additionally, Grad-CAM makes the model more interpretable and the LLM advisory module, accessible via the web, is used only as a post-diagnosis decision-support component (presenting management suggestions to the farmer rather than involvement in disease classification). The entire system is implemented on the Hugging Face platform, facilitating fast and on-the-go inference and deployment in the field of smart agriculture.

3. Dataset Description

3.1. Dataset-1

This study utilizes a publicly available rice leaf disease image dataset collected from real-world agricultural field environments. The dataset was obtained from the Kaggle platform and contains high-resolution images representing different healthy and diseased rice leaf conditions. Since the images were collected in practical field environments, the dataset includes considerable variations in illumination, background complexity, leaf orient