基于改进YOLOv11n-seg的胡麻倒伏区域识别方法——兼顾精度与轻量化的智能化倒伏监测

欢迎您访问《智慧农业（中英文）》官方网站！English

Toggle navigation智慧农业

• 网站首页
• 期刊介绍
• 编委会
• 投稿指南
• 期刊服务
• 下载中心
• 联系我们

Smart Agriculture ›› 2026, Vol. 8 ›› Issue (2): 118-132.doi: 10.12133/j.smartag.SA202510010

• 专题--多源遥感驱动数字农业创新与实践 •上一篇下一篇

面向云遮挡与复杂地块场景下的跨模态注意力多源遥感农作物分类

巫晨旭, 左浩龙, 李刚()

1. 黑龙江工程学院测绘工程学院，黑龙江 哈尔滨 150050，中国

• 收稿日期:2025-10-11 出版日期:2026-03-30

• 作者简介:巫晨旭，硕士研究生，研究方向为时空大数据。E-mail：[email protected]

• 通信作者:李 刚，博士，讲师，研究方向为时空大数据。E-mail：[email protected]

Cross-Modal Attention for Multi-Source Remote Sensing Crop Classification under Cloud Occlusion and Complex Field Scenarios

WU Chenxu, ZUO Haolong, LI Gang()

1. School of Geomatics Engineering, Heilongjiang Institute of Technology, Harbin 150050, China

• Received:2025-10-11 Online:2026-03-30
• Foundation items:Heilongjiang Province Double First-Class Discipline Coordinated Innovation Achievement Project(LJGXCG 2025-P18)
• About author:WU Chenxu, E-mail: [email protected]
• Corresponding author:LI Gang, E-mail: [email protected]

在线阅读 24

知网下载

本地下载 48

## 可视化 0

摘要/Abstract

摘要：

【目的/意义】 本研究旨在突破传统光学遥感在云遮天气与耕地破碎地形下的局限性，通过构建一种具备优秀跨模态融合能力与泛化性能的深度网络模型来提升农业遥感分类精度。 【方法】 提出一种基于注意力机制的3D卷积神经网络（3D Convolutional Neural Network Based on Attention Mechanism, Attention-3DCNN）模型：其通过3D卷积+2D卷积结构处理时序哨兵二号多光谱影像，在空间与时间维度上提取丰富特征；同时以深度可分离卷积形式处理来自哨兵一号的合成孔径雷达（Synthetic Aperture Radar, SAR）数据，实现对全天候可获取信息的高效抽取；进一步，模型引入“通道-时间-空间”三重注意力机制与残差连接策略，对两个模态的特征进行动态加权与深度融合，使得在光学数据缺失或遮挡严重的情形下，SAR数据能够有效补偿并维持分类性能。 【结果和讨论】 为全面评价模型性能，选取法国全景农业卫星时序数据集，以及山东省沂水县实测数据集进行对比实验：在法国数据上模型达到97.5%的整体准确率，在沂水县数据上获得93%的准确率，均显著优于对照基线模型；同时，通过对注意力分布的可视化分析可见，模型聚焦的关键物候期与当地农业实地记录高度一致，其高权重光谱波段亦符合农学机理，这体现出模型在判别机制层面的可解释性。 【结论】 综上，Attention-3DCNN模型在耕地破碎、云遮影响严重的山区条件下显著提升了作物分类精度，具有良好的推广前景与应用价值。

关键词:农作物分类,深度学习,3D卷积神经网络,注意力机制,遥感,合成孔径雷达

Abstract:

[Objective] Accurate and timely crop mapping is fundamental for agricultural management, yield forecasting, and food security assessment. However, in mountainous and hilly regions characterized by frequent cloud cover and highly fragmented farmland, crop classification methods relying solely on optical remote sensing data are severely constrained. Persistent cloud contamination introduces data gaps and temporal inconsistencies in optical image time series, significantly degrading classification accuracy and robustness. To address these limitations, a robust and adaptive deep learning framework is developed capable of effectively integrating multi-modal remote sensing data. The primary objective is to enhance crop classification accuracy and stability under complex conditions where optical observations are scarce or unreliable, thereby supporting reliable agricultural monitoring in cloudy and fragmented landscapes. [Methods] A novel deep neural network architecture named 3D convolutional neural network based on attention mechanism (Attention-3DCNN) was proposed, designed to jointly exploit multi-temporal optical and synthetic aperture radar (SAR) observations. The model integrated Sentinel-2 multispectral time-series imagery with weather-insensitive Sentinel-1 SAR data through a dedicated cross-modal fusion strategy driven by a triple-attention mechanism. The network adopted a dual-branch feature extraction architecture. For the Sentinel-2 data, a hybrid module combining three-dimensional and two-dimensional convolutional neural networks (3D-CNN and 2D-CNN) was employed to capture discriminative spatiotemporal features and crop phenological dynamics across the growing season. This design enabled effective modeling of the spectral-temporal interactions inherent in crop development. For the Sentinel-1 SAR data, depthwise separable convolutions were utilized to efficiently extract spatial and textural features related to crop structure and surface scattering characteristics while reducing computational complexity. Features extracted from both modalities were subsequently integrated using a custom-designed attention-based fusion module. This module consisted of three complementary attention mechanisms: channel attention, temporal attention, and spatial attention. Residual connections were incorporated throughout the network to facilitate stable training and effective gradient propagation. The proposed model was evaluated on two datasets to assess both its performance and generalizability. The first was the publicly available panoptic agricultural satellite time series (PASTIS) benchmark dataset from France, which contained dense time-series observations and multiple crop classes. The second was a real-world dataset constructed for Yishui county, Shandong province, China, which was characterized by high cloud frequency (approximately 33%), highly fragmented farmland (average parcel size < 0.5 hm 2), and a relatively simple crop rotation system. Comparative experiments were conducted against several state-of-the-art models, including 3D-ConvSTAR, UNet++, Self-Attention 3D, CNN-LSTM dual-stream network, and TGF-Net. Ablation studies were also performed to quantify the contribution of each attention component. [Results and Discussions] Experimental results demonstrated that Attention-3DCNN consistently outperformed all baseline methods on both datasets. On the PASTIS benchmark, the model achieved an overall accuracy (OA) of 97.5%, confirming its strong classification capability under favorable observation conditions. On the more challenging Yishui county dataset, Attention-3DCNN attained an OA of 93%, outperforming the other comparison models. Ablation experiments confirmed the effectiveness of the proposed triple-attention mechanism, as removing any attention component resulted in a clear reduction in classification performance. Under heavy cloud coverage, Attention-3DCNN exhibited the smallest accuracy degradation, with an OA drop of only 3.6 percentage points, indicating its ability to adaptively rely on SAR information when optical data quality deteriorated. In regions with highly fragmented farmland, the proposed model also maintained the highest accuracy and the smallest performance decline (2.8 percentage points), benefiting from the spatial attention mechanism. Moreover, attention visualization provided meaningful interpretability. Temporal attention peaks aligned with key crop phenological stages, while channel attention highlighted spectrally and physically informative optical bands and SAR polarizations, which was consistent with established agronomic and remote sensing knowledge. [Conclusions] This study presents the Attention-3DCNN model for accurate and robust crop classification in regions affected by persistent cloud cover and fragmented agricultural landscapes. By fusing Sentinel-2 optical and Sentinel-1 SAR time-series data through a channel-temporal-spatial triple-attention mechanism, the proposed framework enables adaptive integration of complementary multi-modal information. The model achieves outstanding performance on both benchmark and real-world datasets, demonstrates strong robustness under adverse conditions, and offers enhanced interpretability. Overall, the proposed approach provides a reliable and practical solution for crop mapping in complex agricultural environments.

Key words:crop classification,deep learning,3D convolutional neural network,attention mechanism,remote sensing,SAR

中图分类号:

• S127

• TP18

引用本文

巫晨旭, 左浩龙, 李刚. 面向云遮挡与复杂地块场景下的跨模态注意力多源遥感农作物分类[J]. 智慧农业(中英文), 2026, 8(2): 118-132.

WU Chenxu, ZUO Haolong, LI Gang. Cross-Modal Attention for Multi-Source Remote Sensing Crop Classification under Cloud Occlusion and Complex Field Scenarios[J]. Smart Agriculture, 2026, 8(2): 118-132.

使用本文

[收藏文章](javascript:;) /[推荐](javascript:void(null))
导出引用管理器EndNote|Ris|BibTeX

链接本文:https://www.smartag.net.cn/CN/10.12133/j.smartag.SA202510010

https://www.smartag.net.cn/CN/Y2026/V8/I2/118

[图/表 23](javascript:;)

图1

山东省沂水县研究区域示意图 注：该图基于自然资源部标准地图服务网站下载的审图号为GS（2019）3333号标准地图制作，底图无修改。

图1

表1

PASTIS数据集与Sentinel-2波段对应关系及分辨率对比表

PASTIS 波段	Sentinel-2 波段	原始分辨率/m	重采样后分辨率/m
1—4	B2—B4， B8	10	10
5—7	B5—B7	20	10
8	B8A	20	10
9—10	B11—B12	20	10

表1

图2

PASTIS数据集 a. 光学影像 b. SAR数据 c. 分类标签

图2

表2

PASTIS与沂水县数据集核心参数的对比

参数	PASTIS（法国）	沂水县（中国）	对迁移的潜在影响
云覆盖率/%	28	33	沂水县需更强的抗云干扰能力
时相数量（4—9月）	32个时相	23个时相	沂水县时序信息更稀疏，考验时间注意力
平均地块面积/hm 2	1.2	<0.5	沂水县需更高的空间细节捕捉能力

表2

图3

沂水县主要农作物分类数据集

图3

图4

Attention-3DCNN模型总架构图 注：DSConv为深度可分离卷积层；3D-CNN、2D-CNN分别是3D卷积层和2D卷积层；Ad-Pooling为自适应池化。

图4

图5

多光谱分支模型图 注：Conv3D（3×3×3，s=1，p=1，C_out=64），Conv2D（3×3，s=1，p=1，C_out=128），Ad-Pooling（1×1，s=1，p=0，C_out= 128）；s为步长，p为填充，C_out为输出大小；spectral、time、spatial分别代表通道、时间、空间注意力机制。

图5

图6

SAR分支模型架构图 注：BN为批量归一化；LeakyReLu为LeakyReLU激活函数DSConv（3×3，s=1，p=1，C_out=C_in）；Ad-Pooling（1×1，s=1，p=0，C_out=128）；s为步长；p为填充；C_out为输出大小；C_in为输入大小。

图6

图7

注意力机制流程图

图7

图8

通道注意力机制模型图

图8

表3

Attention-3DCNN模型与其他模型在不同数据集上的表现

模型名称	OA（PASTIS）/%	宏平均 F 1 分数（PASTIS）	Kappa（PASTIS）	OA（沂水县）/%	宏平均 F 1 分数（沂水县）	Kappa（沂水县）
标准3D-CNN	94.2	0.932	0.915	87.2	0.845	0.822
光学-SAR简单融合模型	95.3	0.945	0.928	89.5	0.876	0.858
仅SAR 3D-CNN模型	91.6	0.902	0.889	88.3	0.861	0.842
注意力双分支融合模型	95.1	0.938	0.935	89.9	0.882	0.870
通道注意力双分支融合模型	96.5	0.953	0.942	90.0	0.898	0.890
时间注意力双分支融合模型	95.8	0.945	0.942	89.5	0.894	0.887
空间注意力双分支融合模型	96.9	0.959	0.956	91.0	0.906	0.900
Attention-3DCNN	97.5	0.970	0.965	93.0	0.920	0.910

表3

图9

在消融实验中Attention-3DCNN与其他模型在沂水县数据集上的结果

图9

图10

在消融实验中Attention-3DCNN模型与其他模型在PASTIS数据集上的结果

图10

表4

Attention-3DCNN模型与其他模型的对比

模型名称	核心方法简述	OA（沂水县）/%	Kappa（沂水县）	参数量/M	计算量/GFLOPs	推理时间/（ms/景）
3D-ConvSTAR	3DCNN多源融合（固定权重）	89.5	0.858	45.2	128.3	156
Self-Attention 3D	自注意力机制+3DCNN	90.5	0.872	52.7	145.6	183
UNet++	编码器-解码器结构，多尺度特征融合	88.0	0.835	68.9	212.4	245
CNN-LSTM-DS	光学影像时序+纹理特征融合	86.5	0.815	43.0	98.7	132
TGF-Net	基于Transformer与卷积（CNN）架构	90.5	0.900	105.3	285.1	312
Attention-3DCNN	跨模态三重注意力（通道-时间-空间）	93.5	0.910	41.3	97.2	133

表4

图11

在对比实验中Attention-3DCNN模型与其他模型在沂水县数据集上的对比结果

图11

表5

在高云量的条件下Attention-3DCNN模型与其他模型在沂水县数据集的对比

模型名称	主要依赖信息	OA/%	宏平均 F 1	Kappa
3D-ConvSTAR	光学时序为主	83.6	0.802	0.775
Self-Attention 3D	光学 + 时序注意力	85.1	0.821	0.796
UNet++	光学空间特征	82.4	0.789	0.761
CNN-LSTM-DS	光学时序 + 手工特征	81.9	0.781	0.754
TGF-Net	Transformer + CNN融合	86.3	0.836	0.812
Attention-3DCNN	跨模态三重注意力（S2+S1）	89.4	0.872	0.846

表5

表6

多源遥感农作物分类研究中不同模型在高云量条件下的下降幅度

模型	常规OA/%	高云量OA/%	下降幅度/百分点
3D-ConvSTAR	89.5	83.6	5.9
Self-Attention 3D	90.5	85.1	5.4
UNet++	88.0	82.4	5.6
CNN-LSTM-DS	86.5	81.9	4.6
TGF-Net	90.0	86.3	3.7
Attention-3DCNN	93.0	89.4	3.6

表6

[![Image 35](https://www.smartag.net.cn/fileup/2096-8094/FIGURE/2026-8-2/Images/2096-8094-2026-8-2-118/thumbn