Transfer large models to crop pest recognition-A cross-modal unified framework for parameters efficient fine-tuning
文献类型： 外文期刊
作者： Liu, Jianping 1 ; Xing, Jialu 1 ; Zhou, Guomin 3 ; Wang, Jian 4 ; Sun, Lulu 1 ; Chen, Xi 1 ;
作者机构： 1.North Minzu Univ, Coll Comp Sci & Engn, Yinchuan 750021, Peoples R China
2.North Minzu Univ, Key Lab Images & Grap Intelligent Proc State Ethn, Yinchuan 750021, Peoples R China
3.Minist Agr & Rural Affairs, Nanjing Inst Agr Mechanizat, Nanjing 210014, Peoples R China
4.Chinese Acad Agr Sci, Agr Informat Inst, Beijing 100091, Peoples R China
关键词： Parameters efficient fine-tuning; Cross-modal fusion; Computer vision; Large pre-training model; Pest recognition; CLIP
期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.9； 五年影响因子：9.3 ）
ISSN： 0168-1699
年卷期： 2025 年 237 卷
页码：
收录情况： SCI
摘要： Crop pest recognition is an important direction in agricultural research, which is of great significance for improving crop yield and scientifically classifying pests for precision agriculture. Traditional deep learning pest recognition usually trains proprietary models on single categories and scenes as well as unimodal information, achieving excellent performance. However, this scheme has a weak foundation of general knowledge, insufficient transferability, and unimodal information has limited effect on the recognition of pest background and different life stages. In recent years, transferring the general knowledge of Large pre-trained models (LPTM) to specific domains through full fine-tuning has become an effective solution. However, full fine-tuning requires massive data and operator resources to effectively adapt all parameters. Therefore, this paper proposes a cross-modal parameter efficient fine-tuning (PEFT) unified framework for crop pest recognition with the multimodal large model CLIP as the pre-training model. The proposed method employs CLIP as the encoder for both image and text modalities, introducing the Dual-(PAL)G model. Firstly, learnable Prompt sequences are embedded in the input or hidden layers of the encoder. Secondly, multimodal LoRA is parallelly replaced in the dimension expansion layer of the fully connected layer. Then, the Gate unit integrates three PEFT methods-Prompt, Adapter, and LoRA, to enhance learning ability. We designed the GSC-Adapter and the parameter-efficient Light-GCS-Adapter for cross-modal semantic information fusion. To verify the effectiveness of the method, we conducted a large number of experiments on public datasets for crop pest recognition. Firstly, on the public dataset IP102 (for fine-grained recognition), we surpassed ViT and Swin Transformer with 66% of the sample size. In wolfberry pest dataset WPIT9K, using only about 15% of the sample size, it surpasses the previous state-of-the-art model ITF-WPI, achieving 98% accuracy. It also shows excellent performance on eight general tasks. This study provides a new technical solution for the field of agricultural pest recognition . This solution can efficiently transfer the general knowledge of multimodal LPTM to the specific pest recognition field under the condition of a few samples, with only a minimal number of parameters introduced. At the same time, this method has universality in cross-modal recognition tasks. The code for this study will be posted on GitHub (https://github.com/VcRenOne/Dual-PAL-G)
- 相关文献
作者其他论文 更多>>
-
An improved 3D-SwinT-CNN network to evaluate the fermentation degree of black tea
作者：Zhu, Fengle;Wang, Jian;Zhang, Yuqian;Zhao, Zhangfeng;Shi, Jiang;He, Mengzhu
关键词：Black tea fermentation; Hyperspectral imaging; 3D-SwinT-CNN; 3D convolutional neural networks; Swin transformer
-
Soil Inversion with Subsoiling Increases Cotton Yield Through Improving Soil Properties and Root Growth
作者：Li, Pengcheng;Feng, Weina;Dong, Helin;Zheng, Cangsong;Sun, Miao;Shao, Jingjing;Zhang, Zhenggui;Pan, Zhanlei;Wang, Jian;Sun, Guilan;Li, Junhong;Zhang, Yaopeng;Zhao, Wenqi;Zhai, Menghua;Wang, Zhanbiao;Li, Pengcheng;Feng, Weina;Dong, Helin;Zhang, Zhenggui;Pan, Zhanlei;Wang, Jian;Sun, Guilan;Li, Junhong;Zhang, Yaopeng;Zhao, Wenqi;Zhai, Menghua;Wang, Zhanbiao;Wang, Shulin
关键词：soil inversion with subsoiling; soil nutrients; residual effect; cotton yield
-
Long short-term search session-based document re-ranking model
作者：Liu, Jianping;Wang, Meng;Wang, Yingfei;Chu, Xintao;Liu, Jianping;Wang, Jian
关键词：Document re-ranking; Long short-term session search; Memory network; BERT; User intent
-
Lagged precipitation effects on plant production across terrestrial biomes
作者：He, Lei;Li, Zhao-Liang;Wang, Jian;Wang, Jian;Peltier, Drew M. P.;Ritter, Francois;Ciais, Philippe;Penuelas, Josep;Penuelas, Josep;Xiao, Jingfeng;Crowther, Thomas W.;Li, Xing;Ye, Jian-Sheng;Sasaki, Takehiro;Zhou, Chenghu;Li, Zhao-Liang
关键词：
-
Spatiotemporal Transcriptomic Profiling Reveals the Dynamic Immunological Landscape of Alveolar Echinococcosis
作者：Ou, Zhihua;Ren, Peidi;He, Fan;Chen, Jialing;Tan, Qiming;Chen, Qi;Zhang, Qian;Ou, Zhihua;Ren, Peidi;He, Fan;Chen, Jialing;Cai, Huimin;Li, Jiandong;Tan, Qiming;Chen, Qi;He, Xiuju;Zhao, Yanping;Zhang, Qian;Liang, Yingan;Li, Junhua;Li, Li;Zhou, Ting-Ting;Wu, Yao-Dong;Li, Xiu-Rong;Li, Wenhui;Zhang, Nian-Zhang;Chen, Wei-Gang;Sun, Jiwen;Wu, Yan-Tao;You, Jie;Tian, Xue-Qi;Fu, Bao-Quan;Cai, Xue-Peng;Jia, Wan-Zhong;Yan, Hong-Bin;Li, Li;Zhou, Ting-Ting;Wu, Yao-Dong;Li, Xiu-Rong;Li, Wenhui;Zhang, Nian-Zhang;Chen, Wei-Gang;Sun, Jiwen;Wu, Yan-Tao;You, Jie;Tian, Xue-Qi;Fu, Bao-Quan;Cai, Xue-Peng;Jia, Wan-Zhong;Yan, Hong-Bin;He, Fan;Chen, Jialing;Chen, Qi;Zhang, Qian;Cai, Huimin;Li, Jiandong;He, Xiuju;Zhao, Yanping;Liao, Sha;Yang, Huanming;Wang, Jian;Jin, Xin;Xu, Xun;Han, Xiumin;Tan, Qiming;Liang, Yingan;Hu, Guohai;Wang, Jian;Chen, Ao;Chen, Ao;Jin, Xin;Jin, Xin;Xu, Xun;Li, Junhua
关键词：
Echinococcus multilocularis ; macrophages; neutrophils; single-cell RNA sequencing; spatial transcriptomics -
Tracing the carbon footprint of cotton garments from seed to garment: Evidence from an empirical study of multiple sites in China
作者：Wang, Shuchen;Huang, Weibin;Zhang, Zhenggui;Wang, Zhanbiao;Zhao, Wenqi;Zhang, Zhenggui;Wang, Zhanbiao;Chong, Chenxi;Guo, Simeng;Wang, Yuhan;Zhang, Yaopeng;Pan, Zhanlei;Wang, Jian;Li, Xin;Zhang, Zhenggui;Wang, Zhanbiao
关键词：Carbon footprint; Cotton textile; Sustainable production; Environmental impact; Life cycle assessment
-
Host helicase DHX36 inhibits pseudorabies virus proliferation by unwinding the G-quadruplex in the 3'UTR of IE180
作者：Luo, Dehua;Bai, Yuqing;Li, Qingling;Zheng, Yingge;Guo, Lijun;Wang, Daozhong;Chen, Xi;Wei, Dengguo;Luo, Dehua;Zheng, Yingge;Wei, Dengguo;Luo, Dehua;Zheng, Yingge;Wei, Dengguo;Luo, Dehua;Li, Qingling;Zheng, Yingge;Guo, Lijun;Wei, Dengguo;Luo, Dehua;Li, Qingling;Zheng, Yingge;Guo, Lijun;Wei, Dengguo;Wei, Dengguo;Wei, Dengguo
关键词：Pseudorabies virus; G-quadruplex; DHX36; IE180; Viral proliferation
