A Multi-Modal Deep Learning Framework for Graded Diagnosis of Lettuce Water-Nitrogen Stress and Pest Infestation

Abstract

Lettuce is extremely sensitive to the demand for water and nitrogen throughout its growth phase. These elements are crucial for plant photosynthesis and resistance against diseases and pests. When lettuce experiences water or nitrogen stress, it leads to a decline in quality, with the leaves turning yellow, severely affecting its quality, yield, and market value. Therefore, it becomes particularly important to swiftly and accurately identify the water-nitrogen stress status of lettuce and to carry out precise graded diagnosis. This paper explores methods for recognition and graded diagnosis of water-nitrogen stress status in lettuce. We propose a comprehensive deep learning framework that integrates multi-modal data and multi-functional analysis capabilities. Specifically, we construct three specialized deep learning pipelines utilizing phenotypic-, image-, and detection-based methods, with attention 
mechanisms integrated. The DNN network model established using the Feature Phenotype Dataset (FPD) can efficiently and rapidly accomplish the recognition and graded diagnosis of lettuce water-nitrogen stress status, with an accuracy of 0.8302, providing a feasible basis for monitoring and diagnosing water-nitrogen stress status using non-image data. The ResNet50Evo-SE model, which uses image data combined with the Channel-wise Attention Mechanism (SE), achieves an accuracy of 0.9897 for the recognition and graded diagnosis of water-nitrogen stress status. The YoloV8-CBAM target detection model, which uses image data combined with the Convolutional Block Attention Module (CBAM), can accurately detect predefined visual proxies (e.g., leaf color and physiological symptoms) of water-nitrogen stress; these detected proxies are then combined according to diagnostic rules to infer the stress status, achieving an overall precision of 0.9614, recall of 0.9782, and mAP (mean average precision) of 0.9832 and 0.7665, respectively, on the held-out test set. The YoloV8-CBAM model can also effectively identify and detect lettuce leafminer pest damage, with an early-stage pest detection precision of 0.9443, and recall of 0.9531. These methods provide reliable technical support for optimizing planting management and advancing smart agriculture, contributing to the efficient and high-quality development of precision agriculture.