Robot-Enabled Field Phenotyping of Grape Berry Cluster Color Using Multimodal Vision Foundation Models for Genetic Mapping

Yiyuan Lin¹, Madan Pandey², Lance Cadle-Davidson^3,4, Matthew Clark², Soon Li Teh², Yu Jiang ^5,*

¹ School of Electrical and Computer Engineering, College of Engineering, Cornell University, Ithaca, NY, USA
² Department of Horticulture Science, University of Minnesota, Minneapolis, MN, USA
³ Grape Genetics Research Unit, United States Department of Agriculture-Agricultural Research Service, Geneva, NY, USA
⁴ School of Integrative Plant Science, Cornell University, Geneva, NY, USA
^* Corresponding authors.

–>

Abstract

Grape berry color is an important horticultural trait associated with fruit maturity, wine quality, and underlying genetic variation in grape breeding programs. However, scalable field phenotyping of grape berry color remains challenging due to occlusion, uncontrolled outdoor illumination, and substantial biological variability in vineyard environments. In this study, we developed a robot-enabled field phenotyping workflow that integrates stereo RGB imaging, image analysis using vision foundation models, and quantitative trait locus (QTL) analysis to characterize grape cluster color from vineyard imagery. High-resolution stereo RGB images were collected in a research vineyard at the University of Minnesota using a mobile imaging platform equipped with active strobe illumination and synchronized RTK GPS. Grape berry cluster regions were identified using a SAM-CLIP-based segmentation framework, followed by depth-aware filtering and instance-to-vine spatial mapping to construct stable vine-level phenotypic representations. Color descriptors extracted from multiple color spaces were subsequently aggregated at the vine level and treated as quantitative phenotypic traits for successive genetic analysis. Results showed that moderate depth filtering preserved phenotype stability while reducing potentially unreliable distant observations. The resulting image-derived phenotypes captured substantial continuous variation in grape berry coloration across vines and identified a major QTL on chromosome 2 consistent with previous categorical berry color analyses. Overall, this work demonstrates a scalable workflow for transforming vineyard imagery into biologically meaningful quantitative phenotypes for genetic studies and highlights the potential of integrating field robotics, computer vision, and quantitative genetics for future vineyard phenotyping and grape breeding applications.

Keywords: Field phenotyping; grape berry cluster color; quantitative phenotypes; QTL; precision viticulture;