Beyond Seasonality: A High-Fidelity Apple Orchard Digital Twins

Ruiming Du¹, Yiyuan Lin², Daoyuan Jin², Nicholas Gunner³, Yu Jiang ^3,*

¹ Department of Biological and Environment Engineering, Cornell University, Ithaca, NY,
² School of Electrical and Computer Engineering, College of Engineering, Cornell University, Ithaca, NY, USA
³ School of Integrative Plant Science, Cornell University, Geneva, NY, USA
^* Corresponding authors.

–>

Abstract

Agricultural robotics is strongly constrained by seasonality, which limits the efficiency and effectiveness of robot development, testing, and evaluation for field deployment. For example, robotic pruning must be evaluated during the dormant season, when snow and low temperatures create harsh conditions that are impractical for sustained field experimentation. To address comparable constraints in extreme and inaccessible environments, the concept of a digital twin has been adopted by NASA for spacecraft development, enabling validation without continuous reliance on real-world operations. In agriculture, however, existing digital twins, particularly those targeting complex orchard systems, are limited in availability and often exhibit low fidelity, creating a major bottleneck for simulation-to-real (Sim2Real) transfer learning and learning-based robot control. To address this gap, we developed a high-fidelity digital twin of a modern high-density orchard system using NVIDIA Isaac Sim. The digital twin includes a simulated orchard field, dormant apple trees generated by L-TreeGen with high realism, and a field robot based on an Amiga platform, along with RGB-D and LiDAR sensors for 2D and 3D perception tasks. The system enabled configurable downstream tasks for perception model development, including field synthetic data collection and multiple sim2real learning applications. We evaluated Sim2Real perception on real orchard data at Cornell campus using 2D and 3D pipelines trained on synthetic data from the proposed twin: 2D tree-mask segmentation from synthetic RGB-D images and 3D branch instance segmentation with synthetic TLS point clouds. Results showed that models trained on synthetic data consistently improved real-world perception performance, demonstrating the efficacy of the proposed digital twin in facilitating agricultural robot development.

Keywords: Digital twin; Sim-to-real; Synthetic data generation; Robotics; Perception