Publication
Vid2Cuts: A Framework for Enabling AI-Guided Grapevine Pruning
Simon Häring; Sophie Folawiyo; Mariia Podguzova; Stephan Krauß; Didier Stricker
In: IEEE Access (IEEE), Vol. 12, Pages 5814-5836, IEEE, 1/2024.
Abstract
Recent advances in machine learning and computer vision promoted a surge in the development of AI-based approaches aimed at improving various agricultural tasks. In this work, we focus on grapevine pruning, which is one of the labor-intensive tasks in viticulture that requires experienced workers and has a huge impact on grapevine health, future yields and grape quality. Our objective is to develop an AI-based application that provides pruning suggestions according to the “gentle pruning” strategy enabling non-experts in the field to easily engage in the process. To achieve that, we have to deal with multiple challenges such as a large number of grapevine varieties, complicated outdoor conditions characterized by varied light, weather and complex grapevine structures with multiple occlusions. In this work, we present a framework, which allows the generation of pruning suggestions using a video recorded by a smartphone and visualize them in a mobile AR application. Thus, our contributions are the following: 1) we present the collection of a large image segmentation dataset of dormant grapevines; 2) we propose a novel distributed approach to generate pruning suggestions via a semantic 3D grapevine model generated from a smartphone video; 3) we propose a mobile AR application to visualize the pruning suggestions. Results show the robustness of our approach to outdoor conditions as well as reasonable pruning suggestions according to evaluation by domain experts in 71% of cases. We demonstrate the main challenges that must be addressed for such an application and propose a distributed solution to handle them.