To ensure the safety of offshore infrastructure such as wind turbines or oil and gas production facilities, regular inspection and maintenance is essential. However, the work carried out underwater is not only complex and expensive, but also involves considerable risks for the divers who perform it. Remotely operated underwater systems (ROVs) are already being used to monitor the condition of maritime assets. However, there is a tendency towards systems that remain in the water for long periods of time – so-called sub-sea resident AUV'S – and operate there autonomously and at the same time can be remotely controlled if necessary.
An important step in this direction has been taken by a consortium of leading companies and research institutions from the fields of IT, robotics, drive technology and offshore under the leadership of the DFKI Robotics Innovation Center in the Mare-IT project. At the end of the project, the partners, including the DFKI research departments Cognitive Assistance Systems and Embedded Intelligence as well as WITTENSTEIN cyber motor GmbH, SAP SE and ROSEN Technology and Research Center GmbH, presented a holistic solution. This comprises a novel two-arm AUV that can operate both autonomously and remotely and, thanks to two integrated manipulators, can be used for maintenance work and repairs on underwater structures. In addition, the partners are providing a powerful IT infrastructure that not only enables intuitive control and monitoring of the robot underwater but also ensures the smooth bidirectional flow of information with the plant operators and allows integration into existing business processes.
Prof. Dr. Dr. h.c. Frank Kirchner, Director of the DFKI Robotics Innovation Center: "With Mare-IT, we were once again able to demonstrate that the development of autonomous robots for complex underwater applications cannot be implemented without machine learning and artificial intelligence methods. Our research represents an important building block in making the vision of practical autonomous underwater vehicles in the offshore industry a reality. We are delighted that we were able to successfully complete the project, also thanks to the very good cooperation with our strong partners, despite the difficult conditions caused by the Corona pandemic.”
Innovative dual-arm AUV for autonomous and remote inspection and maintenance
The DFKI Robotics Innovation Center has successfully designed and built the AUV "Cuttlefish" in Mare-IT as an intervention AUV that can be freely positioned in the water column. The innovative AUV has two deep-sea gripping systems attached to its ventral side that allow it to manipulate objects underwater. In doing so, thanks to its special design and AI-based control, it is possible to change the center of gravity and buoyancy during a dive and to adopt and maintain any orientation in a stable manner. In addition to fully autonomous operations, the vehicle can be operated in a hybrid mode – so-called supervised autonomy – using a fiber optic cable. The hybrid mode allows humans to intervene and remotely control the AUV during critical operations on underwater structures. For this purpose, in addition to the manipulators, the system is equipped with many sensors for environmental perception, e.g., sonar sensors, cameras, laser scanners and magnetometers. To effectively process the large amount of sensor data, the researchers developed a special architecture concept that enables the decentralized analysis of data streams directly on board the robot. The researchers in Bremen were also responsible for implementing mission planning to carry out inspection or maintenance tasks. In addition, they developed a standard interface that enables bidirectional data exchange between the AUV, the control station and the internal business infrastructure.
Virtual co-pilot for intuitive teleoperation of the AUV and localization with artificial magnetic fields
A virtual co-pilot developed at DFKI's Cognitive Assistance Systems research department supports human personnel in the control station during teleoperation tasks. For this purpose, it is equipped with a Microsoft HoloLens, which can be used either in combination with the control station or as a lightweight separate interaction medium, e.g., on board a ship. Thanks to a holographic 3D representation on the display, the operator is always informed about the status of the AUV and the underwater structure. Sensor information and measured values can be displayed and warnings configured via natural voice interaction. In The AUV and its cameras can also be controlled via voice commands and, in combination with eye tracking, information can be retrieved on elements of the infrastructure that the person is currently looking at. Furthermore, tt is also possible to define own voice terms during the mission and to use them as alias, e.g., for camera or docking positions.
In order to complement the innovative interaction technologies, the DFKI Embedded Intelligence research department in Mare-IT has developed a web application that allows the AUV to be controlled and change parameter settings via mobile devices such as smartphones or tablets. In addition, the researchers investigated the possible use of artificial, oscillating magnetic fields for localizing the underwater robot.
Robust thruster drives with intelligent control for effective locomotion of the AUV
WITTENSTEIN cyber motor GmbH developed and produced the highly robust thruster drives for the AUV's locomotion. These have a thrust of up to 500 newtons, are protected against seawater and are suitable for water depths of up to 6000 meters. No liquid fillers were used in the manufacturing process, which employed special potting technologies to avert chemical hazards to the environment. The servo inverters that go with the thrusters include sensorless control specifically matched to the thruster motor, providing accurate, highly dynamic speed control over the entire speed range up to standstill in both directions of rotation. The AUV is equipped with eight such thruster drives, which in the overall network enable highly precise movements as required for docking with underwater structures. The real-time communication required for this is carried out within the AUV using EtherNet/IP.
Integration of the AUV into the IT infrastructure and interoperability by shaping the management shell
In Mare-IT, SAP investigated how and under what conditions autonomous systems such as AUVs can be integrated into IT infrastructures such as cloud systems. For this purpose, a vendor-independent integration of digital twins into business applications was realized by developing the management shell, a central component of the Industry 4.0 reference architecture. The management shell ensures interoperability across solutions from any manufacturer. It reduces integration costs and efforts, especially if several organizations are to access data of digital twins. SAP prototyped the management shell for planned and unplanned maintenance processes, such as the isolation valve of a subsea structure. The SAP Intelligent Asset Management cloud application can now initiate such maintenance via the isolation valve's management shell and receive execution status back from the underwater robot. Thanks to the management shell, any other systems from other parties can be added or replaced by others without much integration effort.
Development of measurement technology for non-destructive testing of underwater structures
In the project, the ROSEN Group, the leading global provider of integrity management solutions for large-scale industrial plants in the energy industry, developed four different measurement technologies for AUVs to check the condition of various subsea structures: Magnetic field sensors to detect ferromagnetic structures, sensors to measure electric fields of cathodic protection systems, and ultrasonic and eddy current sensors to measure local wall thicknesses using manipulators on the AUV. In addition, a communication buoy was developed to ensure data transmission from the robot through the water column to the central control station. ROSEN has successfully completed all four measurement technologies for use on the AUV, making an important contribution to autonomous underwater applications of complex sensor systems.
The Mare-IT project was funded from August 1, 2018, to November 30, 2021, by the Project Management Agency for Software Systems and Knowledge Technologies of the German Aerospace Center (DLR) with funding from the Federal Ministry of Education and Research (BMBF) of around 5.5 million euros.
Further information on the Mare-IT project page
Mare-IT project video
