Artificial intelligence to help prevent plastic in rivers and oceans
How DFKI aims to eliminate plastic waste in deep sea and on the water surface
DFKI researchers in Bremen and Oldenburg are working on artificial intelligence methods to reduce plastic waste in our oceans. One specializes in underwater robotics, while the other focuses on trash floating on water's surface.
After half an hour in freezing cold, it's shift change. The relief crew arrives, peers over the icebreaker's side down to the Arctic water, and spends the next 30 minutes looking out for trash floating on the surface. Counting bottles, fishing net fragments, lids and bags is one of the scientific crew's tasks during this research expedition in the Arctic Ocean north of Spitsbergen. Diving robots assist the researchers with monitoring and keep counting underwater. At a depth of 1300 meters, at 80 degrees north, the underwater vehicle finally sights a floating plastic bag. Three years have passed since the expedition with RV Polarstern, but Marc Hildebrandt has vivid memories of the plastic bag. "It was upsetting," says the DFKI scientist and diver.
Key experiences often inspire researchers like Hildebrandt to dedicate their knowledge and expertise to the service of the sea, to further research and to protect it. Oliver Zielinski also had such an experience. Having grown up in Niedersachsen, he often spent his vacations on the coast in the late 1970s and early 1980s. At that time, in addition to numerous shells, small clumps of oil could sometimes be found there on the beach. Later, there are political agreements to protect the sea and airplanes scan the North Sea for ships that let oil into the water without permission. This example shows that things can be changed.
We can describe current states scientifically, make them technically measurable and change them societally. The chances are really good that we can use AI to help protect the oceans.
Both scientists, computer scientist Dr.-Ing. Marc Hildebrandt and marine physicist Professor Dr. Oliver Zielinski, work at DFKI; albeit in different research areas and on different projects, one in Bremen and the other in Oldenburg, but they are pursuing the same mission: they are developing artificial intelligence methods to help break down the plastic mountains in the world's oceans. Our oceans cover more than 70 percent of the earth and are vital to our survival as a global ecosystem. They strive for holistic solutions for the responsible handling of problematic, because mostly short-lived, plastic products such as packaging material or disposables. Their idea of a circular economy reaches from producer to end user, from development to recycling. While the Oldenburg team is focusing on drying up sources of plastic on aquatic surfaces, the Bremen team is tackling the underwater challenge using submersible robots.
Plastic waste: Around ten million tons of plastic waste end up in the world's oceans every year. The waste often originates on land and finds its way into the sea via rivers, where it can be found almost everywhere: In five giant swirls in the Pacific, Atlantic and Indian Oceans, in the deep sea and the Arctic, in the open sea, on the seabed and in coastal waters.
In general, plastic that has once entered the sea is incredibly difficult to remove again, and it is much more difficult underwater than on the surface. Divers reach their limits at depths of 30 meters and more; in pressure suits, the maximum depth is 450 meters. The deep sea, which lies below a depth of at least 200 meters and accounts for over 88 percent of the total ocean surface, is considered hostile to humans; it has no light and has been more poorly explored than the moon. At its deepest point, in the Mariana Trench east of the island of Guam on the edge of the Pacific Ocean, it reaches down to more than 11,000 meters; on average, the ocean floor is still about 4,000 meters deep.
Collecting waste from the deep sea requires a lot of money and technically sophisticated equipment, says Marc Hildebrandt. He therefore considers it impossible to collect plastic waste from a large oil drilling ship that costs 150,000 euros a day: "No one will pay for that, and it will never happen." What is needed, he says, are more simple systems that autonomously scan the seafloor and that can remain self-sufficient on site, that is, without needing the infrastructure of a large research vessel, of which there are only a handful in Germany anyway.
The Robotics Innovation Center (RIC) at DFKI Bremen, for which Hildebrandt has worked in the underwater team for 15 years, specializes in underwater robotics. It develops next-generation mobile robots. These can solve complex tasks independently and cooperate safely with humans. DAGON is one of several autonomous underwater vehicles in use in various research projects at RIC and is currently being tested for marine pollution analysis. The robot serves as a carrier system for the monitoring technology developed by PlanBlue GmbH as part of the MtecPla project. This combines hyperspectral and RGB imaging with underwater navigation and artificial intelligence to automatically detect and determine plastic on the seabed over large areas. "If we couple the images obtained with georeferenced data, we can assess the degree of pollution, determine changes over a period of time and plan cleanup measures," says Marc Hildebrandt, representing the project team, which is coordinated by PlanBlue.
© DFKI, Annemarie Popp
© DFKI, Annemarie PoppMtecPla: The MtecPla project is embedded into the ZIM cooperation network (Central Innovation Program for SMEs) Plastic Monitoring in European Waters (PlaMoWa Europe). It is coordinated by PlanBlue GmbH. In addition to DFKI, other project partners include the University of Stuttgart (Institute for Biomaterials and Biomolecular Systems) and Kurt Synowzik Werkzeug u. Maschinenbau GmbH & Co. KG.
Last year, the colleagues from PlanBlue spent a total of four weeks at RIC in Bremen, where they used the so-called black basin of the Maritime Exploration Hall. It has a capacity of 20 m², is 2.20 meters deep and can be completely clouded over. The darkness simulates conditions in the lightless deep sea, where only the underwater vehicles' and cameras' own light illuminates the water. Samples simulating plastic waste are placed in that small test pool. The RIC's gantry crane, which can move in three directions and has been modified with a rotating device, simulates the divers guiding the monitoring technology.
© PlanBlue
© PlanBlue
© PlanBlueDuring final tests in the Chalk Lake in Hemmoor, Niedersachsen, the partners were recently able to demonstrate that the technology tested this way also works with robots such as DAGON in a natural environment. Thanks to the combination with diving robotics, it should even be possible to visualize waste in the deep sea and on the seabed in the future. The know-how from RIC helps to ensure that the area to be searched is completely covered.
Plastic in aquatic environments also plays a crucial role in Oldenburg, 50 kilometers away,
... where the trash is not detected in the depths of the sea, but on the water surface by camera sensors. The cameras are attached to drones that fly over littered waters, for example, or are stuck under bridges. At DFKI Niedersachsen, where Oliver Zielinski manages the Marine Perception research department, scientist Mattis Wolf is responsible for several plastic waste detection projects that focus on polluted rivers.
Plastics in Southeast Asia: Southeast Asia is home of most of the ten rivers responsible for a major part of the world's plastic pollution. The causes also include the fact that waste collection and disposal does not reach parts of the population or that waste disposal sites are sometimes located directly next to rivers, where they are inadequately shielded. Asia is not solely responsible, however, as long as Western industrialized nations export their plastic waste to Asia, including Germany, which ships a large part of it to Malaysia, and thus contribute to the problem.
Cambodia is the first country where DFKI launches a project in 2019 together with the World Bank and the local government to improve waste management in the cities of Siem Reap and Sihanoukville. Later, Myanmar, Vietnam and the Philippines follow suit. Meanwhile, the system is also being used in Europe, including in Germany on the island of Spiekeroog after the annual storm surges or, most recently, in the Ahr Valley after the devastating floods in July 2021. Regardless of whether it is Asia or Europe, in all of the previously mentioned regions, DFKI receives image data that is analyzed with the help of artificial neural networks trained by human and machine interaction.
Based on the image data obtained, Mattis Wolf develops an algorithm for an AI model that can be used to determine the amount as well as the type of waste. The evaluation is fed back to the responsible local authorities. In Cambodia, for example, action plans are then developed with the goal of significantly reducing the amount of waste by 2025.
Labeling the image data: Images and videos of plastic waste are labeled at the DFKI in Oldenburg. Essentially, the images are divided into countless small tiles and assigned to a catalog, previously defined as to what can be seen on each tile. Bottles are relatively easy to label because they are found very often, but it is more difficult with lids and other plastic items, which are recorded less often. In the end, all plastic items must be presented in a balanced way within the data the model is trained with. That' s the only way the AI can reliably distinguish between cups, bags, food packaging or transport containers and deliver reliable results.
Until now, the DFKI has received its data from the air and evaluated it in Oldenburg. The AILHA project, which Mattis Wolf is leading, is now going one step further: it not only wants to generate image data on a rubbish collection boat, but also evaluate it in real time. To do this, a camera is installed on the boat's conveyor belt, which is used to bring the floating rubbish on board. The camera then sends the images to a laptop in the driver's cabin, where they are evaluated with the help of AI.
The person steering the boat will later receive real-time totals of how much plastic is in the waste or what type of plastic it is, for example 10,000 bottles, 5,000 lids, 100,000 pieces of wood. The data can be passed on directly to the waste management company when the waste is unloaded, giving them reliable figures on the amount of combustible or recyclable material that can be recycled. "To generate this information autonomously later, we use machine learning methods as part of AI," explains Mattis Wolf: "With our guidance, the system learns to distinguish waste from one another on the conveyor belt and to determine quantities."
In AILHA, the DFKI is working together with partners. One of them is the mechanical engineering company Berky, which builds special boats. On its company premises in Haren on the Ems in northern Germany, Berky has a test basin, 30 metres long, four metres deep. The latest rubbish collection boat, later christened Moringa by Landmarken, makes a test run one day in November. As the boat driver starts the engine, an employee pours a yellow bag full of prepared plastic waste into the water. The wind blows hard and quickly drives the light packaging into the corners of the basin, but after a few manoeuvres the boatman has carried everything onto the deck.
The machine builder Berky: Berky is a medium-sized all-rounder for environmentally friendly water maintenance. It looks back on a 58-year company history and today builds around a hundred machines a year for customers all over the world, ranging from self-propelled mowing machines to mowing, collection and dredging boats. In March 2020, Berky built its first refuse collection boat. In the meantime, several boats of different sizes are in operation worldwide. Managing director Felix Knoll, whose grandfather was one of the three founders of Berky, says: "More and more people are willing to invest money so that less waste ends up in the rivers. This is not greenwashing to ease one's conscience, but the genuine will to clean up waters. This is the path we want to follow, because we have the potential to do our bit and make a difference in the world."
The waste collection boat arrived in Phnom Penh, Cambodia, after a 50-day journey by sea. Since then, it has been used on a daily basis in the Mekong, one of the ten most polluted rivers in the world, and its tributaries. The Cleanup Mission is led by everwave, a long-standing partner of the DFKI Lower Saxony. Cambodia may be everwave's first mission in Southeast Asia, but it is far from the first successful cleanup that the dedicated team has carried out in cooperation with the DFKI.
In spring last year, another waste collection boat, called CollectiX, caused a stir in Bosnia-Herzegovina, for example. In the border river Drina, near the town of Visegrád, tons of trash such as plastic, refrigerators and car tires piled up in front of a hydroelectric power plant in March 2021. Clemens Feigl, one of the co-founders of everwave explained at that time, "AI has a significant role in our operations here. It helps us identify where what kind of trash is and where we need to use our boat efficiently."
“The ocean and its resilience are limited,” says Oliver Zielinski, who founded the Competence Center “AI for Environment and Sustainability,” or DFKI4planet, in 2020. This is where he intends to bundle and drive forward work at DFKI. On one hand, they should design AI itself to be more sustainable, but on the other hand, they should also develop AI methods for more sustainability and thus directly contribute to the UN's Sustainable Development Goals. If AI can be used in the future as effectively as possible to protect the environment and climate, says Zielinski, “then it can indeed contribute a bit towards saving the planet.”
The UN's Sustainable Development Goals: AI methods such as those being developed in Bremen and Oldenburg for marine conservation pay directly into the global Sustainable Development Goals adopted by the United Nations in New York in 2015 as part of the 2030 Agenda. Number 14 of a total of 17 goals is: Life under water. The German government is committed to significantly reducing all forms of marine pollution and nutrient loads by 2025. For this purpose, scientific knowledge is to be deepened, research capacities expanded and marine technologies passed on.
The Marine Strategy: At the beginning of 2022, the German government adopted a Marine Strategy for clean oceans, the protection of marine biodiversity and against marine litter. In the coming years, it will consolidate all aspects of marine protection in a marine strategy and reconcile the protection of the sea with environmentally compatible use.