More than 15,000 young people study at Münster University of Applied Sciences, one of the largest universities of applied sciences in Germany. Around 2,800 of them graduated with a Bachelor's or Master's degree last year. One of them was Daniel Lukats, who is not only one of the 28 best in 2020, but has now also been awarded the Bernard Rincklake Prize by the UAS for the best Master's thesis, which it awards annually for top performance.
Daniel Lukats dedicated his Master's thesis to reinforcement learning, a sub-discipline of machine learning in which algorithms are taught to make their own decisions independently. Specifically, he examined the learning algorithm "Proximal Policy Optimisation" (PPO) and focused on five Atari video games: "Pong", "Space Invaders", "Seaquest", "Beamrider" and "Breakout". "Video games are a typical field of application for reinforcement learning, because even simple games are constructed quite sophisticatedly," explains the 28-year-old. The Atari games, he says, are an approachable testing ground and inspire people today as they did in the past, when the table tennis game "Pong" became one of the world's most popular computer games in the 1970s.
Lukats ran his tests on 15 computers simultaneously, a total of 47 experiments in a fortnight. In the end, he had not only taught a computer to learn the video games on its own, but also developed a demonstrator for "Pong". In short videos, interested people can watch the artificial intelligence (AI)-controlled gamer as it fails to hit a "ball" with the "paddle" at the beginning and eventually wins by a large margin against the computer.
Today, one year later, Daniel Lukats only plays video games in his spare time. Professionally, as a computer scientist at the DFKI Lab Lower Saxony in Oldenburg, he has specialised in the development of algorithms for analysing data streams. "Of course, I could also develop software for smart assistance systems in cars," he says about reinforcement learning, which is still one of his favourite fields within AI. Nevertheless, it feels better for him to help his science colleagues better understand the North Sea. "I'm doing something useful here with a focus on the environment and sustainability, and maybe we're fighting climate change just a little bit," says the Oldenburg native, who is also privately committed to climate protection, explaining why he decided to join the DFKI after graduating.
Since then, Lukats has been developing an algorithm in the "ChESS" (Change Event based Sensor Sampling) project, which the DFKI is conducting together with the University of Oldenburg and the Jade University of Applied Sciences, that detects changes in underlying patterns and then causes the system to readjust to them. To do this, the young scientist is currently analysing data streams collected by sensors around the island of Spiekeroog, including from a measuring station in the North Sea and artificial islands in the mudflats off Spiekeroog. The sensors record variables such as water and air temperature, humidity and wind speed. Lukats explains: "If you read the data streams, you can say, for example, with a certain probability that it is 20 degrees warm. The idea of ChESS is that this probability can change over time, for example due to a storm with extreme wind speeds or due to the interaction of several variables such as temperature and humidity." Such changes in the environment could cause the algorithms to perform worse: "The systems learn once and then they trust that everything will stay that way."
Ocean observation, however, is not the only issue for which he wants to develop these AI methods for solving this so-called “concept drift” problem, i.e. the change of patterns in data. "I want to develop robust solutions that can also be applied to other areas such as industrial plants or recommendation systems for music platforms," he says and gives an example: "If you liked a band five years ago, that doesn't mean you still like the music today. So, providers have to adapt their recommendations to changing music habits." In order to deal with the topic even more intensively, Daniel Lukats wants to investigate it as a doctoral thesis. The proposal is already on its way.