Scientists from Germany, France, England, and Switzerland are developing new methods that encourage patient communication in a fun way – with a digital, three dimensional self-image. To create and manipulate this image, the patient's body must first be recorded with great accuracy. At the German Research Center for Artificial Intelligence (DFKI) in Kaiserslautern, there is a 3-D full body scanner set up especially for this purpose that can create individual avatars for use with AlterEgo. The scanner captures images from all perspectives in just fractions of a second with the help of 44 SLR cameras. The end result is a virtual deformable image of the real person. Prof. Dr. Didier Stricker, Head of the Augmented Reality Department at DFKI explains: “The scanning system enables the creation of an individual model within a very short time, which can then be animated with software developed at DFKI.”
The virtual twin created in this way is displayed as an interactive partner on the monitor as a mirror image. Sensors and cameras record the patient's movements and transfer them to the avatar on the monitor. The patients first get to know themselves by playing with their alter ego so that later on, they can accept others as interactive partners – after gradually disassociating from their mirror image.
The system is currently undergoing field testing at the University Clinic in Montpellier, France. After an evaluation of the results, possible interactive scenarios will be studied using “iCub,” a humanoid robot borrowed from the RobotCub project. One of the main questions scientists in the project are seeking to answer is how relevant the similarity with the other person is to the patient during the therapy.
Benoît Bardy, professor for Human Movement Sciences and director of the EuroMov Center at the University of Montpellier said: “The early field tests show that gradually adjusting a combination of similarity and dissimilarity is successful in the treatment. Specifically, the patient's social skills seem to improve most when the avatar is very similar to themselves in the beginning, and then, over time, when an increasingly foreign avatar is used. This implies that similarity is the most promising entry point for improving the abilities for interaction. We are currently preparing the final evaluation, which hopefully will confirm these findings.” The creation of the avatars poses some special challenges which are being met by the DFKI researchers with a system that features flexibility.
New uses for automated body scanning
The underlying, innovative scanning approach promises great potential for other application areas besides the field of rehabilitation. The automated creation of digital, three dimensional and moving avatars is of great interest, for example, in film production for rendering human representations, or for interactive computer games, or even for a virtual dressing room for online clothing shops. “Physical characteristics like weight and muscle distribution, posture, and measurements can be calculated with great accuracy. This is a big advantage over the standard conventional models,” said Prof. Stricker.
AlterEgo is sponsored by the European Union under the framework program “ICT 2.9 Cognitive Sciences and Robotics” and managed by the EuroMov Center at the University of Montpellier. The project was launched in the spring of 2013 and funded with 2.9 Million euros over a 3-year term. Also contributing to the project are mathematicians at Bristol University (UK), robotic experts at the Ecole Polytechnique Fédérale de Lausanne (CH), as well as the staff of healthcare specialists, psychologists, and psychiatrists at the University of Montpellier.