Publikation
A Demonstrator for Interactive Image Clustering and Fine-Tuning Neural Networks in Virtual Reality
Alexander Prange; Daniel Sonntag
In: Stefan Edelkamp; Ralf Möller; Elmar Rueckert (Hrsg.). KI 2021: Advances in Artificial Intelligence. German Conference on Artificial Intelligence (KI-2021), Germany, Pages 194-203, ISBN 978-3-030-87626-5, Springer International Publishing, 2021.
Zusammenfassung
We present a virtual reality (VR) application that enables us to interactively explore and manipulate image clusters based on layer activations of convolutional neural networks (CNNs). We apply dimensionality reduction techniques to project images into the 3D space, where the user can directly interact with the model. The user can change the position of an image by using natural hand gestures. This manipulation triggers additional training steps of the network, based on the new spatial information and new label of the image. After the training step is finished, the visualization is updated according to the new output of the CNN. The goal is to visualize and improve the cluster output of the model, and at the same time, to improve the understanding of the model. We discuss two different approaches for calculating the VR projection, a combined PCA/t-SNE dimensionality reduction based approach and a variational auto-encoder (VAE) based approach.
Projekte
pAItient - Innovationszentrum mit integrierter, rechtssicherer Umgebung zur Entwicklung, Testung, und klinischen Bewertung Kl-basierter Anwendungen.,
Ophthalmo-AI - Intelligente, kooperative Diagnose- und Therapieunterstützung in der Augenheilkunde
Ophthalmo-AI - Intelligente, kooperative Diagnose- und Therapieunterstützung in der Augenheilkunde