Publication
Introducing Concept And Syntax Transition Networks for Image Captioning
Philipp Blandfort; Tushar Karayil; Damian Borth; Andreas Dengel
In: ICMR 2016 Conference Proceedings. ACM International Conference on Multimedia Retrieval (ICMR), New York, USA, ACM, New York, NY, USA, 6/2016.
Abstract
The area of image captioning i.e. the automatic generation of short textual descriptions of images has experienced much progress recently. However, image captioning approaches often only focus on describing the content of the image without any emotional or sentimental dimension which is common in human captions. This paper presents an approach for image captioning designed specifically to incorporate emotions and feelings into the caption generation process. The presented approach consists of a Deep Convolutional Neural Network (CNN) for detecting Adjective Noun Pairs in the image and a novel graphical network architecture called “Concept And Syntax Transition (CAST)” network for generating sentences from these detected concepts.