Document Type

Book Chapter

Rights

This item is available under a Creative Commons License for non-commercial use only

Disciplines

1.2 COMPUTER AND INFORMATION SCIENCE

Publication Details

In CLASP Papers in Computational Linguistics Volume 1: Proceedings of the Conference on Logic and Machine Learning in Natural Language (LaML 2017), pages 41-52, Gothenburg, 12–13 June 2017, edited by Simon Dobnik and Shalom Lappin. ISSN: 2002-9764. URI: http://hdl.handle.net/2077/54911

Abstract

This paper examines to what degree current deep learning architectures for image caption generation capture spatial lan- guage. On the basis of the evaluation of examples of generated captions from the literature we argue that systems capture what objects are in the image data but not where these objects are located: the cap- tions generated by these systems are the output of a language model conditioned on the output of an object detector that cannot capture fine-grained location information. Although language models provide useful knowledge for image captions, we argue that deep learning image captioning architectures should also model geometric rela- tions between objects.

Share

COinS