Bridging the Gap: Decoding Abstract Concepts in Cultural Heritage Images

Cultural heritage (CH) institutions are digitizing their invaluable visual data, spanning images of paintings, photographs, and advertisements. A notable challenge emerges in indexing this extensive visual data, rendering manual methods impractical. To adeptly manage these expansive collections, institutions are increasingly turning to advanced computational technologies. This includes enriching visual data with linguistic tags, such as automatically detected objects and scenes. Abstract concepts (ACs) such as comfort, power, or freedom prove to be potent instruments for categorizing visual data due to the culturally embedded layers of meaning within visual forms, termed “connotation” by Barthes. These ACs serve as fundamental instruments for articulating emotions, opinions, and ideas through language, thereby facilitating the indexing and comprehensive understanding of cultural visual data. The primary objective of this chapter is to delve into the technical and socio-cultural challenges and opportunities inherent in the automatic decoding of CH visual data through the lens of abstract concepts. Specifically, it focuses on abstract concept-based image classification (AC image classification) and explores methods and insights from computer vision, cognitive science, and cultural heritage. This work includes a comprehensive analysis of experimental results and case studies, delving into the utilization of subsymbolic (deep learning) methods, symbolic approaches, and hybrid methods. By providing an in-depth examination of both performance and explainability across these approaches, the chapter highlights the potential of interdisciplinary work to bridge the gap between technical systems and hermeneutic work in the CH domain.