Linked Data for Image Annotation – Repository or Software based?

McWebb, Christine
University of Waterloo, Kanada

Table of contents

Since 2011, the International Image Interoperability Framework (IIIF) has been working on developing a common framework for publishing images within repositories. According to its website:

Digital images are a container for much of the information content in the Web-based delivery of images, books, newspapers, manuscripts, maps, scrolls, single sheet collections, and archival materials. Yet much of the Internet’s image-based resources are locked up in silos, with access restricted to bespoke, locally built applications. A growing community of the world’s leading research libraries and image repositories have embarked on an effort to collaboratively produce an interoperable technology and community framework for image delivery (, accessed on January 31, 2015).

The IIIF is an excellent example of a repository-based approach to image annotation. The participating repositories, including Stanford University, St. Louis University, the University of Waterloo, Johns Hopkins University (Sheridan Libraries), e-codices Virtual Manuscript Library of Switzerland and many others frequently are holders of medieval resources, be it manuscripts, manuscript facsimile, manuscript fragments or databases of liturgical chant. Indeed, the initiative was born out of the desire to share information about medieval manuscripts. The IIIF has developed an API that it makes available to repository managers and engineers with the objective of making repositories interoperable. Once an image server that supports the   IIIF Image API is in place and the metadata has been created, repositories integrate image viewing software into the IIIF, such as page-turner technology, Mirador, etc. A great example of this is the annotation tool available on the site of e-codices ( that allows users to annotate manuscript folios, bindings or entire manuscripts where the annotation appears next to the image. The IIIF also offers the capability to annotate regions of a given image, for example a segment of a miniature image found on a particular folio of a particular manuscript. This is a huge step forward towards interoperability of repositories and breaking institutional barriers and silos. However, the fundamental problem still remains in that only those institutions that adhere to the IIIF standards will be able to share information and access.

This begs the following question: would it be helpful to tackle the interoperability issue not from the repository side but from the software side? In other words, I propose that it might serve the issue to develop annotation software that is repository-agnostic and into which image URLs can be imported. In this case, the image metadata and the annotation metadata would be housed and displayed through the software instead of the repository that sources the image. There are a few groups working on such as solution, such as SALSAH (System for Annotation and Linkage of Sources in Arts and Humanities), based at the University of Basel and collaborating with e-codices. According to their website (the software is currently not open source), their objective is “the development of a Web 3.0 application which assists researchers in Arts and Humanities who have to deal with digital sources and linkage. It allows the annotation and linkage of arbitrary digital sources such as digital texts, still and moving images and sound. The implementation is derived from the RDF-datamodel, which is modified in order to comply with all requirements in Arts and Humanities. It is designed to facilitate and encourage collaborative work models.” Interestingly, SALSAH was also born out of the initial desire/need to annotate and comment on early modern prints in the framework of a project in art history that looked at the interaction between text and image. Since its beginnings in 2009, it has found traction also with projects dealing with literary and musical sources. A beta version of the software is available online.

A second example is the Image Markup Annotation Tool (, an open-source, web-based workspace that allows users to annotate images (in png, jpg, gif) through multi-layered mark-up directly on the image, and to create comprehensive commentary on regions of the image itself or in a text editor next to it. Data is sharable and users can work in groups on a given project. Imagemat is the result of a graduate seminar on medieval iconography, where I tried to compare fairly large numbers of miniatures from fourteenth century manuscript and shared my students’ frustration that this could not easily been done. This tool is in development (though it is available online, but not advertised widely because of its beta state). Like SALSAH, serves as a useful example where image annotation is approached from the software side.

Appendix A