Information on the history of architecture is embedded in our daily surroundings, in vernacular and heritage buildings and in physical objects, photographs and plans. Historians study these tangible and intangible artefacts and the communities that built and used them. Thus valuable insights are gained into the past and the present as they also provide a foundation for designing the future. Given that our understanding of the past is limited by the inadequate availability of data, the article demonstrates that advanced computer tools can help gain more and well-linked data from the past. Computer vision can make a decisive contribution to the identification of image content in historical photographs. This application is particularly interesting for architectural history, where visual sources play an essential role in understanding the built environment of the past, yet lack of reliable metadata often hinders the use of materials. The automated recognition contributes to making a variety of image sources usable for research.

@en

We introduce AmsterTime: a challenging dataset to benchmark visual place recognition (VPR) in presence of a severe domain shift. AmsterTime offers a collection of 2,500 well-curated images matching the same scene from a street view matched to historical archival image data from Amsterdam city. The image pairs capture the same place with different cameras, viewpoints, and appearances. Unlike existing benchmark datasets, AmsterTime is directly crowdsourced in a GIS navigation platform (Mapillary). We evaluate various baselines, including non-learning, supervised and self-supervised methods, pre-trained on different relevant datasets, for both verification and retrieval tasks. Our result credits the best accuracy to the ResNet-101 model pre-trained on the Landmarks dataset for both verification and retrieval tasks by 84% and 24%, respectively. Additionally, a subset of Amsterdam landmarks is collected for feature evaluation in a classification task. Classification labels are further used to extract the visual explanations using Grad-CAM for inspection of the learned similar visuals in a deep metric learning models.

@en

Historical photos of towns and villages contain a great deal of information about the built environment of the past. However, it is difficult to evaluate the information of images that are not labeled or incorrectly labeled or not organized in repositories or collections. In order to make the sheer volume of images that are not tagged with metadata found on the Internet or in institutional archives accessible for research, an automated recognition of the image content, in this case of buildings, is necessary. Computer vision can help to address this problem and enable the identification of historical image content. This article describes how artificial intelligence and crowdsourcing are used to identify buildings in nearly half a million historical images of the city of Amsterdam. It explains how computer science and humanities disciplines are linked together to accomplish this task.

@en

The ArchiMediaL project aims to bridge between data science and researches on contemporary and historical built environments by developing state of the art AI algorithms for the automatic linking of available meta-data and image repositories. As a case-study we use the 360,000+ historical images from the Amsterdam Beeldbank database.@en

Built form dominates the urban space where most people live and work and provides a visual reflection of the local, regional and global esthetical, social, cultural, technological and economic factors and values. Street-view images and historical photo archives are therefore an invaluable source for sociological or historical study; however, they often lack metadata to start any comparative analysis. Date and location are two basic annotations often missing from historical images. Depending on the research question other annotations might be useful, that either could be visually derived (e.g. the number or age of cars, the fashion people wear, the amount of street decay) or extracted from other data sources (e.g. crime statistics for the neighborhood where the picture was taken). Recent advances in automatic visual analysis and the increasing amount of linked open data triggered the research described in this paper. We provide an overview of the current status of automated image analysis and linked data technology and present a case study and methodology to automatically enrich a large database of historical images of buildings in the city of Amsterdam. @en

We address the interpretability of convolutional neural networks (CNNs) for predicting a geo-location from an image. In a pilot experiment we classify images of Pittsburgh vs Tokyo and visualize the learned CNN filters. We found that varying the CNN architecture leads to variating in the visualized filters. This calls for further investigation of the effective parameters on the interpretability of CNNs.

@en