Benchmarks have the power to bring research communities together to focus on specific research challenges. They drive research forward by making it easier to systematically compare and contrast new solutions, and evaluate their performance with respect to the existing state of the art. In this chapter, we present a retrospective on the Placing Task, a yearly challenge offered by the MediaEval Multimedia Benchmark. The Placing Task, launched in 2010, is a benchmarking task that requires participants to develop algorithms that automatically predict the geolocation of social multimedia (videos and images). This chapter covers the editions of the Placing Task offered in 2010–2013, and also presents an outlook onto 2014. We present the formulation of the task and the task dataset for each year, tracing the design decisions that were made by the organizers, and how each year built on the previous year. Finally, we provide a summary of future directions and challenges for multimodal geolocation, and concluding remarks on how benchmarking has catalyzed research progress in the research area of geolocation prediction for social multimedia.@en

This paper provides a description of the MediaEval 2013 Placing Task. The primary task of location estimation asks participants to place images on the world map, that is, to automatically estimate the latitude/longitude coordinates at which a photograph was taken. The newly introduced secondary task of placeability prediction asks participants to estimate the error of their predicted location. Annotating images with this kind of geographical location tag, or geotags, has a number of applications in personalization, recommendation, crisis management and archiving. Currently, the vast majority of images online are not labelled with this kind of data. This task encourages participants to find innovative ways of automatically geo-labelling images while at the same time providing a measure of their algorithm’ accuracy. This year’s data were drawn from Flickr. In comparison to previous editions of this task, the test set has not only increased drastically in size but has also been derived according to different assumptions in order to model a more realistic use-case scenario.@en