Objective video quality metrics are designed to be as reliable as the subjective quality assessments on which they are calibrated and validated. However, existing standard methodologies for subjective video quality assessment provide low reliable results for some conditions. We investigate whether an extension of the quality ruler experimental methodology, originally defined for images and shown to be more reliable than, e.g., standard single stimulus (SS) methods, can be adapted to reliably assess the quality of videos. The video quality ruler methodology allows subjects to assess video quality using a set of reference anchor images (the ruler), spanning a wide range of quality altogether, but closely spaced in function of quality one from the other. Subjects are asked to compare the quality of the displayed test video with the quality of these anchor images, displayed on a tablet, and indicate which of the reference images matches in quality the test video. As a result, the video quality assessment task is reduced to a set of visual comparisons between video and reference image quality. We describe how to adapt the original quality ruler methodology to video quality assessment, and we compare the proposed methodology with two other, widely used experimental methodologies: the single stimulus (SS) and the double stimulus (DS) method. Our results show that video quality ruler is a reliable method to assess video quality according to a multitude of criteria.

@en

Understanding the perceptual impact of compression artifacts in video is one of the keys for designing better coding schemes and appropriate visual quality control chains. Although compression and transmission artifacts, such as blockiness, blurriness, and packet-loss, appear simultaneously in digital videos, traditionally they have been studied in isolation. In this paper, we report the results of three subjective quality assessment experiments aimed at studying perceptual characteristics of a set of artifacts common in digital videos. With this goal, first, we study the annoyance of each of three artifacts (blockiness, blurriness, and packet-loss) in isolation and then in combination. Based on the subjective evaluations, we design several models of the annoyance caused by the joint presence of these three artifacts on digital video.@en