Scene Editing using Polarization

Abstract

In the field of computer vision, the difficulty of processing the illumination in an arbitrary scene creates complications for a machine's understanding of the scene. Computer vision algorithms commonly use input images acquired by a camera, where the only measured quantity is the amount of light the camera's sensor receives without any additional information about the shadows, occlusion, highlights, light directionality and interreflections, to help with scene processing and understanding. This thesis applies polarization state measurements of the camera's incoming light to implicitly extract information on these illumination phenomena through a polarization-based intrinsic image decomposition. By defining and measuring the polarization state parameters of an image in the form of polarization cosines, the diffuse and specular intrinsic images can be estimated through an iterative local optimization algorithm to give additional information on the scene illumination. The decomposition algorithm is qualitatively evaluated on three image sample sets, which show that the algorithm performs well as long as the measurements are aligned. The diffuse and specular intrinsic images can then be used to improve existing scene post-processing operations without requiring additional illumination information. One of the three image sample sets is used to evaluate the effects of the decomposition on texture modification, glossiness remapping and tonemapping. The results indicate that the three operations can be either simplified or can have their quality improved with the additional information supplied by the diffuse and specular intrinsic images.