Recovering spatially-varying materials from a single photograph of a surface is inherently ill-posed, making the direct application of a gradient descent on the reflectance parameters prone to poor minima. Recent methods leverage deep learning either by directly regressing reflectance parameters using feed-forward neural networks or by learning a latent space of SVBRDFs using encoder-decoder or generative adversarial networks followed by a gradient-based optimization in latent space. The former is fast but does not account for the likelihood of the prediction, i.e., how well the resulting reflectance explains the input image. The latter provides a strong prior on the space of spatially-varying materials, but this prior can hinder the reconstruction of images that are too different from the training data. Our method combines the strengths of both approaches. We optimize reflectance parameters to best reconstruct the input image using a recurrent neural network, which iteratively predicts how to update the reflectance parameters given the gradient of the reconstruction likelihood. By combining a learned prior with a likelihood measure, our approach provides a maximum a posteriori estimate of the SVBRDF. Our evaluation shows that this learned gradient-descent method achieves state-of-the-art performance for SVBRDF estimation on synthetic and real images.@en

Plastic pollution in the sea is an environmental hazard, negatively impacts marine life, and causes economic damage all over the world. It is estimated that each year 8 million tonnes of plastic are deposited in seas, the vast majority coming from rivers. In recent years, publicly available satellite imagery has been used to attempt to track floating plastic debris, using specialized hand-crafted features. In this work, we present an automatic learning approach based on satellite imagery that can detect floating plastic debris in rivers with high precision. This approach is based on well-proven image segmentation architectures, U-Net (RonnebeRger et al., 2015) and DeeplabV3+ (Chen et al., 2018), which we adapt to process high-dimensional multispectral images. To train and test the approach, we also present a dataset of images from different rivers around the world containing floating plastic debris, which is a key step to creating an automated learning solution. We test the predictive accuracy of our network, showing that our approach can correctly identify floating debris in images from regions not seen in the training set. Our results also show that a more extensive labeled dataset is necessary to generalize the approach to some types of rivers. Furthermore, we also demonstrate how our solution can also be used to monitor single areas over time to understand and predict floating debris accumulation.

@en

Detailed shadows in large-scale environments are challenging. Our approach enables efficient detailed shadow computations for static environments at a low memory cost. It builds upon compressed precomputed multiresolution hierarchies but uses a directed acyclic graph to encode its tree structure. Further, depth values are compressed and stored separately and we use a bit-plane encoding for the lower tree levels entries in order to further reduce memory requirements and increase locality. We achieve between 20% to 50% improved compression rates, while retaining high performance.@en

Ambient occlusion (AO) is a popular rendering technique that enhances depth perception and realism by darkening locations that are less exposed to ambient light (e.g., corners and creases). In real-time applications, screen-space variants, relying on the depth buffer, are used due to their high performance and good visual quality. However, these only take visible surfaces into account, resulting in inconsistencies, especially during motion. Stochastic-Depth Ambient Occlusion is a novel AO algorithm that accounts for occluded geometry by relying on a stochastic depth map, capturing multiple scene layers per pixel at random. Hereby, we efficiently gather missing information in order to improve upon the accuracy and spatial stability of conventional screen-space approximations, while maintaining real-time performance. Our approach integrates well into existing rendering pipelines and improves the robustness of many different AO techniques, including multi-view solutions.@en

Texture is a key characteristic in the definition of the physical appearance of an object and a crucial element in the creation process of 3D artists. However, retrieving a texture that matches an intended look from an image collection is difficult. Contrary to most photo collections, for which object recognition has proven quite useful, syntactic descriptions of texture characteristics is not straightforward, and even creating appropriate metadata is a very difficult task. In this paper, we propose a system to help explore large unlabeled collections of texture images. The key insight is that spatially grouping textures sharing similar features can simplify navigation. Our system uses a pre-trained convolutional neural network to extract high-level semantic image features, which are then mapped to a 2-dimensional location using an adaptation of t-SNE, a dimensionality-reduction technique. We describe an interface to visualize and explore the resulting distribution and provide a series of enhanced navigation tools, our prioritized t-SNE, scalable clustering, and multi-resolution embedding, to further facilitate exploration and retrieval tasks. Finally, we also present the results of a user evaluation that demonstrates the effectiveness of our solution.

@en

Sparse Voxel Directed Acyclic Graphs (SVDAGs) losslessly compress highly detailed geometry in a highresolution binary voxel grid by identifying matching elements. This representation is suitable for highperformance real-time applications, such as free-viewpoint videos and high-resolution precomputed shadows. In this work, we introduce a lossy scheme to further decrease memory consumption by minimally modifying the underlying voxel grid to increase matches. Our method efficiently identifies groups of similar but rare subtrees in an SVDAG structure and replaces them with a single common subtree representative. We test our compression strategy on several standard voxel datasets, where we obtain memory reductions of 10% up to 50% compared to a standard SVDAG, while introducing an error (ratio of modified voxels to voxel count) of only 1% to 5%. Furthermore, we show that our method is complementary to other state of the art SVDAG optimizations, and has a negligible effect on real-time rendering performance.@en

We introduce the Reduced Immersed Method (RIM) for the real-time simulation of two-way coupled incompressible fluids and elastic solids and the interaction of multiple deformables with (self-)collisions. Our framework is based on a novel discretization of the immersed boundary equations of motion, which model fluid and deformables as a single incompressible medium and their interaction as a unified system on a fixed domain combining Eulerian and Lagrangian terms. One advantage for real-time simulations resulting from this modeling is that two-way coupling phenomena can be faithfully simulated while avoiding costly calculations such as tracking the deforming fluid-solid interfaces and the associated fluid boundary conditions. Our discretization enables the combination of a PIC/FLIP fluid solver with a reduced-order Lagrangian elasticity solver. Crucial for the performance of RIM is the efficient transfer of information between the elasticity and the fluid solver and the synchronization of the Lagrangian and Eulerian settings. We introduce the concept of twin subspaces that enables an efficient reduced-order modeling of the transfer. Our experiments demonstrate that RIM handles complex meshes and highly resolved fluids for large time steps at high framerates on of-the-shelf hardware, even in the presence of high velocities and rapid user interaction. Furthermore, it extends reduced-order elasticity solvers such as Hyper-Reduced Projective Dynamics with natural collision handling.

@en

Shadow volumes are a popular technique to compute pixel-accurate hard shadows in 3D scenes. Many variants exist that trade off accuracy and efficiency. In this work, we present an artifact-free, efficient, and easy-to-implement stencil shadow volume method. We compare our method to established stencil shadow volume techniques and show that it outperforms the alternatives.

@en

Over the years, technological improvement has led to the ability to acquire, create and store vast amounts of geometrical and appearance data to use in graphical applications. Nevertheless, efficiently creating images when using such large amounts of data remains an ongoing topic of research, given the computational resources required. This dissertation will focus on a particular kind of algorithms in order to tackle this problem: image-based representations. Entities represented in a virtual 3-dimensional world can be projected into a regular 2-dimensional grid to form an image. This results in a much more compact, albeit incomplete, representation of the original information, since an image is a piece-wise constant discretization of the underlying data. Nevertheless, computing properties of the 3-dimensional data via its 2-dimensional projection is much more efficient. Still, it is important to understand when and how to use these types of representations, since the discretization of the original data can lead to artifacts in the computed results. This work will focus on three key aspects of computer-graphics algorithms where using appropriate image-based representations can result in increased performance: memory requirements, computational efficiency, and interactivity. To do so, it will describe image-based solutions to different relevant problems in the field of computer graphics. Firstly, by exploring how using 2-dimensional intermediate representation can reduce memory requirements for storing large static shadow information in comparison to state-of-the-art voxel representations. Secondly, a hierarchical image-based approach for efficiently computing diffraction patterns will be demonstrated, which can outperform FFT-based solutions. Lastly, an efficient interactive optimization method for editing disparity when creating stereographic images will be described. These algorithms will be described and evaluated in detail, in order to give the reader insight into the usage of image-based representations.@en

Stereoscopic 3D technology gives visual content creators a new dimension of design when creating images and movies. While useful for conveying emotion, laying emphasis on certain parts of the scene, or guiding the viewer's attention, editing stereo content is a challenging task. Not respecting comfort zones or adding incorrect depth cues, for example depth inversion, leads to a poor viewing experience. In this paper, we present a solution for editing stereoscopic content that allows an artist to impose disparity constraints and removes resulting depth conflicts using an optimization scheme. Using our approach, an artist only needs to focus on important high-level indications that are automatically made consistent with the entire scene while avoiding contradictory depth cues and respecting viewer comfort.

@en

We introduce a variational approach for modeling n-symmetry vector and direction fields on surfaces that supports interpolation and alignment constraints, placing singularities and local editing, while providing real-time responses. The approach is based on novel biharmonic and m-harmonic energies for n-fields on surface meshes and the integration of hard constraints to the resulting optimization problems. Real-time computation rates are achieved by a model reduction approach employing a Fourier-like n-vector field decomposition, which associates frequencies and modes to n-vector fields on surfaces. To demonstrate the benefits of the proposed n-field modeling approach, we use it for controlling stroke directions in line-art drawings of surfaces and for the modeling of anisotropic BRDFs, which define the reflection behavior of surfaces.

@en

In this tutorial, we discuss voxel DAGs and multiresolution hierarchies, which are representations that can encode large volumes of data very efficiently. Despite a significant compression ration, an advantage of these structures is that their content can be efficiently accessed in real-time. This property enables various applications. We begin the tutorial by introducing the concepts of sparsity and of coherency in voxel structures, and explain how a directed acyclic graph (DAG) can be used to represent voxel geometry in a form that exploits both aspects, while remaining usable in its compressed from for e.g. ray casting. In this context, we also discuss extensions that cover the time domain or consider an advanced encoding strategies exploiting symmetries and entropy. We then move on to voxel attributes, such as colors, and explain how to integrate such information with the voxel DAGs. We will provide implementation details and present methods for efficiently constructing the DAGs and also cover how to efficiently access the data structures with e.g. GPU-based ray tracers. The course will be rounded of with a segment on applications. We highlight a few examples and show their results. Pre-computed shadows are a special application, which will be covered in detail. In this context, we also explain how some of previous ideas contribute to multi-resolution hierarchies, which gives an outlook on the potential generality of the presented solutions.@en

Far-field diffraction can be evaluated using the Discrete Fourier Transform (DFT) in image space but it is costly due to its dense sampling. We propose a technique based on a closed-form solution of the continuous Fourier transform for simple vector primitives (quads) and propose a hierarchical and progressive evaluation to achieve real-time performance. Our method is able to simulate diffraction effects in optical systems and can handle varying visibility due to dynamic light sources. Furthermore, it seamlessly extends to near-field diffraction. We show the benefit of our solution in various applications, including realistic real-time glare and bloom rendering.@en

We propose a framework for the spectral processing of tangential vector fields on surfaces. The basis is a Fourier-type representation of tangential vector fields that associates frequencies with tangential vector fields. To implement the representation for piecewise constant tangential vector fields on triangle meshes, we introduce a discrete Hodge–Laplace operator that fits conceptually to the prominent cotan discretization of the Laplace–Beltrami operator. Based on the Fourier representation, we introduce schemes for spectral analysis, filtering and compression of tangential vector fields. Moreover, we introduce a splinetype editor for modeling of tangential vector fields with interpolation constraints for the field itself and its divergence and curl. Using the spectral representation, we propose a numerical scheme that allows for real-time modeling of tangential vector fields.@en

The quality of shadow mapping is traditionally limited by texture resolution. We present a novel lossless compression scheme for high-resolution shadow maps based on precomputed multiresolution hierarchies. Traditional multiresolution trees can compactly represent homogeneous regions of shadow maps at coarser levels, but require many nodes for fine details. By conservatively adapting the depth map, we can significantly reduce the tree complexity. Our proposed method offers high compression rates, avoids quantization errors, exploits coherency along all data dimensions, and is well-suited for GPU architectures. Our approach can be applied for coherent shadow maps as well, enabling several applications, including high-quality soft shadows and dynamic lights moving on fixed-trajectories.
@en

Multiresolution Hierarchies (MH) and Directed Acyclic Graphs (DAG) are two recent approaches for the compression of high-resolution shadow information. In this paper, we introduce Merged Multiresolution Hierarchies (MMH), a novel data structure that unifies both concepts. An MMH leverages both hierarchical homogeneity exploited in MHs, as well as topological similarities exploited in DAG representations. We propose an efficient hash-based technique to quickly identify and remove redundant subtree instances in a modified relative MH representation. Our solution remains lossless and significantly improves the compression rate compared to both preceding shadow map compression algorithms, while retaining the full run-time performance of traditional MH representations. @en

Photorealistic modeling and rendering of materials with complex internal mesostructure is a hard challenge in Computer Graphics. In particular, macroscopic porous materials consist of complex translucent substances that exhibit different details and light interaction at several different scales. State-of-the-art techniques for modeling porous materials manage the material either as a surface and set up complex capture procedures or as a volume by employing different instances of procedural noise models for its representation. While the surface solution achieves several desired material properties, it still presents drawbacks in practical applications—high computational costs, complex capture procedures, and poor image variability, among others. Volumetric solutions are more flexible, but the final structure and appearance are difficult to control. To overcome these drawbacks, we propose an algorithm for the procedural generation of porous materials. The method is based on an artistic and physically inspired simulation of the growth of self-avoiding bubbles inside a volume, by means of dynamical systems. The patterns induced by the bubbles can be easily and intuitively controlled. The bubbles adapt to any given shape and have convincing global and local fluid-like patterns as seen in bread and sponges. Our method generates 3D textures that adequately represent porous materials, which can be used as input for creating realistic renderings of different porous objects. As a case study, we present the results of using these 3D textures as input to a direct volume renderer and show that they compare favorably with standard 3D texture synthesis methods.@en

We accelerate volumetric obscurance, a variant of ambient occlusion, and solve undersampling artifacts, such as banding, noise or blurring, that screen-space techniques traditionally suffer from. We make use of an efficient statistical model to evaluate the occlusion factor in screen-space using a single sample. Overestimations and halos are reduced by an adaptive layering of the visible geometry. Bias at tilted surfaces is avoided by projecting and evaluating the volumetric obscurance in tangent space of each surface point. We compare our approach to several traditional screen-space ambient obscurance techniques and show its competitive qualitative and quantitative performance. Our algorithm maps well to graphics hardware, does not require the traditional bilateral blur step of previous approaches, and avoids typical screen-space related artifacts such as temporal instability due to undersampling.@en