The prevalence of partial differential equations (PDEs) in modeling physics and the low speeds of numerical solvers demands more efficient solving methods. For this purpose, machine learning based methods have been proposed, but these are typically discrete, difficult to interpret and require large amounts of ground truth data to train. To address these issues, we propose a novel machine learning based solver that builds upon advancements in Gaussian based reconstruction. Our method represents the solution to a time-dependent target PDE purely in terms of Gaussians, making it completely continuous and meshless. These Gaussians are evolved by means of an autoregressive neural network that is applied to each Gaussian, integrating information from a local neighborhood of Gaussians. This enables the change of position, scale, rotation and value of the Gaussians to model the solution in a particle-like manner. Extensive experiments show the potential for Gaussians to model arbitrary physical phenomena. We also compare our approach to ground truth data and various state of the art methods, which demonstrates that the method performs well on short-term prediction, but does not match the state of the art for long-term prediction.

Multi-view image recognition is crucial for numerous applications such as autonomous vehicles and robotics, where accurate 3D reconstructions from 2D images are essential. However, the presence of various noise factors like motion blur, variable lighting, and changes in the field of view can significantly degrade the quality of these reconstructions. Through a series of experiments using simulated data created in Blender, I explore the effectiveness of different de-noising methods, including AI-based and traditional image processing techniques. My findings indicate that specific adjustments in the Gaussian Splatting data preprocessing can significantly mitigate the adverse effects of noise, leading to more accurate and reliable 3D reconstructions. The study contributes to the field by providing a detailed analysis of noise impact and proposing viable solutions to enhance the fidelity of multi-view image reconstructions in noisy environments. The code and resources developed for this project are available in the GS-preprocessor repository on GitHub. https://github.com/surftijmen/GS-preprocessor.

In this work, we look at the intersection of Sustainable Software Engineering and AI engineering known as Green AI. AI computing is rapidly becoming more expensive, calling for a change in design philosophy. We consider both training and inference of neural networks used for image vision; to reveal energy-efficient practices in an exploratory fashion.

First of all, we examine a modern algorithm for hyperparameter optimisation and compare this to two baseline methods. We find that the baseline algorithms perform considerably worse despite their wide usage and argue that they should not be used when training large models. Furthermore, we look at the layer structure of convolutional networks and conclude that the convolutional layers have the largest influence on the total consumption. We report increases of up to 95% with only marginal improvements in accuracy. Therefore we recommend developers to reduce their network architectures as long as the performance stays within a reasonable margin.

Second, we present a study focused on the inference phase of the deep learning pipeline. We look at the effect of batching for image classification requests. To facilitate the data collection, we make use of a simulated queue and the Pytorch framework. We find that batching has a significant impact on the energy consumption, but the magnitude of this impact can vary a lot for different models. Our recommendation is to treat the batch size as an inference parameter that needs to be tuned first. Additionally, we highlight how the energy consumption of image vision networks has evolved over the past decade. Presenting the findings together with the performance of these networks shows a steady, upward energy trend accompanied by a decreasing slope for the accuracy. The only exception is the model ShuffleNetV2. We mention the design principles that went into the development of this network and present it as a start for future research.

Diminished reality (DR) is an extension of augmented reality (AR) in which real objects are concealed, removed, or replaced. State of the art DR implementations are written and evaluated on desktop platforms, and those which are aimed at smartphones use auxiliary data or hardware for image completion. This paper introduces a DR app for Android smartphones which can select, track, and remove objects in an unrestricted environment from live video at interactive speed. The application relies exclusively on RGB frames from the phone's camera as input to the DR pipeline. Multiple algorithms for object selection and tracking, as well as a patch-based inpainting approach were adapted. Selection and tracking of objects was very reliable at interactive speed. Inpainting led to convincing results for planar surfaces and unstructured textures, often delivering acceptable results even in more complicated scenes. Future optimizations are required for real-time performance of DR algorithms even on a flagship smartphone, as the hardware was a constraining factor. The performance of inpainting in particular needs to be improved to achieve acceptable frame rates.