This thesis introduces a Learned-Based Multi-View Semantic Stereo method, addressing the limitations of traditional and learned-based Multi-View Stereo (MVS) techniques in reconstructing reflective and low-textured regions, particularly prevalent in 3D models of buildings. Traditional methods lack completeness, while learned-based methods struggle with accuracy. Focusing on enhancing 3D models of buildings, this research integrates semantic information into the existing deep learning architecture for depth prediction, specifically CasMVSNet, to guide the reconstruction process. Three key strategies are employed: first, the incorporation of semantic maps into the network through a multi-modal approach; second, the introduction of a multi-modal refinement module at the end of the CasMVSNet model to improve the initial output depth maps; and third, the introduction of two new loss terms designed to enforce varying degrees of smoothness on specific semantic categories. Experimental results, conducted on the DTU dataset, demonstrate a significant enhancement in accuracy at the point cloud level while maintaining the completeness of the reconstructed models. Validation and generalization on the ETH3D dataset show consistent patterns. This research showcases the potential of integrating semantic guidance in 3D reconstruction of buildings, advancing the field of computer vision.

Currently more than 4 billion people live in urban areas around the globe, a trend that is expected to be increased in the upcoming years. While urbanisation provides the space for innovation and new opportunities, in the meantime physical, technical and social challenges are rising and the cities’ vulnerability is increasing. A tool to tackle these issues are Computational Fluid Dynamics
(CFD) simulations, which can provide insight in various topics.

CFD simulations are valuable for modelling complex urban phenomena such as wind flow, microclimates and thermal comfort. A CFD requires as an input a 3D geometric dataset that represents objects in the urban environment which are most commonly buildings and then according to this input the air flow is simulated around it.

When creating geometries automatically for CFD simulations, several clean up tasks must be completed for them to be usable without any issues. One of the problems arising is related to the redundant faces shared between adjacent buildings, which have no purpose for outdoor flow simulations and cause complications when creating the mesh that is needed for the CFD. This
synthesis project focuses on addressing the aforementioned issue by removing the shared faces.

The ultimate goal of this project was to create an open-source product that can efficiently and in an automated way remove the adjacent faces between buildings. The benefits will be imminent during the meshing process, as we strive to reduce the time that consultancies spend fixing the input geometries before running a CFD simulation, along with an overall improved user experience.

This report is organised in four main sections. The first section is the general introduction of the issue that needs to resolved. The second section defines more in depth the problem and sets the research questions, in accordance to that, in the third section the research methodology is developed. In the fourth section the results of both methods are presented. The fifth sectionfocuses on a reflection of the project, while the sixth section presents the final conclusions. Finally, the seventh section contains the specifics of the project management itself.

The project was carried out in cooperation with Dassault Syst`emes and is developed in the context of the GEO1101 course in MSc Geomatics TU Delft. In addition to this report we have created a GitHub repository (https://github.com/Fabisser/facesBgone) that contains the source code of the two methods.