Hybrid knowledge-based/deep learning reduced order modelling of high dimensional chaotic systems

Abstract

Chaotic systems are widespread and can be found everywhere, from small scale processes inside the human body to the large scale dynamics of the entire atmosphere. However, modelling these high dimensional chaotic systems is a difficult task due to the intrinsic nonlinear nature of chaos as well as the accompanied computational cost. Therefore, in this research, the predictive potential of hybrid reduced order models consisting of a knowledge-based (proper orthogonal decomposition Galerkin projection) and a deep learning reservoir computing component was studied for high dimensional chaotic systems, for which two hybrid architectures were designed. In the first, the blending of the knowledge-based and deep learning components occurred within the reservoir of the data-driven component (Hybrid-A) and its performance was assessed through the short-term prediction horizon and long-term statistical predictions and compared to the pure knowledge-based and pure deep learning reduced order models. For the second model (Hybrid-FFNN), the blending occurred in a separate feed-forward neural network and the performance was compared to the Hybrid-A model. In order to obtain efficient deep learning predictions for both models, a low dimensional latent space representation of the physical system was obtained using autoencoders. Both models were tested on two-dimensional Kolmogorov flow at a Reynolds number of Re = 20 (periodic regime) and Re = 34 (chaotic regime).

For the periodic case, the short-term performance of the Hybrid-A and deep learning model were comparable, while the knowledge-based model was outperformed due to instabilities as a result of the truncation of the number of modes from proper orthogonal decomposition. Furthermore, an increase in latent space and/or deep learning reservoir size had no consistent influence on the performance of the Hybrid-A model, due to the periodic and thus simple to learn dynamics for the deep learning component. Finally, little influence was found for the accumulation of (de)compression errors occurring in the Hybrid-A model, which was compensated for by the additional information from the knowledge-based model and/or the Hybrid-A model was able to restore the errors due to the periodic nature of the system.

Increasing the Reynolds number to the chaotic regime resulted in the Hybrid-A model to underperform compared to the knowledge-based and deep learning models for smaller latent space due to the (de)compression error accumulation. In addition, the Hybrid-A model was unable to perform better than the knowledge-based model for larger latent spaces and number of retained modes, from which it was concluded that too much information on the physical system was lost in latent space for the Hybrid-A model to benefit from both components, while the knowledge-based model operated on the (full) physical system. For the Hybrid-FFNN it was shown that the feed-forward neural network introduced a too large error to be beneficial over the Hybrid-A model.

For the long-term statistics of both test cases, the importance of the design for the tuning of the hyperparameters of the deep learning model/component was showed. In general, no clear influence was found in terms of the performance of the Hybrid-A model as a function of the latent space and reservoir size. Furthermore, the comparative performance of the knowledge-based, deep learning and Hybrid-A models showed no relation. This behaviour was expected to be the result of a design strategy that biased the short-term performance over the long-term performance of the deep learning model/component. Furthermore, for the periodic regime, such behaviour could also have originated from the periodicity of the flow.

Up to the authors knowledge, this work proposed the first hybrid knowledge-based/deep learning reduced order model using autoencoders for efficient predictions of high dimensional chaotic systems. Even though it was found that the proposed Hybrid models were unable to show increased performance compared to the knowledge-based and deep learning models, the results can be viewed as a start to design other hybrid architectures for potential performance improvement.