In many fields we are interested in inference for a complex stochastic process given limited observations regarding its state over time. This thesis therefore introduces an expectation propagation approach to backward filtering forward guiding for high-dimensional finite-state space models. The backward filtering forward guiding method is first derived for such models with a specific emphasis on the temporal dynamics, after which factorised guiding terms which exploit the inherent structure of the latent state space are introduced.

Performance of the method is assessed by comparing numerical results for statistical inference of a Susceptible-Infected-Recovered example problem. The expectation propagation approach performs comparably with existing methods in a particularly simple setting where state variables are observed individually, and performs very well in a more difficult setting which the familiar methods can not deal with. We conclude that a more advanced treatment of the approximate likelihood filtering phase may be warranted in such complex settings.

The research summarised in this work provides a first effort towards the development of more general expectation propagation based backward filtering procedures for other types of high-dimensional sequential data models. The work additionally elucidates connections between backward filtering with backward marginalisation and alternative approximate likelihood filtering procedures, suggesting multiple avenues for future research.

The breadth of theoretical results on efficient Markov Chain Monte Carlo (MCMC) sampling schemes on discrete spaces is slim when compared to the available theory for MCMC sampling schemes on continuous spaces. Nonetheless, in [Zan17] a simple framework to design Metropolis-Hastings (MH) proposal kernels that incorporate local information about the target is presented. The class of functions for which the resulting MH kernels are Peskun optimal in high-dimensional regimes is characterized. We will refer to these functions as \textit{balancing functions} and to the class of resulting MH proposal kernels as \textit{pointwise informed proposals}. In [PG19], the class of balancing functions is used to construct Markov Jump Processes (MJP) on discrete state spaces. As a result, the Zanella process is constructed. In the absence of a theoretical result on the optimal balancing function to choose from the class of balancing functions, a heuristic approach is proposed using the Zanella process. To further encourage the mixing behaviour of the simulated chain, the algebraic structure of the state space is exploited to achieve non-reversible Markov chains on short to medium timescales. Simulations are performed for all the considered MCMC sampling schemes by studying the Bayesian record linkage problem.

In anti-cancer therapy, ntiangiogenic treatments are applied and take effect on the vascularization of tissue. To evaluate the efficacy of treatments, we adopt two methods to solve the physiological pharmacokinetic model’s parameter estimation problem, providing discrete, partial, and noisy observations of stochastic differential equations. One is to compute the exact likelihood using the Kalman filter recursion and implement numerical maximization. The other is a novel Markov Chain Monte Carlo algorithm to estimate parameters using guided proposals in a Bayesian setup, namely Backward Filtering with Forward Guiding algorithm. The identifiability of the model and parameters are established before parametric inference. We extend the BFFG algorithm to include an automatic optimal kernel finding scheme for the Metropolis-Hastings-within-Gibbs sampler. In comparison, a Conjugate Gradient algorithm is applied when employing the maximum likelihood method. Besides performing parameter estimation via different methods separately, joint estimation is performed using the Bayesian approach. After that, time delay and Arterial Input Function in the statistical model are estimated via change point detection and piecewise inference. We illustrate the goodness-of-fit of estimates and advantages of the bayesian approach towards the maximum likelihood method.

The derivation of water quality indicators is of importance, especially in coastal areas, as most of the economic activities are located here. However, the availability of high-spatial-resolution water quality information in coastal zones is limited. Nowadays, high-resolution satellite data is becoming available and can fill in this knowledge gap. This satellite data contains spectral reflectances, so a model needs to be designed to map these reflectances to water quality indicators. In this thesis, a Gaussian process regression (GPR) method will be introduced and analyzed extensively in terms of covariance functions, hyperparameters and computational costs. Remote sensing data is collected from the Sentinel-2 mission and the in-situ data is obtained from the ODYSSEA programme. The Matérn 3/2 kernel produces the best results and these are compared with the current models that rely on machine learning techniques. GPR shows promising results in terms of estimation accuracy and chlorophyll-a maps are made for different areas and depths. Various approximation methods are tested to speed up the computation time. Singular value decomposition shows promising results for doing predictions to reduce the computation time. Moreover, GPR can handle limited availability of in-situ data well and uncertainty quantification is induced by the Bayesian framework.

Dit project vergelijkt drie verschillende kernels (Random Walk, Langevin en Barker) die gebruikt kunnen worden in het Metropolis-Hastings algoritme aan de hand van voorbeelden.

The development of computationally ecient algorithms for statistical inference of stochastic dierential equations has been a long-standing research subject ever since the advent of stochastic analysis. Recently, there has been an increasing interest in extending these methods to the statistical study of stochastic partial dierential equations. In this thesis, we are concerned with statistical inference for the linear stochastic convection-diusion equation (SCDE) driven by a stochastic forcing term that is spatially a Matern process. The approach is based on spectral methods for partial dierential equations and approximates a solution to the SCDE via a Fourier spectral decomposition. The resulting spectral processes are given by a family of uncoupled Ornstein-Uhlenbeck processes, for which computationally ecient statistical inference is possible based on the Kalman lter. We give veriable experimental results for all statistical problems - ltering, smoothing and parameter estimation - based on simulated data. We further derive a novel weak solution to the SCDE driven by spatial Matern noise for spatially periodic boundary conditions and show that the solution is indeed approximated by the Fourier spectral decomposition, thereby validating the statistical model. For the heat equation, we generalize the spectral ltering approach on compact Riemannian manifolds. Experimental results for this generalization are given on the two-dimensional unit sphere.

In this bachelor thesis we propose a spatial Markov Chain Cellular Automata
model for the spread of the COVID-19 virus as well as two methods for parameter
estimation. Network topologies are used to model the progression of the epidemic
by considering each individual on a grid and using stochastic principles to determine the transition between different states. The model is able to predict the
time-evolution of outbreaks under different lockdown policies. Additionally, the
impact of variation in infection probability and recovery rates on the amount of
active cases, deaths as well as the length of the epidemic is investigated. These
results can provide us with insights and predictions of the spread of the virus under different scenarios. Parameter estimation is done by using both Maximum likelihood estimation and Bayesian estimation based on simulated data. The produced estimates were relatively accurate, suggesting that these methods can be applied in order to estimate the parameters of the proposed model based on actual data.

The goal of this thesis is to estimate parameters in a bidimensional Ornstein-Uhlenbeck process, namely a diffusion model which can be found in Favetto and Samson (2010), which considers plasma and interstitium concentrations. We first look at a general linear stochastic differential equation and some properties. Then we simulate possible paths and observations based on the diffusion model, and derive the underlying state space model. We consider state estimation, where we use the Kalman filter and smoother algorithm. It turns out that the smoother outperforms the Kalman filter. Next, we apply and derive the Liu and West filter for state estimation. We see that the performance for the Liu and West filter is close to the Kalman filter. Furthermore, we look at the parameter estimation. We again use and derive the Liu and West filter, but now for parameter estimation. We first apply this filter to a linear AR(1) model, which gives good results. Then we start with estimating one parameter in the diffusion model and finally we estimate all 7 parameters in the model, using priors concentrated around the true value and 10.000 particles. For one parameter, we obtain good results. For all 7 parameters, results are satisfactory, with for fully observed data better results than for partially observed data. We also look at the influence of the number of particles: for 5.000 particles estimation results are somewhat worse than for 10.000 particles. Furthermore, we change some priors such that they are no longer concentrated around the true value. From this, we see that the Liu and West filter does not seem to perform as well for certain choices of priors. Next, we look at state and parameter estimation in a non-linear AR(1) model using the Liu and West filter, which gives rather good results. Finally, we apply the Liu and West filter once more, but now with a real data set. In this case we are able to obtain parameter values that provide a reasonable estimate for the sum of the concentrations, but for the interstitium concentration it leaves much uncertainty.

In research there is often a need to choose between multiple competing models. Two popular criteria for model selection are the AIC and BIC. The AIC excels in estimating the best model for the unknown data generating process. The BIC on the other hand is consistent in finding the true model. It is clear that for model selection these two information criterion give answers to different selection criteria. The question that arises is whether it is possible to construct a model selection criterion which combines the strengths of both AIC and BIC. In this study we will show that it is impossible to construct a model selection criterion which shares the above mentioned two strenghts by revisiting the proof of \cite{yang2005can} : That is, any consistent model selection criterion must be sub-optimal in the minimax convergence rate for regression estimation compared to the AIC.

When data in higher dimensions with a certain constraint on it, say a set of locations on a sphere, is encountered, some classical statistical analysis methods fail, as the data no longer assumes its values in a linear space. In this thesis we consider such datasets and aim to do likelihood-based inference on the center of the data. To model the nonlinearity, we consider the data to be a set of points on a Riemannian manifold. The general approach in this thesis comes from the classical result where the center can be repre- sented as the maximum likelihood estimator for the true mean of the dataset. To model an underlying distribution we will model the data as observations of realizations of Brownian motion on the manifold observed at a fixed time and use the transition density of the Brownian motion to construct a likelihood. The likelihood can then be approximated using diffusion bridges. This thesis thus first focuses on differential geometry as well as Itô and Stratonovich calculus. After that, we will introduce methods to construct a likelihood for the center of the dataset on a manifold before using simulated diffusion bridges to approximate this likelihood. We finish the thesis with some numerical experiments in Julia that demonstrate the results on the sphere.

Dit verslag is opgedeeld in twee delen. Middels het eerste deel is onderzocht
of het mogelijk is om synchroniciteit aan te tonen tussen grijze dolfijnen met
behulp van Cross-Recurrence Quantification Analysis. In hoofdstuk 2 wordt
gekeken naar de data die door Nova-Atlantis vergaard is. Op het eerste oog was
al te zien dat er synchroniciteit moest zijn tussen bepaalde dolfijnen, de vraag
echter bleef of dit ook met CRQA mogelijk was. Hiervoor is, middels hoodstuk
3, gekeken naar de recurrente toestanden tussen de tijdreeksen.
Vervolgens is in hoofdstuk 4 gekeken hoe deze recurrente toestanden, via een
recurrentie-matrix, omgezet konden worden naar Cross-Recurrence Plot. Met
behulp van de CRP kan voor elk tweetal dolfijnen bepaald worden of ze gepaard
waren of niet. De CRP van een tweetal dolfijnen geeft niet alleen inzicht in
hun gelijktijdigheid, maar ook in de manier waarop ze ten opzichte van elkaar
zwemmen.
Voor de gelijktijdigheid is dit echter minder van belang. Daarom is in sectie 4.3
gekeken naar de lag-time profielen, een ander facet van CRQA. Deze geven een
globaler overzicht wat er precies gebeurt met het aantal recurrente toestanden
tussen de twee tijdreeksen, op verschillende vertragingen. In feite is het niets
meer dan een generalisering van de diagonalen van de CRP. Dit maakt het dan
ook uitermate geschikt om gelijktijdigheid mee te onderzoeken.
In hoofdstuk 5 worden de bevindingen in 4 gekwantificeerd. Met twee indicatoren
van de lag-time profielen, max RRts en piek-lag max is dit al mogelijk.
Van elk dolfijnenpaar zijn de indicatoren uitgerekend en hier is een heat map
voor opgesteld. Hiermee is snel inzichtelijk gemaakt welke dolfijnenparen en tripletten
wel en niet synchroniciteit vertonen. Bij twijfel wordt de desbetreffende
CRP bekeken om synchroniciteit te verifiëren.
Hoofdstuk 6 geeft antwoord op de onderzoeksvraag en bevat de verdere conclusies
van het onderzoek. In hoofdstuk 7 wordt een laatste keer kritisch gekeken
naar de uiteindelijke conclusies van het onderzoek en worden de nog onbeantwoorde
vraagstukken aangekaart.
In het tweede deel is vanuit een meer theoretisch oogpunt gekeken naar het
toepassen van CRQA op continue tijdreeksen. In hoofdstuk 2 worden twee
continue tijdreeksen met elkaar vergeleken, op dezelfde manier als in deel I is
gebeurd. Middels de recurrentie-matrix wordt gekeken hoe de continue versie
van een CRP wordt geconstrueerd, daarna wordt nog even kort in gegaan op de
lag-time profielen.
Vervolgens is in hoofdstuk 3 gekeken naar het construeren van faseruimte paden
(die we in dit verslag kortweg paden noemen) uit de tijdreeksen en hoe dit effect
heeft op de CRP en andere facetten van CRQA. Dit gaat via de Time-delay methode,
de algemene methode voor het implementeren van tijdreeksen in CRQA.
In het slotwoord wordt het tweede deel gerelateerd aan het eerste deel om zo de
voor en nadelen te kunnen onderscheiden.

The rapid and uncertain penetration of distributed energy resources is changing the way people are consuming electricity. This development increases the risk of instability and congestion in the grid, posing great challenges to system operators in the near future. In order to optimally allocate resources and plan grid enhancement, distribution system operators need new tools to monitor the local energy transition and assess its future impact. Smart meter data is an important resource in gaining this insight, but unlocking the full potential requires new analytical methodologies. Recent research focused on identifying typical daily energy consumption patterns - load shapes - but a clear interpretation of these results is lacking. This research proposes a latent lifestyle model, which models energy consumption as a mixture of different lifestyles, each of which is defined by a distribution over load shapes. This extra layer of abstraction proves to be an effective way of identifying lifestyle-like patterns, that allow for a clear interpretation. Consumers can then be grouped based on their mixture of inferred lifestyles, serving as an input for grid simulation. Such a simulation showed that the all-electric homes cause the aggregated load to nearly triple compared to conventional consumers, due to their increased peak demand and simultaneity.

Free competition in the insurance markets increases the competitiveness and lowers the premiums. If insurers lower their premiums without having a model that accurately quantifies the expected claim size, they can be in serious trouble. This research aims to accurately model the premiums and quantify the uncertainty involved using historic claims data from an insurer. The current approach, Generalized Linear Models (GLMs), is compared to some Machine Learning techniques: Random Forests (RFs) and Gradient Boosting Machines (GBMs). Insights gained from these models and other methods (MARS) are then used to improve the GLMs. Bayesian Additive Regression Trees (BART) and Hierarchical Models (HMs) are then used to quantify the uncertainty. HMs provides the insurer with the means to make proper credible intervals for the total expected claim size of the active portfolio. The HMs also allow the use of risk premium principles that include measures of uncertainty in the pricing of premiums. All relevant models are applied on the dataset of the holdout year. It is apparent that the GLMs and HMs provide too low estimations of the premiums when the profit is tracked. It is therefore prudent to either use the HM with a risk premium principle that incorporates a percentage of the standard deviation in the estimation of the premiums or apply the RF model. The model lift shows that the Machine Learning techniques are better at recognising the risky policies from the non-risky. We recommend the insurer to use RFs to price the premiums and HMs to measure the uncertainty of the active portfolio. It is recommended for further study to either apply other techniques to further improve the predictive performance, to improve the structure of the Hierarchical Model or to include left truncation and right censoring into the model.

In dit verslag zullen drie verschillende GARCH modellen worden onderzocht,
namelijk het univariate GARCH model, het multivariate CCC model en het
multivariate DCC model. GARCH modellen zijn modellen die de volatiliteit
van tijdreeksen kunnen voorspellen. De drie modellen zullen uitgebreid besproken worden, er zullen simulaties mee worden gemaakt en er zal met deze
modellen voorspeld worden.
Bij univariate GARCH modellen wordt er slechts naar één tijdreeks gekeken,
terwijl er bij multivariate GARCH modellen naar meerdere tijdreeksen tegelijk
wordt gekeken waarbij de covariantie tussen deze tijdreeksen wordt meegerekend. Als meerdere tijdreeksen, die niet onafhankelijk van elkaar bewegen, voorspeld zouden worden met het multivariate CCC model of het multivariate DCC model dan zou dit kunnen leiden tot betere predicties vergeleken met de predicties van het univariate GARCH model.
De onderzoeksvraag is als volgt: Zijn er verschillen in de predicties van tijdreeksen, die niet onafhankelijk van elkaar bewegen, verkregen met het univariate GARCH model, het multivariate CCC model en het multivariate DCC
model?