In the process of drilling wells to produce hydrocarbons, an exploration strategy is used to determine which wells should be drilled and in which order. This strategy is vital, as a suboptimal drilling sequence will lead to more expenses and fewer gains.
Furthermore, the wells considered in most exploration strategies are geologically
dependent. Thus, a realistic model of these dependencies will be beneficial and
contribute to a more reliable optimal drilling strategy.
Previous research has shown that modelling similarities between the geological
properties of prospect wells in the same region and updating the drilling strategy dynamically after more information is available can add much value. However, the currently developed models are not realistic enough to use in practice. Previous models separated the process into two parts, first modelling the geological success with deterministic rewards and then applying dynamic programming to obtain the sequential drilling strategy.
This thesis proposes a new model with the addition of flexible uncertainty and
dependency models between the geological characteristics in the hydrocarbon volume calculation of a reservoir. For this purpose, non-parametric Bayesian networks in combination with copulae are used. This model and the input format of the required data align with how data is gathered in the oil and gas industry. The dependencies are modelled based on simple expert assessments.
After the new model for the geological properties of prospect wells and their dependencies is constructed, a decision policy that hinges on the observations of previously drilled wells is constructed. Previously used dynamic programming becomes infeasible with continuous variables and with a large number of wells. Therefore, the reinforcement learning algorithm Q-learning is used, which is better equipped to handle these more complex models.
The analysis shows that Q-learning achieves similar results to dynamic programming. Afterwards, the new model is applied to a case based on real oil and gas industry data. The results show that this case does not lend itself to an improvement when the drilling strategy is sequentially updated as the decisions compared to the original drilling strategy did not change. This occurs because the prospects all have high positive expected rewards and should be drilled, regardless of the other wells’ outcomes and dependencies.
Another case with synthetic data is explored with more favourable characteristics
that, in theory, should lead to the result of the policy from the new model and Qlearning outperforming the current industry policy. However, this was not shown yet within the time constraints of this thesis, as the Q-learner should be run for a more extended period and tuned according to the case.
While the model is created for dynamic sequential decision making in the application of drilling wells, this method can easily be adapted to fit any other application where decisions should be updated sequentially, and the
dependencies can be modelled through a Bayesian network.
In conclusion, the key contributions of this thesis are the creation of a new model for the dependencies between geological properties of a hydrocarbon reservoir and the use of the Q-learning reinforcement learning algorithm to compute the sequentially updated drilling strategy as an alternative to dynamic programming.

OSPAR's Commission has been battling eutrophication since the problem was first established in the 1950s. To battle eutrophication, an important factor is to monitor it. Five indicators are used together to assess the status of eutrophication, determined by the Common Procedure. These are the chlorophyll-a concentration, the turbidity, the nitrate and phosphorus concentration, the oxygen levels and the biological water quality. All five indicators need to be known to obtain the final eutrophication status. However, just looking at the chlorophyll-a concentration on its own is also a good measure. This thesis focuses only on the chlorophyll-a concentration as an indicator for eutrophication.

To monitor the North Sea, the OSPAR's Commission has established eutrophication monitoring zones. The aim of this study is to determine eutrophication monitoring zones based on available satellite data of the chlorophyll-a concentration in the Dutch part of the North Sea. The zones are defined using four clustering algorithms: K-means clustering, Hierarchical clustering, Random Forest clustering and HDBSCAN. The results from these clustering algorithms are compared to both each other and to the previously defined eutrophication zones.

First, the case study region is split into two areas: the coastal area, which lies closer to the shore, and the offshore area, which lies farther away from the shore. The best result for this separation was generated by K-means clustering with two clusters.

Afterwards, the eutrophication zones are determined separately in the offshore area and the coastal area. The clustering results are ranked based on four criteria. The first criterion is correspondence to OSPAR's previously defined eutrophication monitoring zones. The second criterion is the similarity of the clusters to the zones that are visible in the data. The third criterion is the performance determined by validation metrics. This criterion was considered less important because of the lack of ability to capture the goals of the research. The last criterion is confirmation through the HDBSCAN clustering. This was added later during the study when it was found that HDBSCAN yielded very accurate results. Due to how HDBSCAN works these accurate results were not usable directly, as the number of clusters this yields it too high, but they were usable for verification. The best results were found through random forest clustering with respectively nine and five clusters for the offshore and coastal areas.

Subsequently, the zones derived from clustering were compared to other data to see whether the determined monitoring zones also hold over time. This appeared to be the case.
Moreover, the distribution of the chlorophyll-a concentration for each zone is determined. Additionally, the trend of the chlorophyll-a concentration of one determined monitoring zone is analysed over time. Lastly, the defined eutrophication monitoring zones are compared to other defined zones within the Dutch North Sea coast. These other zones were fishery policies, marine protected areas, spatial planning, and bathymetry. The comparison validated the defined monitoring zones.