Temperature plays a critical role in performance and stability of anaerobic digestion processes, subject to frequent meteorological fluctuations. However, state-of-the-art modeling and process control approaches for anaerobic digestion often do not consider the temporal dynamics of the temperature, which can influence microbial communities, kinetics, and chemical equilibrium, and consequently, biogas production efficiency. Therefore, to account for anaerobic digesters operating under fluctuating meteorological conditions, the Anaerobic Digestion Model no. 1 (ADM1) is mechanistically extended in this paper to incorporate temporal changes into temperature-dependent parameters by defining inhibition functions for microbial activities using the cardinal temperature model, and accounting for the lag in microbial adaptation to temperature fluctuations using a time-lag adaptation function. Thereafter, given that temperature fluctuations are a significant disturbance, a control framework based on Model Predictive Control (MPC) is developed to regulate the feeding flow rate and to ensure stable production rates despite temperature disturbances without relying on direct temperature control. An adaptive MPC approach is formulated based on a linear input–output model, where the parameters of the linear model are updated online to capture the nonlinear dynamics of the process and frequent changes in the dynamics accurately. In addition, a fuzzy logic system is employed to assign a reference trajectory for the production rate based on the temperature and its rate of change. Integrating this fuzzy logic system with the MPC controller enhances the production rate on warm days and avoids the operational failure in production on cold days. Additionally, to enhance biogas production rates, the feasibility of utilizing a portion of the produced biogas for external heating purposes is also investigated. It is demonstrated that by utilizing the proposed MPC approach, the additional amount of feed for the digester to produce methane required for a self-consumption biogas-fueled heating system can be calculated according to the meteorological variations. This enhances the process performance and stability. Finally, a thermally optimized dome digester semi-buried in the ground, operating under climate conditions of The Netherlands is considered as a case study to validate the extended model in agreement with biological and physicochemical behaviors of real-world applications, and to demonstrate the effectiveness of the proposed control system in handling temperature changes and enhancing performance.

@en

Greenhouse climate control is concerned with maximizing performance in terms of crop yield and resource efficiency. One promising approach is model predictive control (MPC), which leverages a model of the system to optimize the control inputs, while enforcing physical constraints. However, prediction models for greenhouse systems are inherently inaccurate due to the complexity of the real system and the uncertainty in predicted weather profiles. For model-based control approaches such as MPC, this can degrade performance and lead to constraint violations. Existing approaches address uncertainty in the prediction model with robust or stochastic MPC methodology; however, these necessarily reduce crop yield due to conservatism and often bear higher computational loads. In contrast, learning-based control approaches, such as reinforcement learning (RL), can handle uncertainty naturally by leveraging data to improve performance. This work proposes an MPC-based RL control framework to optimize the climate control performance in the presence of prediction uncertainty. The approach employs a parametrized MPC scheme that learns directly from data, in an online fashion, the parametrization of the constraints, prediction model, and optimization cost that minimizes constraint violations and maximizes climate control performance. Simulations show that the approach can learn an MPC controller that significantly outperforms the current state-of-the-art in terms of constraint violations and efficient crop growth.

@en

This paper presents a novel approach for distributed model predictive control (MPC) for piecewise affine (PWA) systems. Existing approaches rely on solving mixed-integer optimization problems, requiring significant computation power or time. We propose a distributed MPC scheme that requires solving only convex optimization problems. The key contribution is a novel method, based on the alternating direction method of multipliers, for solving the non-convex optimal control problem that arises due to the PWA dynamics. We present a distributed MPC scheme, leveraging this method, that explicitly accounts for the coupling between subsystems by reaching agreement on the values of coupled states. Stability and recursive feasibility are shown under additional assumptions on the underlying system. Two numerical examples are provided, in which the proposed controller is shown to significantly improve the CPU time and closed-loop performance over existing state-of-the-art approaches.

@en

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this article, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate nonconvex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the suboptimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer-linear-programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.

@en

Uncertainty in the behavior of other traffic participants is a crucial factor in collision avoidance for automated driving; here, stochastic metrics could avoid overly conservative decisions. This article introduces a stochastic model predictive control (SMPC) planner for emergency collision avoidance in highway scenarios to proactively minimize collision risk while ensuring safety through chance constraints. To guarantee that the emergency trajectory can be attained, we incorporate nonlinear tire dynamics in the prediction model of the ego vehicle. Further, we exploit max-min-plus-scaling (MMPS) approximations of the nonlinearities to avoid conservatism, enforce proactive collision avoidance, and improve computational efficiency in terms of performance and speed. Consequently, our contributions include integrating a dynamic ego vehicle model into the SMPC planner, introducing the MMPS approximation for real-time implementation in emergency scenarios, and integrating SMPC with hybridized chance constraints and risk minimization. We evaluate our SMPC formulation in terms of proactivity and efficiency in various hazardous scenarios. Moreover, we demonstrate the effectiveness of our proposed approach by comparing it with a state-of-the-art SMPC planner and we validate that the generated trajectories can be attained using a high-fidelity vehicle model in IPG CarMaker.

@en

We consider the growth rate of a switching max-min-plus-scaling (S-MMPS) system in a discrete-event framework. We show that an explicit, time-invariant, monotone, and arbitrarily switching MMPS system has a bounded growth rate. Further, we propose a mixed-integer linear programming problem to calculate the estimates of the smallest upper bound and the largest lower bound of the growth rate of an S-MMPS system.

@en

Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictive control law for constrained linear quadratic regulation (CLQR) problems has been proven to be piecewise affine (PWA) while the value function is piecewise quadratic. We exploit these formal results from MPC to design an ADP method for CLQR problems with a known model. A novel convex and piecewise quadratic neural network with a local–global architecture is proposed to provide an accurate approximation of the value function, which is used as the cost-to-go function in the online dynamic programming problem. An efficient decomposition algorithm is developed to generate the control policy and speed up the online computation. Rigorous stability analysis of the closed-loop system is conducted for the proposed control scheme under the condition that a good approximation of the value function is achieved. Comparative simulations are carried out to demonstrate the potential of the proposed method in terms of online computation and optimality.

@en

In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQG) framework. We introduce a two-phase control algorithm called LQG-NAIVE, which involves an initial phase of injecting Gaussian input signals to obtain a system model, followed by a second phase of an interplay between naive exploration and control in an episodic fashion. We show that LQG-NAIVE achieves a regret growth rate of Õ(√T), i.e., O(√T) up to logarithmic factors after T time steps, and we validate its performance through numerical simulations. Additionally, we propose LQG-IF2E, which extends the exploration signal to a 'closed-loop' setting by incorporating the Fisher Information Matrix (FIM). We provide compelling numerical evidence of the competitive performance of LQG-IF2E compared to LQG-NAIVE.

@en

This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on a numerical example.

@en

Model predictive control (MPC) and deep reinforcement learning (DRL) have been developed extensively as two independent techniques for traffic management. Although the features of MPC and DRL complement each other very well, few of the current studies consider combining these two methods for application in the field of freeway traffic control. This paper proposes a novel framework for integrating MPC and DRL methods for freeway traffic control that is different from existing MPC-(D)RL methods. Specifically, the proposed framework adopts a hierarchical structure, where a high-level efficient MPC component works at a low frequency to provide a baseline control input, while the DRL component works at a high frequency to modify online the output generated by MPC. The control framework, therefore, needs only limited online computational resources and is able to handle uncertainties and external disturbances after proper learning with enough training data. The proposed framework is implemented on a benchmark freeway network in order to coordinate ramp metering and variable speed limits, and the performance is compared with standard MPC and DRL approaches. The simulation results show that the proposed framework outperforms standalone MPC and DRL methods in terms of total time spent (TTS) and constraint satisfaction, despite model uncertainties and external disturbances.@en

The railway timetable rescheduling problem is regarded as an efficient way to handle disturbances. Typically, it is tackled using a mixed integer linear programming (MILP) formulation. In this paper, an algorithm that combines both reinforcement learning and optimization is proposed to solve the railway timetable rescheduling problem. Specifically, a value-based reinforcement learning algorithm is implemented to determine the independent integer variables of the MILP problem. Then, the values of all the integer variables can be derived from these independent integer variables. With the solution for the integer variables, the MILP problem can be transformed into a linear programming problem, which can be solved efficiently. The simulation results show that the proposed method can reduce passenger delays compared with the baseline, while also reducing the solution time.

@en

Purple Phototrophic Bacteria (PPB) are increasingly being applied in resource recovery from wastewater. Open raceway-pond reactors offer a more cost-effective option, but subject to biological and environmental perturbations. This study proposes a hierarchical control system based on Adaptive Generalized Model Predictive Control (AGMPC) for PPB raceway reactors. The AGMPC uses simple linear models updated adaptively to project the complex process dynamics and capture changes. The hierarchical approach uses the AGMPC controller to optimize PPB growth as the core of the system. The developed supervisory layer adjusts set-points for the core controller based on two operational scenarios: maximizing PPB concentration for quality, or increasing yield for quantity through effluent recycling. Lastly, due to competing PPB and non-PPB bacteria during start-up phase, an override strategy for this transition is investigated through simulation studies. The Purple Bacteria Model (PBM) simulates this process, and simulation results demonstrate the control system's effectiveness and robustness.

@en

This paper focuses on the development of linear Switched Box–Jenkins (SBJ) models for approximating complex dynamical models of biological wastewater treatment processes. We discuss the adaptation of these processes to the SBJ framework, showing the model's generality and flexibility as a class of switched systems that can offer accurate predictions for complex and nonlinear dynamics. This approach of modeling enables real-time data reconciliation from experiments and allows the design of model-based control strategies. Through the extension of the Outer Bounding Ellipsoids (OBEs) algorithm, the paper introduces an online two-stage parameter identification algorithm that effectively handles bounded disturbances for SBJ models. Using the OBE method relaxes the stochastic assumptions on disturbances, which may not be satisfied in practice, particularly for biological and environmental fluctuations. The proposed decomposed OBE algorithm separately identifies the switching patterns and parameters of linear submodels, conducting parameter identification in two distinct phases for input/output and disturbance/output submodels. The efficacy of this approach is shown via simulation results validated against both ADM1 and PBM models, demonstrating the proposed algorithm's capability to accurately predict outputs from different biological wastewater treatment models.

@en

Parameterized model predictive control (PMPC) is one of the many approaches that have been developed to alleviate the high computational requirement of model predictive control (MPC), and it has been shown to significantly reduce the computational complexity while providing comparable control performance with conventional MPC. However, PMPC methods still require a sufficiently accurate model to guarantee the control performance. To deal with model mismatches caused by the changing environment and by disturbances, this paper first proposes a novel framework that uses reinforcement learning (RL) to adapt all components of the PMPC scheme in an online way. More specifically, the novel framework integrates various strategies to adjust different components of PMPC (e.g., objective function, state-feedback control function, optimization settings, and system model), which results in a synthesis framework for RL-based adaptive PMPC. We show that existing adaptive (P)MPC approaches can also be embedded in this synthesis framework. The resulting combined RL-PMPC framework provides a solution for an efficient MPC approach that can deal with model mismatches. A case study is performed in which the framework is applied to freeway traffic control. Simulation results show that for the given case study the RL-based adaptive PMPC approach reduces computational complexity by 98% on average compared to conventional MPC while achieving better control performance than the other controllers, in the presence of model mismatches and disturbances.

@en

This paper proposes an approach to find the eigenvalues and eigenvectors of a class of autonomous max-min-plus-scaling (MMPS) systems. First we show that time invariant, monotone and non-expansive MMPS systems with only time variables has a unique structural eigenvalue and eigenvector under some conditions. Then, we propose a mixed integer linear programming (MILP) algorithm to calculate the eigenvalue and the corresponding eigenvector for such systems. Finally, we present a modified linear programming (LP) algorithm to find all the eigenvalues of a general time invariant MMPS system.

@en

In this paper, we discuss the stability of general time-invariant discrete-event systems modelled as max-min-plus-scaling (MMPS) systems. We analyze MMPS systems with two types of states: time states and quantity states. A set of linear programming problems are proposed to find the growth rates of the time states via a normalization of the MMPS system. Then a framework for stability analysis of the general time-invariant MMPS system is discussed with respect to the normalized system. The approach presented in this paper is an efficient way to study the stability of a general MMPS system.

@en

In this article we consider the problem of tether entanglement for tethered mobile robots. One of the main risks of using a tethered connection between a mobile robot and an anchor point is that the tether may get entangled with the obstacles present in the environment or with itself. To avoid these situations, a non-entanglement constraint can be considered in the motion planning problem for tethered robots. This constraint is typically expressed as a set of specific tether configurations that must be avoided. However, the literature lacks a generally accepted definition of entanglement, with existing definitions being limited and partial in the sense that they only focus on specific instances of entanglement. In practice, this means that the existing definitions do not effectively cover all instances of tether entanglement. Our goal in this article is to bridge this gap and to provide new definitions of entanglement, which, together with the existing ones, can be effectively used to qualify the entanglement state of a tethered robot in diverse situations. The new definitions find application in motion planning for tethered robots, where they can be used to obtain more safe and robust entanglement-free trajectories.

@en

Robust object detection is crucial for automating underwater marine debris collection. While supervised deep learning achieves state-of-the-art performance in discriminative tasks, replicating this success on underwater data is challenging. The generalization of these methods suffers due to a lack of available annotated data considering different sources of variation in the unstructured underwater environment and imaging conditions. In this paper, we present the Seaclear Marine Debris Dataset, the first publicly available shallow-water marine debris dataset annotated for instance segmentation/object detection. The dataset contains 8610 images collected using ROVs at multiple locations and with different cameras, annotated for 40 object categories, encompassing not only litter but also observed animals, plants, and robot parts. As part of the technical validation, we provide baseline results for object detection using Faster RCNN and YOLOv6 models. Furthermore, we demonstrate the non-triviality of generalizing the trained model performance to unseen sites and cameras due to domain shift. This underscores the value of the presented dataset in further developing robust models for underwater debris detection.@en

The computational complexity of nonlinear Model Predictive Control (MPC) poses a significant challenge in achieving real-time levels of 4 and 5 of automated driving. This work presents the open-access Hybridization toolbox for MPC (H4MPC), targeting computational efficiency of nonlinear MPC thanks to several modules to hybridize nonlinear MPC optimization problems commonly encountered in automated driving applications. H4MPC is designed as a user-friendly solution with a graphical user interface within the MATLAB environment. The toolbox facilitates intuitive and straightforward customization of the hybridization process for any given function appearing in the equality or inequality constraints within the MPC framework. The initial release, Version 1.0, is freely available from https://bit.ly/H4MPCV1. To provide a clear illustration of the toolbox capabilities, we present two case studies: one to hybridize a vehicle model and another one to approximate tire saturation constraints.@en

In this paper, the disjunctive and conjunctive lattice piecewise affine (PWA) approximations of explicit linear model predictive control (MPC) are proposed. Training data consisting of states and corresponding affine control laws are generated in a control invariant set, and redundant sample points are removed to simplify the construction of lattice PWA approximations. Resampling is proposed to guarantee the equivalence of lattice PWA approximations and optimal MPC control law at the sample points. Under certain conditions, the disjunctive lattice PWA approximation constitutes a lower bound, while the conjunctive version formulates an upper bound of the original optimal control law. The equivalence of the two lattice PWA approximations then guarantees error-free approximations in the domain of interest, which is tested through a statistical guarantee. The performance of the proposed approximation strategy is tested through two simulation examples, and the results show that error-free lattice PWA approximations can be obtained with low offline complexity and small storage requirements. Besides, the online complexity is less compared with the state-of-the-art method.

@en