Spatiotemporal systems are systems whose dynamics depend on time and space and are commonly found in real life. These systems are mathematically modeled using partial differential equations and are also known as distributed-parameter systems. Due to their structure and the high number of variables involved, control and estimation for this class of systems are very challenging. This thesis addresses two problems related to spatiotemporal systems: state estimation and system identification.

Monitoring the states of a control system is important to ensure the behavior of the system is achieving the control objectives. This can be achieved, among others, by using state observers that estimate the states of the systems regularly. First, we present a literature review of observer design methods for distributed parameter systems. In general, the design requires a dimension-reduction approach to implement the observer. From the dimension reduction, the design approaches can be classified into late and early lumping. In the late lumping perspective, model reduction is performed at the end of the observer design. In the early lumping perspective, dimension reduction is applied to the model of the system. We incorporate both approaches in our literature review.

State observer design requires the model of the systems. This thesis also presents a system identification method for distributed-parameter systems. The identification of such systems typically requires spatially dense and regular measurements, followed by selecting sensors that provide significant measurements to the model to reduce the model complexity. However, these requirements may be challenging to fulfill. In case the sensor locations are irregular and sparse in space, we propose the use of lumped-parameter system identification.

For models with a large number of regressors, we propose a method for reducing the number of regressors using a tree representation. The tree is a way to list models with different numbers of regressors. From all possible regressors for the model, the proposed method builds the tree from the simplest models, i.e., models with one regressor. The number of regressors in the models is incrementally increased to one or more models with the best performance. The addition is repeated until the tree contains models with the desired maximum number of regressors.

System identification is typically performed using a complete data set, i.e., for each input sample, there is an associated output sample available. However, there are cases in which some output samples are not recorded in the data set, making the identification data incomplete. This thesis also considers the problem of incomplete data for Takagi-Sugeno (TS) fuzzy system identification using the product space clustering method. This method comprises two steps: fuzzy clustering and rules construction. The first proposed method enables the use of incomplete system identification data to fuzzy c-means clustering algorithm developed for incomplete classification, which yields different estimates for a missing sample. This can be achieved by fusing those different values into a single value. The second proposed method treats missing samples as optimization variables during the identification process. The optimization is repeated until the change of all optimization variables is small.
@en

Reinforcement learning has emerged as a promising approach for enabling robots to learn from interactions with their environments, without relying on predefined behaviors. However, robots face significant challenges when learning directly from real-world interactions. Real-world learning is time-consuming and resource-intensive, often requiring extensive data collection over long periods. Additionally, the risks involved in trial-and-error learning in physical settings are high, as faulty policies can lead to safety issues or system damage. Simulations offer a safer and more efficient alternative, allowing robots to learn in simulated environments at faster-than-real-time speeds. Despite these benefits, simulations often serve as imperfect approximations of reality. As a result, robots may learn behaviors that exploit simulation-specific quirks, which may not perform well in real-world settings, creating difficulties in transferring learned behaviors from simulation to real environments—a challenge known as the sim-to-real gap. Several factors contribute to the sim-to-real gap, such as unmodeled physical phenomena like friction and deformation, and the asynchronous nature of real-world systems that simulations often fail to capture accurately. Additionally, using separate software stacks for simulation and deployment can unintentionally lead to discrepancies. Finally, simulating at faster-than-real-time speeds with asynchronous frameworks that distribute computation across multiple cores may also introduce inaccuracies without proper synchronization.

This thesis focuses on improving simulation tools and methodologies to enhance the efficiency and effectiveness of learning-based approaches in robotics. The work addresses key trade-offs between flexibility, speed, and accuracy in robotic simulations, which are critical for successfully transferring learned policies from simulation to real-world environments. Additionally, it introduces a strategy to improve resilience, ensuring that learned behaviors are robust to irrelevant and unknown dynamics.
By tackling these challenges, this thesis provides insights into the design of effective robotic simulators and presents contributions that help bridge the gap between simulated and real-world robotic learning.@en

In urban environments like Amsterdam, where road congestion and stress on infrastructure are critical issues, the city’s waterways remain underutilized for transportation. Autonomous Surface Vessels (ASVs), making waterway transport cheaper and less labor intensive, present a potential solution by reducing transportation costs and alleviating road traffic. Although companies are developing ASVs, current solutions are mainly limited to larger, less congested waterways, with further innovation needed for denser urban areas.

The challenges in autonomous navigation for ASVs in urban canals stem from the complex nonlinear dynamics of vessels, which require long-term planning and rapid responses to environmental changes. Urban waterways are narrow and unstructured, with loosely defined navigation rules that can lead to discontinuities in the motion planner’s cost function. Typically, motion planners rely on predictions of other agents’ movements and plan around them, resulting in no interaction awareness. This approach can lead to the freezing robot problem in dense environments, where the ASV halts, deeming all the space unsafe. A local motion planner must address these challenges by ensuring long-horizon interaction-aware motions, rule adherence, and real-time planning.

An emerging approach in autonomous navigation is a sampling-based Model Predictive Control (MPC) strategy known as Model Predictive Path Integral control (MPPI). This algorithm approximates the optimal control sequence by sampling from a continuous input distribution. Thanks to its gradient-free nature, MPPI only requires collision checking to avoid collision and allows discontinuous cost functions. Additionally, its computation speed remains largely unaffected by the complexity of the robots’ nonlinear dynamics, enabling longer planning horizons. While this approach has grown in popularity in the motion planning community, its application in dynamic environments with multiple interacting agents remains relatively unexplored.

Building on the state-of-the-art in MPPI, this thesis first introduces an Interaction-Aware Model Predictive Path Integral (IA-MPPI) controller tailored for motion planning in crowded urban canals. While conventional planners passively react to other agents, IA-MPPI actively predicts and plans cooperatively in real-time, addressing the freezing robot problem and handling discontinuous navigation rules. The decentralized, communication-free architecture assumes agent cooperation and employs a two-stage sample evaluation to enhance efficiency. This approach manages nonlinear dynamics, exact obstacle shapes, and longer planning horizons, demonstrating robustness and efficiency compared to state-of-the-art MPC in multi-agent environments.

While IA-MPPI uses a constant velocity model for predicting other agents’ hidden goals, Chapter 4 of this thesis introduces a learning-based trajectory prediction model trained on realistic artificial data to improve accuracy in crowded environments. By extracting the agents’ goals from predicted trajectories and integrating this information into the IA-MPPI controller, the planner’s performance increases significantly in environments with tight interactions. Moreover, this approach outperforms treating the predicted trajectory as occupied space, leading to safer navigation in dense urban canals.

One of MPPI’s main drawbacks is that it struggles in highly dynamic environments because it usually only samples around a previous plan. When rapid changes occur, such as strong disturbances or an obstacle unexpectedly cutting off the path, the previous plan becomes invalid, and all the generated samples may lead to collisions. To address this, Chapter 5 introduces Biased-MPPI, incorporating multiple classic and learning-based ancillary controllers into the sampling distribution. While the mathematical derivations show that this introduces a bias, this significantly improves reactivity, safety, and robustness to local minima. Additionally, this approach reduces the required samples, making the controller more efficient regarding computational demand.

Lastly, Chapter 6 leverages the gradient-free nature of MPPI and applies it to a different domain involving contact-rich manipulation tasks, which are notoriously difficult for model-based planning. With a parallelizable GPU-based physics engine like IsaacGym, MPPI can efficiently roll out hundreds of input sequences in parallel, faster than in real-time, to approximate optimal control. This approach eliminates the need for explicit modeling of contact dynamics by exploiting the models embedded in the simulator, making it particularly suited for manipulation tasks like pushing and picking with both prehensile and non-prehensile manipulators. Experiments with real robots demonstrate the effectiveness of this method in handling discontinuous, contact-heavy environments.

Overall, this thesis explores key aspects of motion planning, particularly for ASVs, with extensions to ground robots and mobile manipulators. It advances the MPPI framework for autonomous navigation by adapting it for multi-agent systems and introducing biases through classic and learning-based ancillary controllers. Additionally, the thesis compares MPPI to MPC in navigation experiments and to optimization fabrics in manipulation tasks. The work presented promotes the use of MPPI, especially in domains where complex nonlinear dynamics and discontinuous costs complicate the use of real-time optimization-based MPC.@en