Most real-world systems are affected by external disturbances, which may be impossible or costly to measure. For instance, when autonomous robots move in dusty environments, the perception of their sensors is disturbed. Moreover, uneven terrains can cause ground robots to deviate from their planned trajectories. Thus, learning the external disturbances and incorporating this knowledge into the future predictions in decision-making can significantly contribute to improved performance. Our core idea is to learn the external disturbances that vary with the states of the system, and to incorporate this knowledge into a novel formulation for robust tube model predictive control (TMPC). Robust TMPC provides robustness to bounded disturbances considering the known (fixed) upper bound of the disturbances, but it does not consider the dynamics of the disturbances. This can lead to highly conservative solutions. We propose a new dynamic version of robust TMPC (with proven robust stability), called state-dependent dynamic TMPC (SDD-TMPC), which incorporates the dynamics of the disturbances into the decision-making of TMPC. In order to learn the dynamics of the disturbances as a function of the system states, a fuzzy model is proposed. We compare the performance of SDD-TMPC, MPC, and TMPC via simulations, in designed search-and-rescue scenarios. The results show that, while remaining robust to bounded external disturbances, SDD-TMPC generates less conservative solutions and remains feasible in more cases, compared to TMPC.

@en

Ensuring safety in autonomous systems is essential as they become more integrated with modern society. One way to accomplish this is to identify and maintain a safe operating space. To this end, much effort has been devoted in the field of reachability analysis to obtaining control-invariant sets which ensure that a system inside of these sets can remain in these sets, and are thus essential for guaranteeing a system's safety. However, control invariance does not imply that a system can move from any state in the control-invariant set to any other state in the control-invariant set, within a given time horizon. In this paper, we develop an algorithm to obtain a control-invariant set that allows a given system to move from any state in the set to any other state in the set within a given time horizon without having to leave the set. We call this the 'maneuver set', M. We substantiate the algorithm's efficacy through mathematical proof, affirming that the maneuver set obtained through the algorithm is indeed control-invariant. Furthermore, we prove that the system is indeed able to move from any state within this set to any other state in the set. To illustrate the use of our algorithm, we provide the numerical example of a Dubins car, utilising Hamilton-Jacobi-Bellman reachability analysis along with the proposed algorithm in order to obtain M.

@en

Parameterized model predictive control (PMPC) is one of the many approaches that have been developed to alleviate the high computational requirement of model predictive control (MPC), and it has been shown to significantly reduce the computational complexity while providing comparable control performance with conventional MPC. However, PMPC methods still require a sufficiently accurate model to guarantee the control performance. To deal with model mismatches caused by the changing environment and by disturbances, this paper first proposes a novel framework that uses reinforcement learning (RL) to adapt all components of the PMPC scheme in an online way. More specifically, the novel framework integrates various strategies to adjust different components of PMPC (e.g., objective function, state-feedback control function, optimization settings, and system model), which results in a synthesis framework for RL-based adaptive PMPC. We show that existing adaptive (P)MPC approaches can also be embedded in this synthesis framework. The resulting combined RL-PMPC framework provides a solution for an efficient MPC approach that can deal with model mismatches. A case study is performed in which the framework is applied to freeway traffic control. Simulation results show that for the given case study the RL-based adaptive PMPC approach reduces computational complexity by 98% on average compared to conventional MPC while achieving better control performance than the other controllers, in the presence of model mismatches and disturbances.

@en

Model predictive control (MPC) and deep reinforcement learning (DRL) have been developed extensively as two independent techniques for traffic management. Although the features of MPC and DRL complement each other very well, few of the current studies consider combining these two methods for application in the field of freeway traffic control. This paper proposes a novel framework for integrating MPC and DRL methods for freeway traffic control that is different from existing MPC-(D)RL methods. Specifically, the proposed framework adopts a hierarchical structure, where a high-level efficient MPC component works at a low frequency to provide a baseline control input, while the DRL component works at a high frequency to modify online the output generated by MPC. The control framework, therefore, needs only limited online computational resources and is able to handle uncertainties and external disturbances after proper learning with enough training data. The proposed framework is implemented on a benchmark freeway network in order to coordinate ramp metering and variable speed limits, and the performance is compared with standard MPC and DRL approaches. The simulation results show that the proposed framework outperforms standalone MPC and DRL methods in terms of total time spent (TTS) and constraint satisfaction, despite model uncertainties and external disturbances.@en

Recent advances in robotics and artificial intelligence have made it necessary or desired for humans to get involved in interactions with social robots. A key factor for the human acceptance of these robots is their awareness of environmental and social norms. In this paper, we introduce SONAR (for SOcial Norm Aware Robots), a novel robot-agnostic control architecture aimed at enabling social agents to autonomously recognize, act upon, and learn over time social norms during interactions with humans. SONAR integrates several state-of-the-art theories and technologies, including the belief-desire-intention (BDI) model of reasoning and decision making for rational agents, fuzzy logic theory, and large language models, to support adaptive and norm-aware autonomous decision making. We demonstrate the feasibility and applicability of SONAR via real-life experiments involving human-robot interactions (HRI) using a Nao robot for scenarios of casual conversations between the robot and each participant. The results of our experiments show that our SONAR implementation can effectively and efficiently be used in HRI to provide the robot with environmental and social and norm awareness. Compared to a robot with no explicit social and norm awareness, introducing social and norm awareness via SONAR results in interactions that are perceived as more positive and enjoyable by humans, as well as in higher perceived trust in the social robot. Moreover, we investigate, via computer-based simulations, the extent to which SONAR can be used to learn and adapt to the social norms of different societies. The results of these simulations illustrate that SONAR can successfully learn adequate behaviors in a society from a relatively small amount of data. We publicly release the source code of SONAR, along with data and experiments logs.

@en

Humans and autonomous vehicles will jointly use the roads in smart cities. Therefore, it is a requirement for autonomous vehicles to properly handle the information and uncertainties that are introduced by humans (e.g., drivers, pedestrians, traffic managers) into the traffic, to accordingly make proper decisions. Such information is commonly available as linguistic, fuzzy (non-quantified) terms. Thus, we need mathematical modeling approaches that, at the same time, handle mixed (i.e., quantified and non-quantified) data. For this, we introduce novel type-2 sets and membership functions to translate such mixed traffic data into mathematical concepts that handle different levels and types of uncertainties and that can undergo mathematical operations. Next, we propose rule-based data processing and modeling approaches to exploit the advantages of these sets. This is inspired by the rule-based reasoning of humans, which has proven to be very effective and efficient in various applications, especially in traffic. The resulting models, hence, handle more than one level and type of uncertainty, which results in precise estimations of traffic dynamics that are comparable in accuracy with similar analyses if only one level of uncertainty (either probabilistic or fuzzy) would exist in the dataset. This will significantly improve the analysis, prediction, management, and safety of traffic in future smart cities.

@en

Model Predictive Control (MPC) optimizes an objective function within a prediction window under constraints. In the presence of bounded disturbances, robust versions are used. Recently, a promising robust MPC was introduced that outperforms SOTA approaches. However, solving the optimization problem online is computationally expensive. An efficient approximation method, such as neural networks (NN), can be substituted to accelerate the online computation. There are discrepancies between the control inputs due to the approximation. We propose to model them as bounded state-dependent disturbances to robustly control nonlinear wheeled robots. We consider a spiking NN to ensure that small robots could use it.

@en

Search and rescue (SaR) is challenging, due to the unknown environmental situation after disasters occur. Robotics has become indispensable for precise mapping of the environment and for locating the victims. Combining flying and ground robots more effectively serves this purpose, due to their complementary features in terms of viewpoint and maneuvering. To this end, a novel, cost-effective framework for mapping unknown environments is introduced that leverages You Only Look Once and video streams transmitted by a ground and a flying robot. The integrated mapping approach is for performing three crucial SaR tasks: localizing the victims, i.e., determining their position in the environment and their body pose, tracking the moving victims, and providing a map of the ground elevation that assists both the ground robot and the SaR crew in navigating the SaR environment. In real-life experiments at the CyberZoo of the Delft University of Technology, the framework proved very effective and precise for all these tasks, particularly in occluded and complex environments.

@en

Robots are increasingly deployed for search-and-rescue (SaR), in order to speed up rescuing the victims in the aftermath of disasters. These robots require effective mission planning approaches to determine time and space-efficient trajectories that steer them faster towards (moving) victims, while dealing with uncertainties. Model predictive control (MPC) is an effective optimization-based control approach that has been used to steer robots along reference trajectories determined by higher level controllers. Determining the trajectory of the robots directly via MPC has the advantage of optimizing multiple SaR criteria while handling the constraints. We, thus, introduce a path planning approach based on MPC for indoor SaR robots that allows the robot to systematically chase the moving victims, when no reference trajectory is provided. The proposed approach combines target-oriented and coverage-oriented search, and allows for systematic handling of environmental uncertainties, by deploying a robust tube-based version of the introduced MPC formulation. In addition, we model the movements of the victims for MPC, by adopting an existing evacuation model. We present a case study, using Gazebo, MATLAB, and ROS, where the performance of the proposed MPC controller is evaluated compared to four state-of-the-art methods (two target-oriented methods based on MPC and A* and two heuristic algorithms for area coverage). The results show that, while robust to uncertainties, our approach overall outperforms the other methods, with regards to victim detection, area coverage, and mission time.@en

Interactive machines should establish and maintain meaningful social interactions with humans. Thus, they need to understand and predict the mental states and actions of humans. Based on Theory of Mind (ToM), in order to understand and interact with each other, humans develop cognitive models of one another. Our main goal is to provide a mathematical framework based on ToM to improve the understanding of interactive machines regarding the perception, cognition, and decision-making of humans. Most state-of-the-art models of behavioral theories based on machine learning are focused on input-output black-box representations. Thus, they lack transparency and generalizability, and exhaustive training procedures are needed to personalize them for various humans. Moreover, these models lack dynamics, i.e., they do not mathematically describe the evolution of the mental states and actions of humans in time. Following a systems-and-control-theoretic point-of-view, we represent for the first time the perception, cognition, and decision-making of humans via a dynamic, mathematical framework by introducing a novel formalization and an extension to Fuzzy Cognitive Maps (FCMs). The resulting models are given in a general state-space representation, which can be used by interactive machines within known model-based state estimation and control methods. In a case study, the resulting models were identified and validated for 21 participants, in scenarios where predicting the intentions and behavior of the participants required understanding the dynamics of their mental procedures. The results of these experiments show that our model is capable of incorporating the dynamics to estimate the intentions and predict the behavior of the participants, with an accuracy of, respectively, 81.55% and 66.06%. Moreover, we compared our model with a state-of-the-art formalization of human cognition, which was made dynamic using our introduced FCM framework. Our model, which in addition to the elements of the state-of-the-art model included emotions, personality traits, and biases (thus providing a more transparent insight about the mental procedures of the participants) showed 6.25% and 2.45% more accuracy in, respectively, estimating the intentions and predicting the behavior of the participants.

@en

Search-and-rescue (SaR) in unknown environments is a crucial task with life-threatening risks. SaR requires precise, optimal, and fast decisions to be made. Robots are promising candidates expected to execute various SaR tasks autonomously. While humans use heuristics to effectively deal with uncertainties of SaR, optimisation of multiple objectives (e.g., the mission time, the area covered, the number of victims detected), in the presence of physical and control constraints, is a mathematical challenge that requires machine computations. Thus including both human-inspired and mathematical capabilities in decision making of SaR robots is highly desired. However, developing control approaches that exhibit both capabilities has been significantly ignored in literature. Moreover, coordinating the decisions of the robots in large-scale SaR missions with affordable computation costs is an open challenge. Finally, in real-life, due to defects (e.g., in the sensors of the robots) or environmental factors (e.g., smoke) data perceived by SaR robots may be prone to uncertainties. We introduce a hierarchical multi-agent control architecture that simultaneously provides the following advantages: exploiting non-homogeneous and imperfect perception capabilities of SaR robots; improving the global performance as it is provided by centralised controllers; computational efficiency and robustness to failure of the central controller as offered by decentralised control methods. The integrated structure of the proposed control framework allows to combine human-inspired and mathematical decision making methods, via respectively fuzzy logic and model predictive control, in a coordinated and computationally efficient way. Our results for various computer-based simulations show that while the area coverage with the proposed control approach is comparable to existing heuristic methods that are particularly developed for coverage-oriented SaR, our approach has a significantly better performance regarding locating the trapped victims. Furthermore, with comparable computation times, the proposed control approach successfully avoids conflicts that may appear in non-cooperative control methods. In summary, the proposed multi-agent control system is capable of combining coverage-oriented and target-oriented SaR in a balanced and coordinated way.

@en

We propose a novel method to improve the convergence performance of model predictive control (MPC) for setpoint tracking, by introducing sub-references within a multilevel MPC structure. In some cases, MPC is implemented with a short prediction horizon due to limited on-line computation capacity, which could lead to deteriorated dynamic performance. The introduced multi-level optimization method can generate proper sub-references for the MPC setpoint tracking problem, and efficiently improve the dynamic performance. In the higher level a specific performance criterion is taken as the objective, while explicit MPC is utilized in the lower level to represent the control input. The generated sub-references are then used in MPC for the real system with prediction horizon restrictions. Setpoint-tracking MPC for linear systems is used to illustrate the approach throughout the paper. Numerical simulations show that MPC with sub-references significantly improves the convergence performance compared with regular MPC with the same prediction horizon. Thus, it can be concluded that MPC with sub-references has a high potential to tackle more complicated control problems with limited computation capacity.@en

Model predictive control (MPC) has been widely used for traffic management, such as for minimizing the total time spent or the total emissions of vehicles. When long-term green urban mobility is considered including e.g. a constraint on the total yearly emissions, the optimization horizon of the MPC problem is significantly larger than the control sampling time, and thus the number of the variables that should be optimized per control time step becomes very large. For systems with dynamics that involve nonlinear, non-convex, and non-smooth functions, including urban traffic networks, this results in optimization problems that are computationally intractable in real time. In this paper, we propose a novel bi-level temporal distribution of such complex MPC optimization problems, and we develop two mathematically linked short-term and long-term MPC formulations with small and large control sampling times that will be solved together instead of the original complex optimization problem. The resulting bi-level control architecture is used to solve the two MPC formulations online for real-time control of urban traffic networks with the objective of long-term green mobility. In order to assess the performance of the bi-level control architecture, we perform a case study where a rough version of the model of the urban traffic flow, S-model, is used by the long-term MPC level to estimate the states of the urban traffic networks, and a detailed version of the model is used by the short-term MPC level. The results of the simulations prove the effectiveness (with respect to the objective of control, as well as computational efficiency) of the proposed bi-level MPC approach, compared to state-of-the-art control approaches.@en

Traffic control is essential to reduce congestion in both urban and freeway traffic networks. These control measures include ramp metering and variable speed limits for freeways, and traffic signal control for urban traffic. However, current traffic control methods are either too simple to respond to complex traffic environment, or too sophisticated for real-life implementation. In this paper, we propose an adaptive parameterized control method for traffic management by using reinforcement learning algorithms. This method takes advantage of the simple structure of parameterized state-feedback controllers for traffic; meanwhile, a reinforcement learning agent is employed to adjust the parameters of the controllers on-line to react to the varying environment. Therefore, the proposed method requires limited real-time computational efforts, and is adaptive to external disturbances. Furthermore, the reinforcement learning agent can coordinate multiple local traffic controllers when adjusting their parameters. The method is validated by a numerical case study on a freeway network. Results show that the proposed method outperforms conventional controllers when the system is exposed to a changing environment.@en

While Model Predictive Control (MPC) is a promising approach for network-wide control of urban traffic, the computational complexity of the, often nonlinear, online optimization procedure is too high for real-time implementations. In order to make MPC computationally efficient, this paper introduces a parameterized MPC (PMPC) approach for urban traffic networks that uses Grammatical Evolution to construct continuous parameterized control laws using an effective simulation-based training framework. Furthermore, a projection-based method is proposed to remove the nonlinear constraints that are imposed on the parameters of the parameterized control laws and to guarantee the feasibility of the solution of the MPC optimization problem. The performance and computational efficiency of the constructed parameterized control laws are compared to those of a conventional MPC controller in an extensive simulation-based case study. The results show that the parameterized control laws, which are automatically constructed using Grammatical Evolution, decrease the computational complexity of the online optimization problem by more than 80% with a decrease in performance by less than 10%.

@en

Socially Assistive Robots (SARs) are increasingly used in dementia and elderly care. In order to provide effective assistance, SARs need to be personalized to individual patients and account for stimulating their divergent thinking in creative ways. Rule-based fuzzy logic systems provide effective methods for automated decision-making of SARs. However, expanding and modifying the rules of fuzzy logic systems to account for the evolving needs, preferences, and medical conditions of patients can be tedious and costly. In this paper, we introduce EFS4SAR, a novel Evolving Fuzzy logic System for Socially Assistive Robots that supports autonomous evolution of the fuzzy rules that steer the behavior of the SAR. EFS4SAR combines traditional rule-based fuzzy logic systems with evolutionary algorithms, which model the process of evolution in nature and have shown to result in creative behaviors. We evaluate EFS4SAR via computer simulations on both synthetic and real-world data. The results show that the fuzzy rules evolved over time are not only personalized with respect to the personal preferences and therapeutic needs of the patients, but they also meet the following criteria for creativity of SARs: originality and effectiveness of the therapeutic tasks proposed to the patients. Compared to existing evolving fuzzy systems, EFS4SAR achieves similar effectiveness with higher degree of originality.

@en

In general, the performance of model-based controllers cannot be guaranteed under model uncertainties or disturbances, while learning-based controllers require an extensively sufficient training process to perform well. These issues especially hold for large-scale nonlinear systems such as urban traffic networks. In this paper, a new framework is proposed by combining model predictive control (MPC) and reinforcement learning (RL) to provide desired performance for urban traffic networks even during the learning process, despite model uncertainties and disturbances. MPC and RL complement each other very well, since MPC provides a sub-optimal and constraint-satisfying control input while RL provides adaptive control laws and can handle uncertainties and disturbances. The resulting combined framework is applied for traffic signal control (TSC) of an urban traffic network. A case study is carried out to compare the performance of the proposed framework and other baseline controllers. Results show that the proposed combined framework outperforms conventional control methods under system uncertainties, in terms of reducing traffic congestion.

@en

Novel personalized and affordable approaches are needed in order to provide efficient therapeutic interventions for people with autism spectrum disorder. In this paper, we introduce a new category of socially assistive robots, i.e., socially assistive drones (SADs) for therapeutic interactions with humans. SADs autonomously perform and engage humans in dance movement therapy (DMT), which has proven to be highly effective for people with autism spectrum disorder when it is personalized and adapted carefully. The main focus of this paper is on developing adaptive, personalized, and interactive control approaches based on fuzzy logic, which efficiently deals with nonlinear dynamics and directly introduces the expert (linguistic) knowledge into the control system. The developed approaches are implemented via a small quadcopter in real-life interactions with 10 participants. The results of the case study prove the excellent performance of the SAD in adapting and personalizing its decisions with respect to each user and thus keeping them engaged in the proposed DMT plans.

@en

A bi-Ievel architecture based on model predictive control is proposed for air traffic management. The aim is to reduce the computational complexity of online optimization in air traffic control, while producing accurate control decisions. We integrate a slow-rate centralized MPC controller including a global, long-term prediction window (top level of control), with several fast-rate decentralized MPC controllers including local, short-term prediction windows (bottom level of control). All aircraft are steered towards their destination in the shortest possible time, while forbidden areas are avoided. The performance of the proposed bi-Ievel control architecture is compared via a case study with (1) a slow-rate centralized MPC controller similar to the top control level, (2) a fast-rate decentralized MPC controller similar to the bottom control level, and (3) a fast- rate centralized MPC controller with a prediction horizon the same as the top control level and decision sampling times the same as the bottom control level: The results show that while the bi-Ievel MPC controller is significantly more efficient - w.r.t. the computation time - than the fast-rate centralized MPC controller, these two controllers are comparable w.r.t. accuracy of the decisions. Moreover, with a comparable computation time, the bi-Ievel MPC controller attains a superior control performance - considering the accuracy of decisions - compared to the slow-rate centralized and fast-rate decentralized MPC controllers individually. This shows that the integration of these two control approaches in the proposed bi-Ievel architecture can significantly boost their performance.@en

Presents corrections to the paper, “Integrated urban traffic control for the reduction of travel delays and emissions,” (Lin, S., et al), IEEE Trans. Intell. Transp. Syst., vol. 14, no. 4, pp. 1609–1619, Dec. 2013.

@en