Approximate dynamic programming (ADP) faces challenges in dealing with constraints in control problems. Model predictive control (MPC) is, in comparison, well-known for its accommodation of constraints and stability guarantees, although its computation is sometimes prohibitive. This paper introduces an approach combining the two methodologies to overcome their individual limitations. The predictive control law for constrained linear quadratic regulation (CLQR) problems has been proven to be piecewise affine (PWA) while the value function is piecewise quadratic. We exploit these formal results from MPC to design an ADP method for CLQR problems with a known model. A novel convex and piecewise quadratic neural network with a local–global architecture is proposed to provide an accurate approximation of the value function, which is used as the cost-to-go function in the online dynamic programming problem. An efficient decomposition algorithm is developed to generate the control policy and speed up the online computation. Rigorous stability analysis of the closed-loop system is conducted for the proposed control scheme under the condition that a good approximation of the value function is achieved. Comparative simulations are carried out to demonstrate the potential of the proposed method in terms of online computation and optimality.

In this paper, we analyze the regret incurred by a computationally efficient exploration strategy, known as naive exploration, for controlling unknown partially observable systems within the Linear Quadratic Gaussian (LQG) framework. We introduce a two-phase control algorithm called LQG-NAIVE, which involves an initial phase of injecting Gaussian input signals to obtain a system model, followed by a second phase of an interplay between naive exploration and control in an episodic fashion. We show that LQG-NAIVE achieves a regret growth rate of Õ(√T), i.e., O(√T) up to logarithmic factors after T time steps, and we validate its performance through numerical simulations. Additionally, we propose LQG-IF2E, which extends the exploration signal to a 'closed-loop' setting by incorporating the Fisher Information Matrix (FIM). We provide compelling numerical evidence of the competitive performance of LQG-IF2E compared to LQG-NAIVE.

Networks of dynamical systems play an important role in various domains and have motivated many studies on the control and analysis of linear dynamical networks. For linear network models considered in these studies, it is typically pre-determined what signal channels are inputs and what are outputs. These models do not capture the practical need to incorporate different experimental situations, where different selections of input and output channels are applied to the same network. Moreover, a unified view of different network models is lacking. This work makes an initial step towards addressing the above issues by taking a behavioral perspective, where input and output channels are not pre-determined. The focus of this work is on behavioral network models with only external variables. By exploiting the concept of hypergraphs, novel dual graphical representations, called system graphs and signal graphs, are introduced for behavioral networks. Moreover, connections between behavioral network models and structural vector autoregressive models are established. In addition to their connections in graphical representations, it is shown that the regularity of interconnections is an essential assumption when choosing a structural vector autoregressive model.

In a dynamic network of interconnected transfer functions, it is not necessary to use all the node signals for estimating a local transfer function. Given the network topology, detailed conditions are available for selecting inputs and outputs in a (MIMO) predictor model that warrants consistent and minimum variance estimation of a target module through the so-called local direct method. Motivated by the existing minimum-input signal selection approach that gradually incorporates additional signals, an alternative graphical algorithm for signal selection is developed in this work by directly exploiting the complete network graph. Then, as a straightforward application of existing analytical results, graphical conditions for consistent identification are derived for the novel signal selection approach. We show by an example that in some cases, for the consistent estimation of the target module, the developed method leads to fewer selected signals than the original minimum-input method.

Identifiability of a single module in a network of transfer functions is determined by whether a particular transfer function in the network can be uniquely distinguished within a network model set, on the basis of data. Whereas previous research has focused on the situations that all network signals are either excited or measured, we develop generalized analysis results for the situation of partial measurement and partial excitation. As identifiability conditions typically require a sufficient number of external excitation signals, this article introduces a novel network model structure such that excitation from unmeasured noise signals is included, which leads to less conservative identifiability conditions than relying on measured excitation signals only. More importantly, graphical conditions are developed to verify global and generic identifiability of a single module based on the topology of the dynamic network. Depending on whether the input or the output of the module can be measured, we present four identifiability conditions which cover all possible situations in single module identification. These conditions further lead to synthesis approaches for allocating excitation signals and selecting measured signals, to warrant single module identifiability. In addition, if the identifiability conditions are satisfied for a sufficient number of external excitation signals only, indirect identification methods are developed to provide a consistent estimate of the module. All the obtained results are also extended to identifiability of multiple modules in the network.

For the identification of switched systems with measured states and a measured switching signal, this work aims to analyze the effect of switching strategies on the estimation error. The data is assumed to be collected from globally asymptotically or marginally stable switched systems under switches that are arbitrary or subject to an average dwell time constraint. Then the switched system is estimated by the least-squares (LS) estimator. To capture the effect of the parameters of the switching strategies on the LS estimation error, finite-sample error bounds are developed in this work. The obtained error bounds show that the estimation error is logarithmic of the switching parameters when there are only stable modes; however, when there are unstable modes, the estimation error bound can increase linearly as the switching parameter changes. This suggests that in the presence of unstable modes, the switching strategy should be properly designed to avoid the significant increase of the estimation error.

A recent research direction in data-driven modeling is the identification of dynamic networks, in which measured vertex signals are interconnected by dynamic edges represented by causal linear transfer functions. The major question addressed in this article is where to allocate external excitation signals such that a network model set becomes generically identifiable when measuring all vertex signals. To tackle this synthesis problem, a novel graph structure, referred to as directed pseudotree, is introduced, and the generic identifiability of a network model set can be featured by a set of disjoint directed pseudotrees that cover all the parameterized edges of an extended graph, which includes the correlation structure of the process noises. Thereby, an algorithmic procedure is devised, aiming to decompose the extended graph into a minimal number of disjoint pseudotrees, whose roots then provide the appropriate locations for excitation signals. Furthermore, the proposed approach can be adapted using the notion of antipseudotrees to solve a dual problem, which is to select a minimal number of measurement signals for generic identifiability of the overall network, under the assumption that all the vertices are excited.

Identifiability conditions for single or multiple modules in a dynamic network specify under which conditions the considered modules can be uniquely recovered from the second-order statistical properties of the measured signals. Conditions for generic identifiability of multiple modules, i.e. a subnetwork, are developed for the situation that all node signals are measured and excitation of the network is provided by both measured excitation signals and unmeasured disturbance inputs. Additionally, the network model set is allowed to contain non-parametrized modules that are fixed, and e.g. reflect modules of which the dynamics are known to the user. The conditions take the form of path-based conditions on the graph of the network model set. Based on these conditions, synthesis results are formulated for allocating external excitation signals to achieve generic identifiability of particular subnetworks. If there are a sufficient number of measured external excitation signals, the formulated results give rise to a generalized indirect type of identification algorithm that requires only the measurement of a subset of the node signals in the network.

Identifiability of linear dynamic networks requires the presence of a sufficient number of external excitation signals. The problem of allocating a minimal number of external signals for guaranteeing generic network identifiability in the full measurement case has been recently addressed in the literature. Here we will extend that work by explicitly incorporating the situation that some network modules are known, and thus are fixed in the parametrized model set. The graphical approach introduced earlier is extended to this situation, showing that the presence of fixed modules reduces the required number of external signals. An algorithm is presented that allocates the external signals in a systematic fashion.

Several studies claim that listening to Mozart music affects cognition and can be used to treat neurological conditions like epilepsy. Research into this Mozart effect has not addressed how dynamic interactions between brain networks, i.e. effective connectivity, are affected. The Granger-causality analysis is often used to infer effective connectivity. First, we investigate if a new method, Bayesian topology identification, can be used as an alternative. Both methods are evaluated on simulation data, where the Bayesian method outperforms the Granger-causality analysis in the inference of connectivity graphs of dynamic networks, especially for short data lengths. In the second part, the Bayesian method is extended to enable the inference of changes in effective connectivity between groups of subjects. Next, we apply both methods to fMRI scans of 16 healthy subjects, who were scanned before and after the exposure to Mozart's sonata K448 at least 2 hours a day for 7 days. Here, we investigate if the effective connectivity of the subjects significantly changed after listening to Mozart music. The Bayesian method detected changes in effective connectivity between networks related to cognitive processing and control in the connection from the central executive to the superior sensori-motor network, in the connection from the posterior default mode to the fronto-parietal right network, and in the connection from the anterior default mode to the dorsal attention network. This last connection was only detected in a subgroup of subjects with a longer listening duration. Only in this last connection, an effect was found by the Granger-causality analysis.