This paper presents a novel, unified approach for generating high-quality datasets for training machine-learned models for real-time security assessment in power systems. Synthetic data generation methods that extrapolate beyond historical data can be inefficient in generating feasible and rare operating conditions (OCs). The proposed approach balances the trade-off between historically relevant OCs and rare but feasible OCs. Unlike conventional methods that rely on historical records or generic sampling, our approach results in datasets that generalise well beyond similar distributions. The proposed approach is validated through experiments on the IEEE 118-bus system, where a decision tree model trained on data generated using our approach achieved 97% accuracy in predicting the security label of rare OCs, outperforming baseline approaches by 41% and 20%. This work is crucial for deploying reliable machine-learned models for real-time security assessment in power systems undergoing decarbonisation and integrating renewable energy sources.

@en

Machine learning (ML) for real-time security assessment requires a diverse training database to be accurate for scenarios beyond historical records. Generating diverse operating conditions is highly relevant for the uncertain future of emerging power systems that are completely different to historical power systems. In response, for the first time, this work proposes a novel split-based sequential sampling approach based on optimisation that generates more diverse operation scenarios for training ML models than state-of-the-art approaches. This work also proposes a volume-based coverage metric, the convex hull volume (CHV), to quantify the quality of samplers based on the coverage of generated data. This metric accounts for the distribution of samples across multidimensional space to measure coverage within the physical network limits. Studies on IEEE test cases with 6, 68 and 118 buses demonstrate the efficiency of the approach. Samples generated using the proposed split-based sampling cover 37.5% more volume than random sampling in the IEEE 68-bus system. The proposed CHV metric can assess the quality of generated samples (standard deviation of 0.74) better than a distance-based coverage metric which outputs the same value (standard deviation of <0.001) for very different data distributions in the IEEE 68-bus system. As we demonstrate, the proposed split-based sampling is relevant as a pre-step for training ML models for critical tasks such as security assessment.

@en

Developing innovative electricity market designs to facilitate a sustainable transition to (near) 100% renewable power systems while meeting societal needs is a crucial and actual topic of research. This article presents preliminary key findings from the H2020 European project TradeRES, addressing this critical topic. The project uses agent-based and optimization models to effectively capture the behaviour of different market players, and to analyse the current and future power system energy mixes of selected European case studies with different physical and spatial scales from: i) local energy communities and local energy markets (LEMs); ii) national/regional - the Netherlands, Germany, and Iberia (Portugal and Spain); and iii) pan-European energy markets. The first results on LEMs indicate a substantial economic benefit for participants and enhanced revenue streams for distributed energy resources, able to i) incentivise further decentralised investments; ii) promote the growth of variable renewable energy systems (vRES) and iii) increase flexibility at the local level. The outcomes are sensitive to the tariffs’ structure, while the retail sector competitiveness was identified as a critical parameter affecting its efficiency. For the pan-European and national/regional case studies, the first set of simulations had consistent outcomes, namely, by pointing out current design of energy-only markets to be insufficient to incentivize the high levels of vRES foreseen in Europe. Different support schemes (e.g., fixed market premia, contract for differences) were tested and results suggest they may play a relevant role in effectively covering the cost of vRES in a market environment.

@en

Due to the increasing system stability issues caused by the technological revolutions of power system equipment, the assessment of the dynamic security of the systems for changing operating conditions (OCs) is nowadays crucial. To address the computational time problem of conventional dynamic security assessment tools, many machine learning (ML) approaches have been proposed and well-studied in this context. However, these learned models only rely on data, and thus miss resourceful information offered by the physical system. To this end, this paper focuses on combining the power system dynamical model together with the conventional ML. Going beyond the classic Physics Informed Neural Networks (PINNs), this paper proposes Selected Physics Informed Neural Networks (SPINNs) to predict the system dynamics for varying OCs. A two-level structure of feed-forward NNs is proposed, where the first NN predicts the generator bus rotor angles (system states) and the second NN learns to adapt to varying OCs. We show a case study on an IEEE-9 bus system that considering selected physics in model training reduces the amount of needed training data. Moreover, the trained model effectively predicted long-term dynamics that were beyond the time scale of the collected training dataset (extrapolation).

@en

The classical formulation of the transmission switching problem as a mixed-integer problem is intractable for large systems in real-time control settings. Several heuristics have been proposed in the past to speed up the computation time, which only limits the number of switchable lines. In this paper, a real-time switching heuristic based on neural networks that provides almost instantaneous switching actions, are presented. The findings are shown on case studies of the IEEE 118-bus test system, and the results show that the proposed heuristic is robust to out of distribution data. Additionally, the proposed heuristic has significant computational savings while all other performance metrics like accuracy are similar to state-of-the-art machine learning methods proposed for transmission switching.

@en

Machine Learning (ML) for real-time Dynamic Security Assessment (DSA) promises a probabilistic approach to secure lower safety margins and costs. However, future systems with a high share of renewables have low inertia and converter-interfaced devices resulting in faster dynamics. Past research on ML-based DSA used high inertia systems to study ‘the best’ ML data, features, and models building upon each other's work for decades. Seldom has ML-based research for DSA questioned whether the underlying assumptions for (and the conclusions of) these studies are still valid for low inertia systems. This work studies exemplary changes in assumptions (and conclusions) for ML-based DSA when moving from High Inertia (HI) to Low Inertia (LI) systems. The dynamical system of the LI system is brought in perspective with the most typical ML-based approaches, which are organised in sequential steps. The steps consider the generation of the training database, the data pre-processing and feature selection, the model training and validation. This work analyses each step individually for the changed assumptions in the dynamical LI system, and subsequently, a case study provides the evidence that considering a LI system to identify the ‘best’ ML approaches is important. The case studies on IEEE 14 and 68 bus systems confirm that LI systems must be optimised for security (otherwise, they result in 80% less security than HI systems). The key findings, however, are that using ML makes significantly more sense in LI systems than in HI systems as the LI dynamics are in shorter timescales (and the advantage of ML is to predict security in milliseconds) and that secure/insecure operations can be separated more straightforwardly in LI systems as ML increases the accuracy by up-to 40% towards close to 100% when using neural networks.

@en

Digitalization is one of the key drivers for energy system transformation. The advances in communication technologies and measurement devices render available a large amount of operational data and enable the centralization of such data storage and processing. The greater access to data opens up new opportunities for a more efficient and decentralized management of the energy system. At the distribution level of the energy system, local electricity markets (LEMs) provide new degrees of flexibility by trading and balancing the energy locally and offering ancillary services to the wider transmission and distribution system operators. Maximizing the grid impact from this flexibility calls for novel data analytics and artificial intelligence techniques to enhance the system's security and reduce the energy costs of local prosumers. At the same time, however, relying on data-based approaches increases the risk of cyberattacks, and robust countermeasures are, therefore, needed as an integral aspect of digitalization efforts. This article discusses the key role of centralized data analytics to fully benefit from the advantages of LEMs in terms of system's security enhancement and energy costs' reduction. Data-driven paradigms are investigated that allow for flexibility from decentralized markets, mitigate the physical security risks, and devise defensive strategies shielding the system from cyber threats.

@en

This paper proposes a method to compute corrective control actions for dynamic security in real-time and quantifies the economic value of corrective control. Lowered inertia requires fast control methods in real-time to correct system operation and maintain system security when equipment fails. However, using corrective control beyond such emergency failure measures does not make fully use of them. The key contribution of this work is the optimal use of corrective control applications in combination with preventive strategies to enhance the network utilisation, reduce the normal operating costs while maintaining adequate security levels. The proposed approach learns a neural network for safety certificates and models the predicted safe dynamic post-fault state as algebraic constraints in an AC optimal power flow (OPF) deciding close to real-time on the optimal corrective control. Considering these safety constraints within the ACOPF can balance simultaneously the system transient stability with the costs for preventive and corrective control. This proposed approach outperforms sub-optimal approaches aiming at sequentially finding the balance. Case studies were based on the IEEE 9-bus system with integrated electrical vehicles and shares of wind power up-to 40% and on the IEEE 39-bus and 118-bus systems. The proposed approach outperforms baseline control approaches in stability, economics, and carbon emissions. One baseline approach was preventive wind curtailment, against which the proposed approach reduced operating costs by up-to 60%, decreased unstable operations by 50% and reduced carbon emissions by 60% in the IEEE 9-bus. In the IEEE 39-bus and 118-bus systems, the approach was promising for larger systems.

@en

Machine learning has been used in the past to construct predictors, also known as classifiers, for dynamic security assessment. Although accurate classifiers can be trained for a single topology, often they do not work for another. However, the power system topology can change frequently during operation due to maintenance and control actions. At one topological configuration, the system may have a different response to a fault than at another as the underlying distribution of power flows can be completely different. Quantifying the impact of changes in the topology on the predictive models’ performance is an important step forward to minimize inaccurate predictions and improve their reliability. In this paper, for the first time, a metric for quantifying the impact of a topology change on the accuracy of the classification model is proposed. The key novelty is to first select a subset of power flow features with a physically informed feature selection technique and subsequently compute the metric with a novel convex hull-based analysis. In addition, the approach can advise to effectively constructing new training databases that improve the accuracy of new machines trained after high-impact topology changes. Through a case study using transient stability on the IEEE 68-bus system, the use of the proposed metric in real-time operation was demonstrated. 17 high-impact topology changes were successfully detected among 42 studied topological changes. The subsequent effective construction of the training database improved the predictive accuracy by around 10%. An interesting finding is the amount of newly generated data can be reduced by up to 85% as often the generated data is the barrier for data-driven DSA. The proposed workflow significantly reduces data and trains robust classifiers against topological changes marking a fundamental step forward.

@en

Power systems transport an increasing amount of electricity, and in the future, involve more distributed renewables and dynamic interactions of the equipment. The system response to disturbances must be secure and predictable to avoid power blackouts. The system response can be simulated in the time domain. However, this dynamic security assessment (DSA) is not computationally tractable in real-time. Particularly promising is to train decision trees (DTs) from machine learning as interpretable classifiers to predict whether the system-wide responses to disturbances are secure. In most research, selecting the best DT model focuses on predictive accuracy. However, it is insufficient to focus solely on predictive accuracy. Missed alarms and false alarms have drastically different costs, and as security assessment is a critical task, interpretability is crucial for operators. In this work, the multiple objectives of interpretability, varying costs, and accuracies are considered for DT model selection. We propose a rigorous workflow to select the best classifier. In addition, we present two graphical approaches for visual inspection to illustrate the selection sensitivity to probability and impacts of disturbances. We propose cost curves to inspect selection combining all three objectives for the first time. Case studies on the IEEE 68 bus system and the French system show that the proposed approach allows for better DT-selections, with an 80% increase in interpretability, 5% reduction in expected operating cost, while making almost zero accuracy compromises. The proposed approach scales well with larger systems and can be used for models beyond DTs. Hence, this work provides insights into criteria for model selection in a promising application for methods from artificial intelligence (AI).

@en

Thermostatically controlled loads such as refrigerators are exceptionally suitable as a flexible demand resource. This article derives a decentralized load control algorithm for refrigerators. It is adapted from an existing continuous time control approach, with the aim to achieve low computational complexity and an ability to handle discrete time steps of variable length - desirable features for embedding in appliances and high-throughput simulations. Simulation results of large populations of heterogeneous appliances illustrate the accurate aggregate control of power consumption and high computational efficiency. Tracking accuracy is quantified as a function of population size and time step size, and correlations in the tracking error are investigated. The controller is shown to be robust to errors in model specification and to sudden perturbations in the form of random refrigerator door openings.

@en

Probabilistic security assessment and real-time dynamic security assessments (DSA) are promising to better handle the risks of system operations. The current methodologies of security assessments may require many time-domain simulations for some stability phenomena that are unpractical in real-time. Supervised machine learning is promising to predict DSA as their predictions are immediately available. Classifiers are offline trained on operating conditions and then used in real-time to identify operating conditions that are insecure. However, the predictions of classifiers can be sometimes wrong and hazardous if an alarm is missed for instance. A probabilistic output of the classifier is explored in more detail and proposed for probabilistic security assessment. An ensemble classifier is trained and calibrated offline by using Platt’ scaling to provide accurate probability estimates of the output. Imbalances in the training database and a cost-skewness addressing strategy are proposed for considering that missed alarms are significantly worse than false alarms. Subsequently, risk-minimised predictions can be made in real-time operation by applying cost-sensitive learning. Through case studies on a real data-set of the French transmission grid and on the IEEE 6 bus system using static security metrics, it is showcased how the proposed approach reduces inaccurate predictions and risks. The sensitivity on the likelihood of contingency is studied as well as on expected outage costs. Finally, the scalability to several contingencies and operating conditions are showcased.

@en

The integration of renewable energy sources increases the operational uncertainty of electric power systems and can lead to more frequent dynamic phenomena. The use of classifiers from machine learning is promising to include dynamics in the security assessment of the power system. The training of these classifiers is typically performed offline on synthetically generated operating conditions (OCs) that are similar to real-time operation. However, the uncertainty in the generated OCs and the classifier’s inaccuracy is larger the longer the time between offline and real-time operation. Moving the classifier training closer to real-time operation is an important step forward to reduce inaccurate predictions and improve reliability. In this paper, a novel causality-based feature selection approach for an online dynamic security assessment (DSA) framework is proposed. The key novelty is to use the system’s physics to learn the causal structure between the features and then select the features based on this causal structure. The proposed approach results in faster computations, is more robust and more interpretable. Moreover, classifiers can be trained closer to real-time operation which enhances the predictive performance. Through a case study using transient stability on the IEEE 68-bus system, the proposed method reduces computational time by 75% in comparison to state of the art feature selection techniques. The proposed workflow showed superior performance in accuracy and robustness against uncertainty compared to conventional machine learning approaches for DSA. The computational benefit was also projected to a dataset of the French transmission system where the approach has the potential to achieve computational savings of up-to two orders of magnitudes.@en

Dynamic Security Assessment (DSA) for the future power system is expected to be increasingly complicated with the higher level penetration of renewable energy sources (RES) and the widespread deployment of power electronic devices, which drive new dynamic phenomena. As a result, the increasing complexity and the severe computational bottleneck in real-time operation encourage researchers to exploit machine learning to extract offline security rules for the online assessment. However, traditional machine learning methods lack in providing information on the confidence of their corresponding predictions. A better understanding of confidence of the prediction is of key importance for Transmission System Operators (TSOs) to use and rely on these machine learning methods. Specifically, from the perspective of topological changes, it is often unclear whether the machine learning model can still be used. Hence, being aware of the confidence of the prediction supports the transition to using machine learning in real-time operation. In this paper, we propose a novel Conditional Bayesian Deep Auto-Encoder (CBDAC) based security assessment framework to compute a confidence metric of the prediction. This informs not only the operator to judge whether the prediction can be trusted, but it also allows for judging whether the model needs updating. A case study based on IEEE 68-bus system demonstrates that CBDAC outperforms the state-of-the-art machine learning-based DSA methods and the models that need updating under different topologies can be effectively identified. Furthermore, the case study verifies that effective updating of the models is possible even with very limited data.

@en

Buildings are responsible for about a quarter of global energy-related CO ₂ emissions. Consequently, the decarbonisation of the housing stock is essential in achieving net-zero carbon emissions. Global decarbonisation targets can be achieved through increased efficiency in using energy generated by intermittent resources. The paper presents a co-design framework for simultaneous optimal design and operation of residential buildings using Model Predictive Control (MPC). The framework is capable of explicitly taking into account operational constraints and pushing the system to its efficiency and performance limits in an integrated fashion. The optimality criterion minimises system cost considering time-varying electricity prices and battery degradation. A case study illustrates the potential of co-design in enhancing flexibility and self-sufficiency of a system operating under different conditions. Specifically, numerical results from a low-fidelity model show substantial carbon emission reduction and bill savings compared to an a-priori sizing approach.

@en

Accurately and efficiently estimating system performance under uncertainty is paramount in power system planning and operation. Monte Carlo simulation is often used for this purpose, but convergence may be slow, especially when detailed models are used. Previously published methods to speed up computations may severely constrain model complexity, limiting their real-world effectiveness. This paper uses the recently proposed Multilevel Monte Carlo (MLMC) framework, which combines outputs from a hierarchy of simulators to boost computational efficiency without sacrificing accuracy. It explains which requirements the MLMC framework imposes on the model hierarchy, and how these naturally occur in power system adequacy assessment problems. Two adequacy assessment examples are studied in detail: a composite system and a system with heterogeneous storage units. An intuitive speed metric is introduced for easy comparison of simulation setups. Depending on the problem and metric of interest, large speedups can be obtained.@en

Distribution networks are undergoing fundamental changes at medium voltage level. To support growing planning and control decision-making, the need for large numbers of short-term load forecasts has emerged. Data-driven modelling of medium voltage feeders can be affected by (1) data quality issues, namely, large gross errors and missing observations (2) the presence of structural breaks in the data due to occasional network reconfiguration and load transfers. The present work investigates and reports on the effects of advanced data cleansing techniques on forecast accuracy. A hybrid framework to detect and remove outliers in large datasets is proposed; this automatic procedure combines the Tukey labelling rule and the binary segmentation algorithm to cleanse data more efficiently, it is fast and easy to implement. Various approaches for missing value imputation are investigated, including unconditional mean, Hot Deck via k-nearest neighbour and Kalman smoothing. A combination of the automatic detection/removal of outliers and the imputation methods mentioned above are implemented to cleanse time series of 342 medium-voltage feeders. A nested rolling-origin-validation technique is used to evaluate the feed-forward deep neural network models. The proposed data cleansing framework efficiently removes outliers from the data, and the accuracy of forecasts is improved. It is found that Hot Deck (k-NN) imputation performs best in balancing the bias-variance trade-off for short-term forecasting.@en

Concerns about system adequacy have led to the establishment of capacity mechanisms in a number of regulatory areas. Against this background, it is essential to accurately quantify the contribution to security of supply that results from interconnectors to neighbouring systems. This paper introduces a definition of capacity value for interconnection between two systems in the form of a capacity allocation curve. Four power flow policies are proposed to encompass the full range of possible market outcomes that may affect the capacity value. A convolution-based method is presented to efficiently compute and compare capacity allocation curves, and it is applied to a model system that is inspired by Great Britain's interconnection with the continental Europe. The results indicate areas of interest for the coordination of capacity mechanisms.@en

We present a method to find the maximum magnitude of any supply-shortfall service that an aggregator of energy storage devices is able to sell to a grid operator. This is first demonstrated in deterministic settings, then applied to scenarios in which device availabilities are stochastic. In this case we implement chance constraints on the inability to deliver as promised. We show a significant computational improvement in using our method in place of straightforward scenario simulation. As an extension, we present an approximation to this method which allows the determined fleet capability to be applied to any chosen service, rather than having to re-solve the chance-constrained optimisation each time.@en