- Research
- Open Access
- Published:
Agent-based dynamic optimization of local controller configurations in converter dominated electricity grids using decoder functions
Energy Informatics volume 1, Article number: 46 (2018)
Abstract
With an increasing share of distributed energy resources (DER) in the electrical energy system it is becoming more and more important that DER not only take part in active power provision but are also involved in the provision of ancillary services like frequency control or voltage regulation. Due to the large number of DER connected to the lower voltage levels via power-electronic converters the distribution grid evolves from a formerly mostly passive to a highly active system with a high number of actuating variables distributed over multiple stakeholders. The coordination and optimization of this kind of distribution grid requires new control and optimization approaches, not only with regard to the distribution grid itself, but also with regard to the coordination with the overlying transmission grid. This abstract presents first ideas of a PhD-project that aims to use machine learning surrogate models and decoder functions for agent-based dynamic optimization of local controller configurations particularly with regard to voltage regulation. Decoder functions derived from machine learning surrogate models abstract optimization problems from technical system specifications and allow for constraint-free optimization with standard optimization heuristics such as evolutionary optimization methods.
Background
The electric energy system has undergone major changes in recent years. Gradually, large scale fossil or nuclear power plants are being replaced by distributed energy resources (DER) often based on renewable energies like wind power and solar energy. In contrast to large scale thermal power plants, DER are mostly connected to the low and medium voltage grids. Up to now, ancillary services like frequency or voltage control, which are essential for a stable operation of the electric power system, are predominantly provided by the remaining large scale thermal power plants (Braun 2009). To reduce carbon dioxide emissions large coal-fired power plants are to be replaced by further expansion of DER. This makes it increasingly important that DER not only contribute to active power provision, but are also involved in the provision of ancillary services (Deutsche Energie-Agentur GmbH 2012).
DER such as photovoltaic plants or wind turbines are normally connected to the grid via power-electronic converters. In the future converter dominated grid DER must take over ancillary services so far merely provided by large scale power plants (e.g. primary reserve for frequency control or reactive power provision for voltage regulation in the transmission grid). At the same time, DER meet new challenges, which arise from their distributed nature, their connection to the low- and medium voltage grid and the fluctuating and to some extent uncertain supply of wind power and solar energy. Among these challenges are voltage deviations and voltage band violations in the distribution grid, congestion of lines and transformers and a changing dynamic behavior of the power system.
Power-electronic converters are technically capable of providing arbitrary reactive power along with active power as long as current and voltage limitations are met. In Germany, reactive power control capability for low- and medium voltage connected DER is defined in the technical guidelines VDE AR 4105 (low voltage) and the BDEW Medium Voltage Guideline (VDE, FNN 2011; BDEW Bundesverband der Energie- und Wasserwirtschaft e.V 2008). The guidelines define local control characteristics such as cosφ(P) or Q(U) for reactive power supply of converter-connected DER. At the same time, grid operators incorporate new operating equipment to solve voltage balancing problems. This is changing the formerly largely passive distribution grid to an active system with a great number of actuating variables, which have to be coordinated to optimize overall system operation and to avoid controller conflicts (i.e. adverse control actions directed at one or more units based on different control targets and stakeholders) leading to inefficiencies or even instabilities of the system.
The DFG-project “Reliable Operation of Converter-Dominated ICT-Reliant Energy Systems”, in the context of which this work is conducted, aims to develop and evaluate a decentralized agent-based control and information exchange structure for grid operation with the help of DER and their coordination with top-level control centers. The main idea is to combine local converter-based control with multi-agent based optimization to form a hybrid control system as depicted in (Fig. 1). The converter-based control guarantees real-time behavior and relaxes time constraints for the overlying multi-agent system, which in turn ensures long-term optimization of system operation and controller conflict mitigation by dynamically optimizing the configuration of the local controllers.
Hybrid control system
For optimization a formal description of the system boundaries is needed. This is typically done by providing constraints next to optimization objectives. For dynamic optimization of controller configurations this approach is not suitable, as it would require the reformulation of the optimization problem each time the system changes, for example when new DER are connected to the system or when the flexibility of an energy unit with regard to one service is temporarily reduced by other higher prioritized services.
A promising approach to deal with dynamic optimization problems is to use a machine learning surrogate model to describe the flexibilities of the underlying systems. A surrogate model is a black-box model that abstracts from the technical model of a complex system and approximates the observable behavior (without the internal causal relations) based on a limited set of sampled data. From the surrogate model a so-called decoder can be derived, which maps the unrestricted search space to the solution space and thus allows for contraint-free optimization with standard optimization heuristics such as evolutionary optimization methods (Bremer 2015).
In the PhD-project presented in this abstract, this approach shall be applied and evaluated for dynamic agent-based optimization of local controller configurations, particularly with regard to voltage regulation in active distribution grids. As mentioned before, the decoder approach does not require the reformulation of the optimization problem when parameters of the underlying system change and is thus more flexible and easier to automate. But still surrogate model and decoder have to be learned resp. derived from scratch in case of system changes, which is computationally expensive and time consuming. Therefore, one main research goal in this PhD-project is to develop a method, which allows for dynamic adoption of surrogate model and decoder to changes in scenario settings.
To allow for the optimization of higher grid levels based on lower level flexibilities, a description of the aggregated flexibilities resp. constraints of downstream entities is required. Therefore, another important research goal is to develop fast training set sampling techniques for ensembles (groups) of DER avoiding the distribution folding problem (the distribution of the sum of independent random variables results from the convolution of the individual random variables) when combining independently sampled training sets from different downstream entities for training aggregated decoders in upstream agents.
Research questions
As already mentioned in the previous section, the overall objective of the PhD-project presented in this abstract is to answer the following main research question:
-
(RQ 1) How can decoder functions be used for solving dynamic optimization problems and mitigating controller conflicts in agent-based distributed optimization of local controller configurations?
So far, two important requirements for the applicability of the decoder approach in the above-mentioned use case have been identified: First, sampling strategies for groups of DER are required to enable learning of aggregated flexibilities, which is needed for optimization on different grid levels. Second, methods for an on-line update of decoders are required. These update methods not only allow for fast adjustment to changes in downstream units (e.g. operational state, behavior or commitment to higher priority ancillary services) but also open up the possibility to mitigate controller conflicts by modelling a prioritization of arriving set points from upstream units, rephrase them as restrictions of the feasible regions and finally update the concerned decoder accordingly. From these requirements two sub-research questions have been derived:
-
(RQ 1.1) How can training sets for surrogate models of aggregated flexibilities from downstream units be sampled avoiding the distribution folding problem, which occurs with the naive approach of combining independently sampled training sets from downstream units?
-
(RQ 1.2) How can existing surrogate models and decoders be adopted on-line to changes in underlying scenario settings to avoid computationally expensive and time consuming retraining of decoders from scratch?
From the research questions mentioned above two main development objectives (DOs) can be derived: First the development of a fast training set sampling technique for training groups of DER, which avoids the distribution folding problem (DO 1). And as a second development objective, the extension of the decoder approach by a method which allows for updating of decoders based on the mere difference between two successive situations (DO 2). Figure 2 shows how the DOs fit into the decoder approach. The decoder approach is explained in more detail in the next section.
Process chain of decoder approach according to (Bremer 2015) and intended adaptions/extensions (denoted by dashed lines)
Related work
In this section related work from the fields of constraint-free optimization with decoders and agent-based ancillary service provision by DER is presented.
In (Bremer 2015) constraint-free optimization with support vector decoders is used for predictive scheduling of energy units. From a methodological perspective, the concept presented in (Bremer 2015) realizes the abstraction from the technical system in several steps (see Fig. 2): First, in a simulation based sampling process for each participating DER a set of example schedules is generated. In a second step, these schedules are not interpreted as time series but as vectors. This allows for a geometric interpretation of the solution space and opens up the possibility to learn the surface boundary, which separates valid from invalid schedules, by means of support vector data description (SVDD). Finally, this type of description offers a possibility to derive a decoder, which maps the unrestricted search space to the restricted solution space, and thus allows for constraint-free optimization. (Fröhling 2017) suggests a modified variant of the approach in (Bremer 2015), which allows for the abstract description not only of single energy units but also for coalitions of energy units. For this purpose, in (Fröhling 2017) an appropriately adapted search space model is developed, the so-called cascade classification model. It consists of a cascade of overlapping low-dimensional feature classifiers taking advantage of the fact that neighboring time steps employ a high correlation and the correlation decreases with increasing distance between two time steps. One problem with this type of search space representation is that so far it lacks an appropriate decoder. Furthermore by exploiting the high correlation between neighboring time steps, the cascade classification model takes advantage of a problem specific property of predictive scheduling, which hinders the applicability to other problem statements. Nevertheless, it is a promising entry point for answering RQ 1.1.
Concerning dynamic adaption to changes in scenario settings (RQ 1.2) a master’s thesis was conducted at the Energy Informatics Department of Oldenburg University (Faerber 2018). The thesis was originally intended to develop approaches for cutting and combining search space models by the example of predictive scheduling of CHP units and heat pumps. However, it turned out that for the intended use case even small changes in heat demand lead to largely disjunct search space models. The similarity measures for search space models used in the thesis are relevant for RQ 1.2 nevertheless. Regarding RQ 1.2 there exist few approaches for implementing methods for adaptive learning with support vector approaches in general (e.g. (Gieseke et al. 2012; Laskov et al. 2006)), but for the case of SVDD, comparable support is not yet available. Nevertheless, concepts for incremental learning of SVDD models exist (e.g. (Yin et al. 2014)) and may be candidates for adaption.
In (Lehnhoff et al. 2013) an integrated approach for identifying distributed coalitions of agents representing units capable of providing frequency response reserve is introduced. For each participating unit individual droop control parameters are tuned taking into account opportunity costs, device specific reliabilities and small-signal stability of such a coalition. Implications of frequency response for voltage levels are not considered in this paper. In the literature, agent-based approaches for the provision of ancillary services by DER are often discussed in the context of the microgrid concept. Microgrids are distribution systems with DER, storage devices and controllable loads, operated connected to the main power network or islanded, in a controlled, coordinated way (Hatziargyriou 2014). In connected mode, from the grid operators point of view, a microgrid can be regarded as a controlled entity within the power system that can be operated as a single aggregated load or generator and also provide ancillary services for the distribution grid it is connected to (Hatziargyriou 2014). In (Olivares et al. 2014) an overview of agent-based concepts applied to microgrid control is provided. The use of decoder functions in optimization of local controller configurations so far has not been discussed in microgrid related literature.
Methodology
To answer the research questions the following steps are envisaged: In a first step the degrees of freedom of DER usable for frequency and voltage control and possibly conflicting optimization goals are assessed. In a second step simulation models of different types of DER including their power-electronic converters reflecting these degrees of freedom are required. They serve as input for the Ensemble Sampling (DO 1) which is to be added to the decoder approach from (Bremer 2015) (see Fig. 2). The Ensemble Sampling can be taken as a distributed optimization problem with the objective of optimizing the distribution of instances in the resulting aggregated ensemble training set. To evaluate the distribution of the aggregated training set, suitable metrics have to be found. Candidates for this were identified in (Bremer and Lehnhoff 2018a; Bremer and Lehnhoff 2018b). In these articles also two central approaches to overcome the folding problem in ensemble scheduling using a Covariance Matrix Adaption Evolution Strategy (Bremer and Lehnhoff 2018a) and Simulated Annealing (Bremer and Lehnhoff 2018b) are proposed. Both are to be investigated with regard to their applicability in a fully distributed approach for example with the distributed optimization heuristic COHDA (Hinrichs and Sonnenschein 2017). Work on DO 2 is still at a preliminary stage. A possible first step would be to apply the methods for comparing the similarity between consecutive situations proposed in (Faerber 2018).
The hybrid control system depicted in (Fig. 1) and the DOs presented before are to be evaluated by means of co-simulation based computer experiments. The co-simulation framework mosaik (Lehnhoff et al. 2015) is intended to be a central component of the evaluation framework and shall be used to couple the MATLAB-based steady-state electricity grid simulator provided by the project partners, the Python-based agent framework aiomas (Scherfke 2014) and in a later stage an ICT simulator, which has yet to be defined.
Abbreviations
- DER:
-
Distributed energy resources
- DO:
-
Development objective
- RQ:
-
Research question
- SVDD:
-
Support vector data description
References
BDEW Bundesverband der Energie- und Wasserwirtschaft e.V. Technische Richtlinie Erzeugungsanlagen am Mittelspannungsnetz. 2008. https://www.enercity-netz.de/pool/downloads/netze/erzeugungsanlagen-am-mittelspannungsnetz.pdf. Accessed 9 Apr 2018
Braun M (2009) Provision of Ancilliary services by distributed generators – technological and economic perspectives. Universität Kassel. http://www.upress.uni-kassel.de/katalog/abstract.php?978-3-89958-638-1. Accessed 18 Mar 2018
Bremer J (2015) Constraint-Handling mit Supportvektor-Dekodern in der verteilten Optimierung. Carl von Ossietzky Universität Oldenburg. https://oops.uni-oldenburg.de/id/eprint/2336. Accessed 2 Mar 2018
Bremer J, Lehnhoff S (2018a) Phase-space sampling of energy ensembles with CMA-ES. In: Sim K, Kaufmann P (eds) Applications of evolutionary computation. Springer International Publishing, Cham, pp 222–230. http://link.springer.com/10.1007/978-3-319-77538-8_16. Accessed 11 Apr 2018
Bremer J, Lehnhoff S (2018b) Unfolding ensemble training sets for improved support vector decoders in energy management. In: Proceedings of the 10th International Conference on Agents and Artificial Intelligence. Science and Technology Publications, Funchal, pp 322–329. https://doi.org/10.5220/0006543503220329
Deutsche Energie-Agentur GmbH. dena-Verteilnetzstudie – Ausbau- und Innovationsbedarf der Stromverteilnetze in Deutschland bis 2030. 2012. https://www.dena.de/themen-projekte/projekte/energiesysteme/dena-verteilnetzstudie/. Accessed 18 Mar 2018
Faerber B (2018) Ein adaptives Supportvektor-Suchraummodell für dynamische, verteilte Energiesysteme. Masterarbeit CvO Universität Oldenburg, Oldenburg
Fröhling J (2017) Abstract flexibility description for virtual power plant scheduling. Carl von Ossietzky Universität Oldenburg. https://oops.uni-oldenburg.de/id/eprint/3373. Accessed 30 Mar 2018
Gieseke F, Airola A, Pahikkala T, Kramer O (2012) Sparse quasi-newton optimization for semi-supervised support vector machines. In: Proceedings of the 1st Conference on Pattern Recognition Applications and Methods, vol 1, pp 45–54. https://doi.org/10.5220/0003755300450054
Hatziargyriou N (ed) (2014) Microgrids – architectures and control. Wiley, Chichester
Hinrichs C, Sonnenschein M (2017) A distributed combinatorial optimisation heuristic for the scheduling of energy resources represented by self-interested agents. Int J Bio-Inspired Comput 10:69–78
Laskov P, Gehl C, Krüger S, Müller K-R (2006) Incremental support vector learning: analysis, implementation and applications. J Mach Learn Res 7:1909–1936
Lehnhoff S, Klingenberg T, Blank M, Calabria M, Schumacher W (2013) Distributed coalitions for reliable and stable provision of frequency response reserve. In: IEEE International Workshop on Intelligent Energy Systems (IWIES), pp 11–18. https://doi.org/10.1109/IWIES.2013.6698554
Lehnhoff S, Nannen O, Rohjans S, Schlogl F, Dalhues S, Robitzky L et al (2015) Exchangeability of power flow simulators in smart grid co-simulations with mosaik. In: 2015 Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES). IEEE, Seattle, pp 1–6. https://doi.org/10.1109/MSCPES.2015.7115410
Olivares DE, Mehrizi-Sani A, Etemadi AH, Canizares CA, Iravani R, Kazerani M et al (2014) Trends in microgrid control. IEEE Trans Smart Grid 5:1905–1919
Scherfke S. Aiomas documentation. 2014. https://aiomas.readthedocs.io/en/latest/
VDE, FNN. VDE-AR-N 4105 Erzeugungsanlagen am Niederspannungsnetz – Technische Mindestanforderungen für den Anschluss und Parallelbetrieb von Anlagen an das Niederspannungsnetz. 2011
Yin G, Zhang Y-T, Li Z-N, Ren G-Q, Fan H-B (2014) Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128:224–231
Acknowledgements
This research is conducted at Carl von Ossietzky Universität Oldenburg as part of the project “Reliable Operation of Converter-Dominated ICT-Reliant Energy Systems” funded by the German Research Foundation (DFG) under the supervision of Prof. Dr. Sebastian Lehnhoff. The valuable comments of Prof. Dr. Wilfried Elmenreich during the shepherding process of the PhD Workshop “Energy Informatics” are gratefully acknowledged.
Funding
Publication costs for this article were sponsored by the Smart Energy Showcases – Digital Agenda for the Energy Transition (SINTEG) programme.
Availability of data and materials
Not applicable
About this supplement
This article has been published as part of Energy Informatics Volume 1 Supplement 1, 2018: Proceedings of the 7th DACH+ Conference on Energy Informatics. The full contents of the supplement are available online at https://energyinformatics.springeropen.com/articles/supplements/volume-1-supplement-1.
Author information
Affiliations
Contributions
The author read and approved the final manuscript. The content of the manuscript was created entirely by JG.
Corresponding author
Correspondence to Johannes Gerster.
Ethics declarations
Competing interests
The author declares that he has no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Gerster, J. Agent-based dynamic optimization of local controller configurations in converter dominated electricity grids using decoder functions. Energy Inform 1, 46 (2018). https://doi.org/10.1186/s42162-018-0049-8
Published:
Keywords
- Ancillary services
- Voltage regulation
- Machine learning
- Decoder
- Distributed control
- Multi agent systems
