Towards optimized exchange topologies in smart distribution grids

Smart Grids are complex systems which involve multiple domains such as electrical grids and the information and communication technology infrastructure. Especially the integration of DER into the distribution grid has led to an increased complexity of grid control with a multitude of intricate optimization problems to solve. In this thesis the motivating use case are regional ancillary services. Formerly ancillary services were provided by large generation units. The shift of generation into the distribution grids leads to the necessity to provide these services at least partly with DER. As part of ancillary services, congestion management in distribution grids will be the use case with the main focus. For congestion management, a set of DER and controllable loads have to coordinate their schedules to achieve a necessary adjustment in power generation or consumption. The distribution of this deviation from the planned power output among the various units is a reactive scheduling task. The results obtained by the investigation of congestion management may be applicable to other regional ancillary services that can be performed by reactive scheduling, like voltage control.

The provision of such services with a large number of distributed energy resources and controllable loads leads to an increased complexity for the control mechanisms. This rise in complexity can be handled through the use of distributed coordination approaches. A natural way to implement these distributed algorithms are Multi-Agent Systems (MAS). MAS are a form of distributed artificial intelligence. They consist of a number of autonomous agents. An intelligent agent is a computer system which can sense its environment and take autonomous actions to accomplish its individual goals. Furthermore it is capable of social interaction with other agents or even humans. Agents can react to changes in their environment as well as to information received from social interaction. But they are also able to act in a pro-active manner. Global system behavior emerges through the interaction of agents who aim to achieve potentially contradictory individual goals (Wooldridge 2009).

In literature a vast number of possible applications for MAS in smart grids can be found: For example microgrid operation in (Pipattanasomporn et al. 2009), load management in (Amini et al. 2013) or demand side management in (Ramchurn et al. 2011).

Figure 1 shows how a MAS could be realized in a smart grid setting. The way the agents are situated in the grid is only one possibility. It shows the case where each agent is located directly on-premise, close to the DER or controllable loads they belong to. Here the agents communicate with each other via the internet. There are many degrees of possible centralization and decentralization of the agents. The alternative design concept that differs the most from the one shown in Fig. 1 would be complete centralization of the agents, for example in a data center.

The location of the agents in a central facility revokes some of the major benefits of the distributed algorithm: The robustness of the system is reduced through the reintroduction of a single point of failure. Furthermore the scalability is decreased.

Another concern are privacy issues. Especially when small DER and loads that belong to private households are included. It is likely possible to derive private data like behavioral patterns from the sets of feasible schedules agents need for the optimization process. When the agents are hosted centrally this data has to be transferred entirely. When the agents are hosted on-premise only a small fraction of this information is communicated to the outside.

This thesis focuses on cases where the agents are either completely or at least to a great extent distributed. The reasons for this limitation are the advantages this approach offers that are especially relevant for the considered use case, namely robustness, scalability and privacy.

In either case the communication of the agents is characterized by certain properties.

In this work, the primary focus lies on the exchange topology. The exchange topology determines which agents communicate directly. It is often modeled as an overlay network. Only the entities that are direct neighbors in the overlay network are exchanging information during the execution of the algorithm. The exchange topology is therefore part of the algorithm itself and effects the (runtime) behavior of the algorithm profoundly. Talbi refers to this type of algorithms as algorithmic-level parallel heuristics (ALP-heuristics).

This is the reason why the exchange topology must be designed with care, as it has a great impact on both, the output of optimization and runtime characteristics, in particular the speed of convergence, robustness to failures and communication efficiency (Baras et al. 2009).

The construction of the communication topology should lead to an appropriate degree of connectedness. Fully connected graphs are not scalable enough, whereas very sparse graphs will lead to a slow convergence speed. Furthermore certain graph topologies can result in undesirable behavior. For example, individual agents can exert greater influence than others (Baras et al. 2009). In (Nieße et al. 2017) the authors stated the communication topology as a possible cause for the convergence to local optima. This is another example for the affects the communication topology can have on the results of the algorithms.

The creation of optimal overlay networks presents a challenge that can be solved with a variety of approaches. In (Elias 2009), the author gives a broad overview of works from different contexts, which address the problem with methods from completely different areas, which range from game theory to using heuristics like simulated annealing. Furthermore, graph theory plays an important role in analyzing and modeling of overlay networks. One example is the theoretical framework for the analysis of consensus algorithms in (Olfati-Saber et al. 2007).

When the agents are actually distributed in the field, the characteristics of the underlying physical communication infrastructure (CIS) have to be considered as well. Each connection between two nodes in the overlay network represents a possibly large number of underlying physical links. Those links can have various characteristics, like delay and throughput. Different exchange topologies lead to the usage of different communication lines and hence to different communication delays. The delayed delivery of the messages between the agents can also impact the behavior and performance, especially the speed of convergence, of distributed algorithms (Meisels 2008).

In the context of smart grid applications there are even two underlying network infrastructures for a MAS: the communication infrastructure and the electric grid.

The aspects that have to be considered in the process of the exchange topology design are outlined below, using an example scenario.

Figure 2 displays the different grid-layers involved in a simple example scenario and extends thereby Fig. 1. The bottom layer shows a smart distribution grid with multiple households (blue), each provided with some combination of DER (photo-voltaic plants (green), CHP (orange)) or some kind of controllable battery (battery storage or electric vehicle (purple)). The red arrow next to the transformer indicates a transformer overload due to an oversupply of power in the grid segment. The schedules of these DER and controllable loads have to be adjusted in order to achieve a reduction of the power that is flowing over the transformer to prevent an imminent transformer overload.

A MAS is used for the coordination of the schedule adjustments between the various resources. The top-layer represents an exemplary exchange topology for the MAS. The middle layer displays the underlying communication infrastructure (CIS) that connects the agents which are placed in the households. In the design process of the exchange topology the CIS has to be modeled on an adequate abstraction level. For this purpose, the relevant characteristics have to be determined. This could be specific for the algorithm of the MAS. Quality of Service (QoS) parameters such as latency, throughput and losses will likely be of importance. A preliminary examination appears to be useful, during which the effect of the various parameters is assessed. This investigation will probably be conducted by simulation. Furthermore these aspects can be subject to changes during runtime. This aspect is not considered in this paper, as it is covered by other concepts, such as Service Level Agreements, which are implemented through Software-Defined Networking (Kreutz et al. 2015).

Other relevant input parameters for the construction of the exchange topology have to be derived from the requirements of the supported algorithm. For example, certain minimal or maximal degrees of connection could be required. Depending on the algorithm, the topology of the smart grid and properties of the resources may also be of relevance. It might be beneficial to connect nodes particularly well that are associated with resources that provide lots of flexibility or have high power outputs.

To our knowledge, there is no methodology for creating communication topologies that takes into account not only the network topology but also the characteristics of the electrical grid and the resources involved, as well as the specific communication requirements of the algorithms to be supported.

This methodology will be developed following the approach specified below.

A first step will be the identification of a number of ALP-heuristics that can be used for the case study. They must be suitable for congestion management and their exchange topology must be interchangeable. The further analysis of these supported algorithms (SUPAs) leads to a refinement of the problem definition through a detailed examination of their communication requirements. An example of the evaluation of communication behavior of distributed energy management algorithms can be found in (Hölker et al. 2016). In order to investigate the impact on the performance of the SUPAs, it is essential to first determine pertinent Key Performance Indicators (KPIs). The quality of the optimization result and the convergence speed of the SUPA are of major interest.

Robustness against agent, resource or communication line failures and communication efficiency are also likely to be of significance.

As mentioned in the previous section, a preliminary study has to be conducted to investigate the influence of different QoS-parameters on the performance of the SUPAs. Thereby, the parameters of the CIS that are relevant for the creation of the topology can be identified. Another aspect that can presumably be explored in the preliminary study is to which extend the topology of the smart distribution grid and the properties of the involved resources are of relevance for the exchange topology design. The significant parameters of both, the CIS and the smart distribution grid, have to be modeled on an appropriate abstraction level.

After the identification of the KPIs, the relevant parameters of the CIS and the smart grid and the derived communication requirements of the SUPAs, the algorithm for the creation of the exchange topology can be developed. This will be done in an iterative process of design, theoretical analysis, experimental evaluation and redesign based on the knowledge gained. The designed algorithm will probably be a heuristic as most topological design problems are NP-Hard. But further investigation will be required to classify and formalize the problem at hand.

The theoretical analysis of the runtime behavior will be conducted using the framework from (Olfati-Saber et al. 2007) and subsequent works to perform performance and convergence analysis based on tools from matrix theory, algebraic graph theory and control theory.

Since simulation models from different domains are to be used, a co-simulation approach is applied. A suitable MAS framework, which can be conveniently integrated with the co-simulation framework, has yet to be determined.

Based on the unit setup each relevant unit can be modeled and assigned to one or more agents depending on the chosen SUPA. This specifies the degrees of flexibility the agent can contribute to the optimization. The exchange topology defines the neighborhood of the individual agents. An agent communicates to all other agents, which are in his neighborhood. To investigate the impact of the delayed communication between agents this message exchange cannot be handled directly but has to be forwarded to the communication simulation and passed on from there to the target agent.

The CIS will be simulated on an abstract level, since all relevant factors like delay, throughput and loss can be modeled with varying delays. Nevertheless all QoS-parameters that were determined as relevant in the preliminary study have to be considered.

After the theoretical analysis and after the experimental evaluation, conclusions are drawn about the quality of the algorithm and improvement measures are derived. Especially in early phases a complete redesign of the algorithm is entirely possible. After the adjustments a renewed theoretical and experimental analysis is required.

The use of distributed control strategies is a promising approach to handle the increased complexity in energy scheduling problems for system stabilizing measures with large numbers of DER and controllable loads. The distribution of the control mechanism results in a need of communication between the distributed entities. The exchange topology defines between which entities information is exchanged. Hence this topology has a strong influence on the performance of the control strategy. The requirements of the control strategy as well as the characteristics of the CIS must therefore be considered during the design of the exchange topology. Furthermore, the properties of the smart distribution grid and the resources involved may also be pertinent.

The aim of this work is to develop a method for the creation of optimal exchange topologies for existing distributed control algorithms. The topology optimization shall lead to a performance improvement of the supported algorithms. Next steps include the further search for ALP-heuristics that are suitable for the intended approach and the implementation of the simulation environment. Afterwards simulation scenarios have to be defined and first own solution approaches can be experimentally tested.