A flexible ICT architecture to support ancillary services in future electricity distribution networks: an accounting use case for DSOs

With the increased penetration of distributed renewable energy sources (DRES) in the grid, new pathways are required to keep the electricity distribution system stable. The provision of ancillary services (AS) by the DRES can contribute in this regard. However, it is necessary to communicate the need for AS from the third party providers such as distribution system operator (DSO) to the DRES in an efficient and scalable manner. To this end, a flexible information and communication technology (ICT) architecture is presented in this paper, and the requirements for the architecture are elaborated. We argue that this architecture is capable of supporting the present and future needs of electricity distribution networks. To illustrate its utility and effectiveness, an accounting use case for DSOs has been presented; it describes a remuneration scheme for the AS provision. A dashboard has been developed to enable communication via this architecture and to allow control of the grid. In addition, a distributed ledger technology for the realization of accounting has been analysed with respect to its scalability and performance capabilities.

By adopting a unified bottom-up approach, the EASY-RES consortium is focused on developing novel control algorithms for all converter-interfaced distributed renewable energy sources (DRES), to enable them to operate similarly to conventional synchronous generators (SGs), providing the grid with inertia, damping of transients, reactive power, fault ride through and fault-clearing capabilities, and adaptable response to primary and secondary frequency control. These new functionalities are being transparent to all grid voltage levels.
The EASY-RES approach is based on the distribution network segmentation into small Individual Control Areas (ICA), where the DRES and properly sized storage systems optimally coordinate via suitably designed information and communication technology (ICT) infrastructure. This infrastructure provides Ancillary Services (AS) such as inertial response, reactive power support, power smoothing, and contribution to fault-clearing in a bottom-up approach: prosumers and independent RES producers to DSOs, and DSOs to TSOs. By evaluating the costs and benefits of the developed functionalities, viable business models are being developed for the aforementioned stakeholders. Finally, modifications to the existing grid codes can be suggested for the implementation of the developed AS.

Project key objectives
The key objectives of the project are: Increasing the robustness of the power system towards abrupt frequency changes by introducing virtual inertia and damping in DRES, thereby adopting characteristics similar to SG. Contributing to grid stability by providing frequency-dependent active power. Increasing the DRES penetration levels at both low voltage (LV) and medium voltage (MV) level, while avoiding investments for grid reinforcement. Making the RES more grid-friendly by reducing the short-term electric power fluctuations at both DRES and HV/MV substation level and introducing active harmonics filtering to each DRES converter. Preserving the long-term grid security even under very large DRES penetration, by reducing reserve requirements after fault recovery. Developing viable business models for all the stakeholders by proposing new metrics for the quantification of the various AS and evaluating the economic cost and benefit of all the developed AS. Flexible ICT architecture for ancillary services

Project partners (Continued)
One of the key outcomes in this project for providing secure ASs is the development of a secure and flexible ICT architecture for the future electricity distribution networks (i.e. the grid) where a wide range of DRES are being used, forming a large DRES network. An important part of this large energy network is to integrate all the DRES to provide ASs such as frequency regulation (Gerard et al. 2018). This is due to the fact that the DRES energy output is volatile, caused by, for example, changes in wind or sunshine (Moriarty and Honnery 2012). However, to enable efficient services for DRES in such a large network and to maintain Quality of Service (QoS) with an everincreasing load on this network, a novel ICT network architecture is required. To prevent power fluctuations, the AS must be provided directly from the DRES, giving more power to the DSO over control and the regulation of the grid. Combining all these aspects, this paper proposes a flexible ICT architecture and describes how a third party such as an aggregator, retailer, or DSO can communicate the demand for AS to the DRES. This flexible platform allows these third parties to make more-informed decisions, enforce energy policies, and to have a close interaction with the renewable energy market. The associated third party can then deploy several applications such as accounting, authentication, etc. to communicate with the DRES and end-users via the architecture. In this regard, the operational and functional requirements for these applications need to be analysed and addressed in the system architecture (Niedermeier et al. 2018;Rohjans et al. 2012). These requirements are as follows.

Communication and bandwidth
Communication between the components can be realized via cellular networks, powerline, Ethernet, or fibre for example. The network needs to be able to transfer the set points without major latencies. The set points of inertial response and primary frequency response are required to be sent every half hour, and for reactive power every 2 minutes.

Monitoring
The values of inertial response and primary frequency response have to be recorded every 0.5 s; values for fault clearing, harmonic mitigation, high frequency power smoothing every 10 s; and for reactive power exchange every two minutes.

Storage
The monitored data have to be stored for further analysis and for legal reasons. A high granularity results in a huge amount of data that has to be both sent and stored. It is better to store the data (which is intermediately aggregated in the microgrid) in the low voltage grid and then forward it to the DSO. This data can total approximately 1.4 GB/ day.

Hardware requirements
The system needs edge devices at the converters (e.g., an embedded system), a central controller in each medium voltage (MV) grid, as well as (cloud) servers at the DSO.

Accounting
The process of accounting needs to fulfil requirements such as availability, privacy, scalability, and security. The monitored data has to be of suitably fine granularity to guarantee a fair payment.

Security and resilience
Security and resilience are necessary to meet the requirements of a critical infrastructure (Neureiter et al. 2016;Hutchison and Sterbenz 2018) because the grid is subjected to cyberattacks as well as other challenges including natural disasters and component failures. A public key infrastructure for authentication, confidentiality and integrity should be deployed. Clearly, resilience needs to be designed natively in the architecture (Hutchison and Sterbenz 2018).

Openness and extensibility
The architecture must provide numerous interfaces and wrapper classes that mediate between different components. Further, the extensibility of the architecture must be ensured.

Scalability
Scalability is necessary in a sense that numerous DRES are actively supported in the future grid ecosystem. The modular structure of the architecture with open interfaces allows this. Size scalability is realized with load balancers and virtualized network functions (VNF). Geographic scalability is possible because every medium and low voltage grid has its own controllers and network functions.

Accounting use case
To show the successful working of the proposed architecture by adhering to these operational and functional requirements, we present an accounting use case, which is one of the major activities required by the DSO for decision making. For this purpose, a dashboard has been implemented and the feasibility of the accounting process has been evaluated. Demand for an AS comes either automatically based on calculations or forecasts on the monitored data or manually from the DSO via the dashboard. An AS provider fulfils the demand by offering adequate AS. A contract is generated via the dashboard, specifying amount, time, and type of AS. For resilience and security, the contract is stored encrypted on several distributed replicated databases. After the service delivery, the DSO can remunerate the AS provider for the service. This transaction is again stored in the distributed database.

ICT Architecture & Methodology
To meet the requirements as listed in the previous section, a suitable ICT architecture has been designed as illustrated in Fig. 1. This architecture is capable of monitoring the data from DRES, detecting anomalies in the system, providing secure and resilient data processing (data communication, data storage, and privacy), and reliable accounting.
The proposed framework provides an easy (yet efficient) way to incorporate and utilize the DRES in smart grid IoT networks. This ICT framework provides the intended capabilities using the virtualization aspects of programmable and re-usable functions to provide the services based on the gathered data. The control signals to manage the physical infrastructure for providing, for example, ancillary services would be sent to the hardware infrastructure using the application programming interface (API). The various components of this framework are explained as follows.

Hardware infrastructure
This component comprises all the hardware devices ranging from renewable sources like solar PV panels, wind power plants to the grid's transmission and distribution infrastructure consisting of transmission lines and microgrids.

Existing system
This is the present and traditional system of the grid functionality which consists of the management of physical infrastructure, energy management and battery management systems.

Present capabilities
The present capabilities involve authentication (and security overall), accounting, dashboard management, network function virtualization, etc. to carry out the various tasks in the smart grid. All of these capabilities are achieved by using the API that acts as an interface between the software and the hardware. These different capabilities use different methods to communicate with the API; e.g., authentication requires public key cryptographic primitives, the dashboard uses SQL database libraries, etc.

Future applications
The applications related to providing ancillary services (using the present capabilities as well as adding new capabilities in the near future), the mobile apps (to visualize the dashboards) and other future extensions (such as introducing distributed ledgers for accounting and authentication) are included in this component of the ICT framework, and these would be supported accordingly. API: The API is the main component of the proposed ICT framework and has been discussed above briefly. All the components in the API are placed on virtual infrastructure which can be scaled up according to the service requirements. This has a dual benefit: firstly, multiple requests do not wait for each other to be served first in order to free up the requested service (thus decreasing the response time) and secondly, the virtual functions are destroyed after the request is served to save the network infrastructure resources. The components of the API are described in detail as follows: a. Request Engine: It is responsible for receiving (and sending) the request (and response) signals from (and to) the other components of the grid. It can be considered as a first and only point of contact to outside entities of the API. The request engine also contains a temporary memory to store the requests and the response results. b. Orchestrator: The orchestrator is the central component of the API. It has a bi-directional link with all other entities in the API and does not communicate outside these entities. It communicates to the software components (outside the API) using the request engine. It is responsible for processing the request received at the request engine and for taking actions accordingly. To process the request, it invokes instances from the protocol stack API, resource API and application API; and it ultimately communicates with the associated hardware infrastructure (identified in the request) using the interfaces. c. Protocol Stack API: The protocol stack API holds the instances of standards and protocols that are to be used for communicating different requirements between various entities such as IEEE 1547, IEEE 61850, MODBUS, etc. (Jindal et al. 2019). Depending on what is to be communicated, the orchestrator calls for the protocols/standards to communicate between the software and hardware infrastructure. d. Resource API: The resource API stores the resources necessary to carry out various functionalities and for providing flexibility to the entire grid network. It has resources for virtual machine migration (for virtualization), database engine (for dashboard management and data analytics), key exchange (for ensuring security), etc. e. Application API: The application API has the application related instances such as JSON, WebSocket, REST, etc. which are required to make a connection with the interfaces to extract the data from the hardware infrastructure. f. Interfaces: These act as a gateway between the physical infrastructure and orchestrator. The orchestrator (or ultimately the external software application) connects to the physical hardware with the help of the various interfaces such as serial interface, IP interface, PLC, Ethernet, etc.

Dashboard and accounting
This section presents how accounting services communicate with the grid over the ICT architecture. In this regard, a dashboard for monitoring, control, accounting, and communication with the grid was implemented. The accounting process enabled by the dashboard and the ICT architecture is shown in Fig. 2. Moreover, results on scalability and performance of this use case are discussed as well.

Dashboard
The dashboard is used by DSO, TSO, and AS providers. To restrict access of users on system resources, role-based access control (RBAC) policies are enforced. For example, the DSO can monitor and control the grid, and use it for accounting purposes. The dashboard is accessible via a web browser. Figure 2 shows the steps in the dashboard and the respective calls in the architecture for the accounting use case described above. The dashboard pages are shown in white boxes, the respective architectural elements in coloured boxes. Function calls are illustrated in angle brackets. For reasons of clarity, function parameters are not shown. The complete process is briefly described as follows. First, the DSO logs in with username and password. The authentication service verifies the credentials. Based on monitoring and forecasts, the necessary AS are calculated and then requested via request engine and orchestrator. Suitable interfaces are used to communicate with the hardware infrastructure to check the availability of AS. If the request can be fulfilled, a contract is created, specifying the amount, type and the time/ duration of the AS. This contract is stored via the architecture in replicated databases, where the access is enabled by the resource API.

Feasibility and scalability of accounting
The contract creation and storage require a certain time and can be computationally complex. The question arises whether the architecture and the accounting service can realize those requirements. Adequate accounting technology must be found in this case. Tendermint 1 is a distributed ledger technology that appears promising. It provides replicas and Byzantine fault tolerance without the need of trusted third party. It is able to handle numerous contracts and transactions for AS. Therefore, an analysis was conducted how long it takes for a transaction (i.e., the demand for and offer of an AS with the contract agreement upon it) to be persisted, i.e., written in the database. For every analysis, each of the N replicas had two Intel Xeon E7 CPUs (2.4 GHz) and 128 GB of RAM available, connected by a 1000 MBit/s Ethernet. Fifteen thousand transactions were sent per second to the platform. Figure 3 shows that the persistence time is still less than 2 seconds with 20 replicas. Within this time, the contract is communicated and stored in every replica. The more replicas, the higher the availability. The DSO is able to define the number of replicas within its grid.
Considering scalability, the number of transactions (contracts) that can be performed per second were analysed. The results for which are shown in Fig. 3. With 20 replicas, about 5000 transactions are possible per second, meaning that about 5000 offers and demands in the form of contracts can be performed every second. Considering that each DSO will have this capacity, these numbers show the feasibility of a secure and reliable accounting service in future grids.

Conclusion
This paper has presented the requirements for a secure and resilient ICT architecture for future electricity distribution networks. The operational and functional requirements were specified which should be accounted for while designing any communication-based architecture for large DRES systems to provide AS. The use case of accounting was presented from a DSO perspective to show the working of the proposed ICT architecture. With the developed dashboard, the DSO can control and monitor the grid. An analysis of scalability and feasibility of the accounting functionality in the presented scenario has shown that it is secure, fast, and resilient.
In the future, we will integrate other AS such as voltage regulation and frequency control and will make use of open source virtual infrastructure management and 5G networking to provide additional ICT capabilities and a faster response.