Performance evaluation of German smart meter infrastructure for load management through grid operators

Abstract

The transition to a climate-neutral energy system involves the integration of many small components—on the generation as well as on the consumption side. Their controllability is a crucial part of running a reliable energy system, for example, to prevent or resolve grid congestions. The smart meter infrastructure enables a secure integration of these small system units in grid operator processes. The technical realization of German smart meter infrastructure is detailed in the guidelines of the German Federal Office for Information Security. It must undergo certification before being deployed. In order to assess whether the standardized German infrastructure meets the requirements for the use case, we conducted a performance analysis including the three factors “overall reliability”, “emerging data volume”, and “latency time”. The investigated use case is “Load Management through Grid Operators”. For data collection, we used log records from all participating devices along the process chain and recorded all network traffic. In total, we executed over 2000 commands within the test series and analyzed over 3.0 mio. data points. We concluded 98.2% successful power limitations for the overall reliability. The average emerging data volume is 14.97 kB when executing one command within one communication session. The latency time from the command until the reaction is on average 51 s, referring to the case of a curative power limitation and needing to establish a communication channel. Thus, the requirements for the considered use case are fulfilled, provided that the capacity of communication infrastructure is made available according to the number of controllable components.

Introduction

The electrification of transportation is seen as a crucial part of the decarbonization of the mobility sector. The increased installed electric vehicle supply equipment (EVSE) imply challenges for grid operators, especially at lower voltage levels. Besides the rising load, simultaneities of charging behavior can lead to grid congestions (Müller 2021). One possibility to prevent grid overloads is intelligent load management through grid operators. Following the European legislation, a standardized smart meter infrastructure is being rolled out in Germany (European Parliament 2009; Bundesamt für Sicherheit in der Informationstechnik (BSI) 2020). Perspectively, over five million load components must be equipped with a smart meter in Germany, making these controllable components for grid operators ( Bundesnetzagentur 2021). Smart meters are broadly seen as a tool for load management and various other use cases. For example the UK government suggests using a smart meter for the cost-effective realization of demand-side response for households (Departement for Bussines, Energy & Industrial Strategy 2016). The European Commission (2022) declares that smart meters are supposed to enable “Time-of-use Energy Prices” and “Dynamic Pricing”. Also Marcus Tjäder (2020) calls smart meter devices a tool for customers to interact with grid operators. In Amirshahram et al. (2019), the performance of smart meter infrastructure is analyzed based on simulations, focusing on network traffic for data transmission over long term evolution (LTE). Performance analysis for controlling loads using data from field trials has only been implemented within the C/sells project (Estermann and Bruckmeier 2021; Estermann et al. 2019). There, the analysis includes the setting of power limits via a control box.

The direct communication between smart meter infrastructure, certified according to BSI (2022), and controllable devices has not yet been analyzed. This work addresses this research gap by evaluating the performance of the German smart meter infrastructure when controlling the charging processes of electric vehicles. The communication from the distribution system operator (DSO) runs via the smart meter gateway (SMGW), which was further developed in the bidirecional charging management (BCM) project so that it translates the specification into the EEBUS standard and passes it on to the EVSE. In our work, we implemented a setup for testing the setting of power limitations for EVSE using the smart meter infrastructure. The concept includes tools that allow for a detailed process logging and measuring emerging data volume. With this data, we obtain a detailed performance analysis of the smart meter infrastructure, including overall reliability, latencies and emerging data volume for the use case “Load Management through Grid Operators”.

The investigations are part of the project BCM (Hinterstocker 2019), which analyses the use of smart meter infrastructure for the comprehension of various uni- and bidirectional electromobility use cases. The goal is to derive recommendations for further development of the smart meter infrastructure. Beside laboratory tests, 50 bidirectional vehicles are part of a field trial with pilot customers in the BCM project. The smart meter performance analysis is conducted in close cooperation and with solid support from Bayernwerk Netz GmbH and Power Plus Communications AG.

Load management through grid operator

Load Management is a proven tool for grid operators to obtain grid and system stability. With increasing loads due to the electrification of the mobility and heating sectors, load management is getting increasingly important for DSO. Generally, similar consumption behavior leads to peak loads and thus the need for a significant expansion of grid capacities when the load is not controlled (Jones et al. 2021). Depending on the ramp-up of electromobility, controlled charging can significantly reduce investments in the distribution grid infrastructure: according to Agora Energiewende (2019), savings of over 40% even up to 50% are possible (see Fig. 1). Thus, to reduce the expansion of grid capacities, DSO should have the option to control loads.

Fig. 1

Investments in distributions grids depending on electromobility ramp-up and control of charging processes (Agora Energiewende 2019)

Use case description

So far, controllable loads in the lower voltage level could agree on grid usage contracts with their grid operator. These contracts allow DSO to reduce the power for assets within a certain range and for a limited amount of time per day and year: at times when high loads are typically expected, they set a power limit to contracted consumers in a specific grid area or line. In return, the asset operators pay reduced grid usage fees (Bundesministerium der Justiz 2015). Figure 2 shows the e3-value model of the described use case for a controllable EVSE. The product is defined as flexibility provided from the owner of the EVSE to the DSO. Initially, the actors exchange information about the flexibility potential. In case of an activation, the concrete power limits are transferred. The EVSE owner receives a compensation from the energy supplier, who pays reduced grid fees to the DSO.

Fig. 2

e3-Value model according to Gordijn and Akkermans (2001) for the use case “Load Management through Grid Operator”

Technical realization

Currently, ripple control technology and time switches are mainly used to broadcast control signals (Bundesnetzagentur 2021). These are unidirectional and unflexible communication technologies. The act on the digitalization of the energy transition (GDEW-Gesetz zur Digitalisierung der Energiewende) stipulates that all controllable consumers must be equipped with a smart meter (Bundesministerium für Wirtschaft und Energie 2016). These enable bidirectional communication. Thus grid operators can query the current status of devices and receive feedback about the execution of power limitations. Smart meters consist of two main units:

• digital meter: records various values with timestamps,

• SMGW: communication unit for distributing meter data and transmitting commands.

In order to be allowed to communicate commands over the SMGW, actors must be certified as an external market participant (EMT). Also, the administrator of the SMGW is a certified role (gateway administrator (GWA), that is responsible for the configuration, operation, continuous monitoring, maintenance, and regular updates of the devices (BSI 2022)

Figure 3 shows the technical use case for setting a power limit through a DSO. In the case of a planned command, the DSO identifies the need to set a power limit for a defined time slot for specific EVSE. He transfers the limitations within the value $P_{lim}$, start time $t_{start}$ and duration $t_{lim}$ to the GWA. About one minute before reaching $t_{start}$, the GWA updates the communication profiles of the desired SMGW devices which contain the parameters set by the DSO. In case of a curative instruction, the profiles are immediately updated by the GWA. The SMGW then translates the parameters into the EEBUS communication standard and sends the commands to its connected device. The EVSE, as well as most energy management systems, is compatible with this standard (EEBUS 2022). The charging process is controlled by the EVSE so that it complies with the grid operators’ specification whilst still reaching the customers charging target.

Fig. 3

Technical use case description for the load management through DSO

Performance test

To test the use case and the data collection in a protected environment, we built a test setup. Within the further project progress, we plan to extend the data collection to pilot customers that are part of the field trial. We added logging devices in the process chain to collect data for the performance analysis.

Test setup

The test setup can be grouped into on- and off-site components and processes. The on-site setup consists of a server, a SMGW and a EVSE emulator. The server is used to transmit $P_{lim}$, $t_{lim}$ and $t_{start}$ to the GWA via HTTP (Hypertext Transfer Protocol). Thus, it replaces the role of the DSO. The EVSE emulator is connected to the SMGW via a network cable at the interface for controllable loads. The off-site part of the test setup is the system of the GWA which runs in real operation. In the considered use case, the GWA takes care of transmitting power limitations to the SMGW in time. The communication between SMGW and GWA takes place via a transport layer security (TLS) channel over LTE as the SMGW is equipped with a sim card. Thus all components and roles that are part of the smart meter infrastructure are running under real conditions, whereas the DSO and the EVSE are replaced. Figure 4 shows the test setup.

Fig. 4

Test setup, data collection and its transmission for the performance analysis

Test conduction

After submission, more than 2000 power limitations on one SMGW have been conducted and considered in the following analysis. The use case was applied for preventive and curative load management scenarios. In case of a preventive measure, the DSO transmits the limitation at least 5 min before $t_{start}$. The GWA then stores the planned power limitations and takes care of the transmission to the SMGW in time. The power limitation can then be transmitted to the EVSE and become effective in time. On a curative intervention, the load management has an immediate starting time. Thus the GWA transfers the information immediately to the SMGW. In this case, the implementation and effectiveness of the power limitation in the EVSE will be delayed by the processing time of GWA, SMGW and EVSE. In order to communicate with the SMGW, the GWA initiates a TLS session. By settings, this connection will be held open for about 3 min since its last usage. The existing session will be used if the GWA sends another request within this period. Taking above described behaviour into account, we conducted tests by varying the following settings to consider all aspects of the setup:

• the EMT sends the power limitation to the GWA either 5 min before starting time (preventive load management) or when it starts (curative load management),

• the subsequent power limitation is sent

  • within 3 min after last communicating with the SMGW (using existing communication channel)

  • more than 3 min after last communicating with the SMGW (establishing a new communication channel),

• variations in $t_{lim}$ and the time between two power limitations (including overlapping time ranges).

Data collection

The test setup is equipped with various data collectors enabling a detailed performance analysis. For assessing the emerging data volume, we equipped the SMGW with a tool that records network traffic which is then sent automatically to our central data server. In addition, the SMGW device and the EVSE emulator note relevant events and provide the records as log files. On the other side of the communication, the GWA logs relevant events and provides them as log files too. All log files are transferred to the central data server via secure file transfer protocol (SFTP). The DSO, sending power limitations to the GWA, records its activity and also transmits the data to the central server. Figure 4 shows the setup with the data collection and its transfer to the data server. For clock synchronisation, the network time protocol (NTP) is implemented on all devices of our test setup, satisfying our demands on measurement accuracy.

Evaluation

With the collected data of various logs from GWA, EVSE, and SMGW, network traffic records, and data from the DSO, we conducted evaluations on overall reliability, emerging data volume and latencies. Therefore, we analyzed log entries for relevant events, parsed network traffic, and assigned the data to power limitations.

Methodology

Along with the entire process of a power limitation setting, we extracted following events from log entries with their timestamps to analyze their occurrences, and to gather latencies between the events:

• DSO starts sending the power limitation setting to the GWA,

• GWA has received the power limitation setting,

• GWA starts connecting to the SMGW,

• TLS session is created between GWA and SMGW,

• completed handshake between GWA and SMGW,

• power limitation has been received by the SMGW,

• SMGW has changed the profile (desired setting has been processed),

• EVSE has accepted the power limit,

• GWA has been notified about the successful conduction (of the SMGW, not the EVSE),

• session between GWA and SMGW has finished.

Besides, the logs show points of failure for power limitations that have not been implemented and provide data for statistics about the overall reliability of the entire system and individual system components. The network traffic records are provided as a listing of single data packets. First, irrelevant communication for the evaluation is filtered out. Then, communication between SMGW and GWA is divided into sessions and analyzed from the beginning to the end to fully obtain the data volume. Finally, network traffic records and event timestamps are logically assigned to conduct power limitation settings.

Results

The preprocessed and combined data from different sources enables for assessment of the smart meter infrastructure for the given use case by the following three characteristics “overall reliability”, “data volume” and “latencies”.

Overall reliability

We have conducted 2039 power limitations on one SMGW with varying parametrization. Thereof, 36 commands failed, leading to a success rate of 98.2%. As reasons for the failures we found, among other things, a concurrent closing of an existing TLS session with an attempt to start a new session at the GWA. This leads to a stop of transmitting the current power limitation to the SMGW. Here, the GWA could change the system configuration to conduct various attempts to connect with the SMGW after it had failed. When regarding the communication from the SMGW to the emulated EVSE, we currently see a success rate of 51.0%. The relatively low success rate between SMGW and EVSE can be attributed to a software issue on the SMGW side, the fix for which was available at the time of publication but could not yet be tested. It is worth mentioning that this communication channel is still under development and not yet applied in real operation.

Data volume

In our tests, we varied the parametrization of power limitations and the temporal distance between them. When a power limitation setting is sent within 3 min since the last communication, the still open TLS session is used, and thus establishing a new session is unnecessary. This decreases the amount of data volume per power limitation. On the other side, when we waited for the TLS session to close until sending another power limitation, we assigned one session to one limitation setting, including establishing and closing a communication channel. Figure 5a depicts the average data volume for the transmission of one power limitation depending on the number of sent settings per session. We clearly see a decrease in the necessary data volume per transmission of one power limitation when an open session is used in contrast to the single usage of a session. Consequently, we can estimate the average data volume for establishing and closing a session to approximately 5.5 kB, and assign approximately 9.5 kB of the data volume to transmit one power limitation.

Fig. 5

Characteristical data volume for the transmission of a power limitation

Figure 5b shows a box plot representation¹ with the data volume for transmitting a power limitation within one session. One can recognize two major groups of needed data volume. We suppose this might be due to a frequent retry of sending packets.

Latencies for preventive load management

For preventive power limitations, the command is sent to the GWA 5 min before $t_{start}$. Figure 6 depicts the statistical occurrences of events along the process of a preventive power limitation setting, respective to its planned starting time $t_{start}$ each. It shows, that on average the GWA starts connecting to the SMGW 50 seconds before the planned starting time. This is due to the parametrization at the GWA and could be set to an earlier point in time. After establishing the connection and receiving the power limitation via a PUT request, the profile in the SMGW has changed on average 14 s before $t_{start}$. The exact moment, when the EVSE accepts the power limit, cannot be stated here because of too few successful communication occurrences between SMGW and EVSE. The GWA is informed about the changed profile in the SMGW on average by $t_{start}$, and the session is closed approximately 180 s after that. Note that the GWA is informed about a successful profile change in the SMGW, independent from the successful acceptance by the EVSE.

Fig. 6

Occurrence of events for a single preventive power limitation within one session, respective to the planned starting time. The command is sent to the GWA 5 min before starting time and thus not shown here

The box plot in Fig. 7 shows the duration of procedures between the above-shown events. This representation emphasizes the single long-lasting and strongly differing procedures. Our data shows that processing the power limitation request within the SMGW takes the longest amount of time: approximately 27 s, with a standard deviation of roughly 4 s.

Fig. 7

Duration of selected procedures for a preventive power limitation. Session is opened and closed, request is sent to the GWA 5 min before starting time

Power limitations that are transmitted to the SMGW within an opened session and that are being sent to the GWA 5 min before start, depict vaguely the same pictures as above; they solely do not contain the events of opening and closing their session. Therefore, it would be unnecessary to show those figures here.

Latencies for curative load management

For curative power limitations, the command is sent to the GWA at the start time $t_{start}$. Figure 8 depicts the occurrences of events along the process of a curative power limitation, respective to the starting time $t_{start}$ each. On top of that, Fig. 9 shows the duration of two additional procedures between the beforehand shown events. The other duration box plots are not depicted because they are similar to the values in Fig. 7.

Fig. 8

Occurrence of events for a single curative power limitation within one session, respective to the planned start time. The request is sent to the GWA at start time

Fig. 9

Duration of selected processes to conduct a curative power limitation. Session is opened and closed, request is sent to the GWA at start time

After the DSO has sent the power limitation request to the GWA, we can observe a delay of an average of 11 s at the GWA before starting to connect to the SMGW with a noticeably differing duration. There is definitely potential for improving rapidity. The following steps creating a connection to the SMGW, receiving the power limitation in the SMGW and changing the profile, have similar latencies as in the case of preventive load management. After an average time of 48 s, the SMGW has updated its profile. And 3 s later, after roughly 51 s since sending the command, the EVSE has accepted the power limitation. Power limitations transmitted to the SMGW during an already started session depict approximately the same results as before. Thus, it would be unnecessary to show those graphics again.

Conclusion

The results obtained in this paper, provide information about the characteristic values for overall reliability, data volume and latencies for the use case “Load Management through Grid Operators”. Within the obtained values, we are able to assess whether the smart meter infrastructure meets the requirements for the described application. The present findings confirm that the latency time from sending a power limitation request until the reaction of the component is sufficiently small for preventive measures. For curative measures, reaction time could be reduced by improving process efficiency and configuration parameters at the GWA. It is worth mentioning, that we performed our test on a setup that demonstrates the proof of concept. Future work at the GWA and SMGW focuses on an improved performance and uses more powerful hardware and optimized software. Regarding the evaluation of the overall reliability, we need to differentiate depending on the framework conditions. For example, when only a very limited amount of controllable load components is connected to a specific grid line, it is crucial that a limitation setting is successful. The DSO must consider the overall reliability in his load management measures. We must also consider the specific framework conditioning when analyzing the emerging data volume. Depending on the number of simultaneously communicating SMGW, the digital infrastructure needs to be provided sufficiently to avoid bottlenecks in communication, leading to delays.

Besides the described use case, the characteristic values obtained from the tests also enable the assess of further use cases for their suitability to be performed on the smart meter infrastructure. This especially addresses the verification of requirements regarding latency. A comparison to previous studies can hardly be drawn, as the communication cascades differ. But when making a high-level comparison of the findings of the C/sells project (Estermann and Bruckmeier 2021) and our results, we observe significant performance improvements. The latency for the transmission of a limitation is much smaller (reduction of about 50%) and data volume is reduced from 24 kB to 15 kB. Results for the general reliability are equivalently high for the communication ways in C/sells and BCM.

Ideally, the finding from this work should be replicated in a study where a larger number of SMGW is included. Also, processes should be evaluated when the roles of DSO and the EVSE are not substituted but taken over by real roles and components. Future research on smart meter performance should also extend the scope to other communication technologies. The LTE connection to the SMGW used here can be replaced by power line communication, for example, in order to investigate the effect of the communication technology on the performance of the smart meter infrastructure. Furthermore, it should be mentioned that the test was conducted in an urban area with very stable LTE connections, which is not the case in all parts of Germany. Further tests, both in urban and rural areas are necessary to obtain representative results for different framework conditions. Notwithstanding the relatively limited sample and the evaluation of only one SMGW, this work offers valuable insights into the processes and performance of the German smart meter infrastructure, including the setup of a test environment that simulates real conditions.