A hierarchical and modular agent-oriented framework for power systems co-simulations

Abstract

During the last decades, numerous simulation tools have been proposed to faithfully reproduce the different entities of the grid together with the inclusion of new elements that make the grid “smart”. Often, these domain-specific simulators have been then coupled with co-simulation platforms to test new scenarios. In parallel, agent-oriented approaches have been introduced to test distributed control strategies and include social and behavioural aspects typical of the consumer side. Rarely, simulators of the physical systems have been coupled with these innovative techniques, especially when social and psychological aspects have been considered. In order to ease the re-usability of these simulators, avoiding re-coding everything from scratch, we propose a hierarchical and modular agent-oriented framework to test new residential strategies in the energy context. If needed, the presented work enables the user to select the desired level of details of the agent-based framework to match the corresponding physical system without effort to test very different scenarios. Moreover, it allows adding on top of the physical data, behavioural aspects. To this end, the characteristics of the framework are first introduced and then different scenarios are described to demonstrate the flexibility of the proposed work: (i) a first stand-alone scenario with two hierarchy levels, (ii) a second co-simulation scenario with a photovoltaic panel simulator and (iii) a third stand-alone scenario with three hierarchy levels. Results demonstrate the flexibility and ease of use of the framework, allowing us to compare several scenarios and couple new simulators to build a more and more complex environment. The framework is in the early stages of its development. However, thanks to its properties in the future it could be extended to include new actors, such as industries, to get the full picture.

Introduction

The Smart Grid is a complex system that encompasses many fields of expertise. Consequently, co-simulation platforms are emerging as the tool for testing new strategies and future scenarios thanks to the ability to couple domain-specific simulators in a broader context (Czekster 2020).

In parallel, with the paradigm shift in the electrical system, distributed methods are gaining ground since they offer major scalability, security and stability for the grid (Le et al. 2020). For this reason, agent-oriented approaches are being adopted to implement and test distributed control methods. Only recently it has been started to take advantage of co-simulation together with Multi-agent systems (MASs). As an example, Le et al. (2020) proposed a co-simulation-based platform to test distributed algorithms with cyber-physical system integration.

Moreover, the energy transition involves complex dynamics that require a social-scientific perspective aside from technical knowledge. To include this aspect, Agent-based modelling (ABM) has started being used mainly in six areas (Hansen et al. 2019): (i) Electricity market, (ii) consumption dynamics and consumer behaviour, (iii) policy and planning, (iv) New technologies and innovation, (v) energy system, (vi) energy transitions. However, the majority of these studies do not take advantage of other pre-existing simulators and implement everything from scratch, such as in Mahmood et al. (2020).

Most of the ABMs in the smart grid field consider the residential user as a core element of the simulation. Indeed, simple residential users might adopt photovoltaic panels becoming prosumers (Schiera et al. 2019) or might join and create an Energy Community, participating in a Peer-to-Peer market (Sousa et al. 2019). Even as simple customers, they might be part of Demand Side Management (DSM) programs that use dynamic tariffs or might be part of more complex programs. Indeed, DSM covers a wide range of strategies that involve the demand side of an energy system, going from energy efficiency to demand response (DR) (Palensky and Dietrich 2011). The time horizon, as well as the user involvement, might vary. Thus, it is undeniable that even in a simple fashion the end-user should be analysed.

For more complex analysis, modelling the behaviour of a single user would contribute to a higher level of realism and a deeper understanding of the phenomena underneath. As the simulation scale in size, there might be the need to parallelise the algorithms used in the simulation. For these reasons, adopting agent-oriented programming could be the right solution.

For a better understanding of the proposed framework, we have to clarify first the differences among Agent-based modelling, Multi-Agent System and agent-oriented programming. Indeed, ABM “is a method which examines the interactions of entities rather than a particular thing or collection of things to be replicated [...] it falls, along with several related fields that also focus on interacting things, on a spectrum with fuzzy boundaries and confusing overlaps” (VanDam et al. 2012). ABM tries to replicate the real-world behaviour and it is able to discover emergent behaviour from a bottom-up point of view. Instead, in MAS, the agents are set up with the features needed to achieve a particular and desired emergent state, i.e. control it (VanDam et al. 2012). In a nutshell, the main difference between ABM and MAS can be summarised in modelling characteristics of the real world versus modelling characteristics needed to solve a particular problem, respectively. MAS “can sometimes look quite a lot like an agent-based model if the problem to be solved involves exploring the unpredictable behaviour of human-like agents” (VanDam et al. 2012). Agent-oriented programming is a programming paradigm centred on the concept of the agent, i.e. dynamic software objects with a certain level of control over their actions and states. It can be used for developing both MAS and ABM.

On these premises, we consider modelling the residential user following agent-oriented programming as our starting point. Indeed, the BRIDGE (European_Commission 2021), a European Commission initiative to obtain a structured insight of cross-cutting issues encountered in the demonstration projects, underlined the need for understanding and strategising collective actions of citizens. Unlocking collective residential flexibility would translate into a benefit for the grid. Although we do not have the possibility to interact with real users, we can simulate what-if scenarios to create and understand the consequences of certain DSM programs from a simulative point of view. In particular, we are interested both in the single user and the whole household. Starting from a model of the residential user, adding new modules, researchers might then test new strategies or add social and behavioural models to catch the behaviour of the user.

Indeed, there is a lack of “plug-and-simulate” frameworks with high flexibility, that enables an easy re-usability of previously developed modules and that can be used in co-simulation environments. Moreover, the multidisciplinary nature of the issue underlined by the BRIDGE requires the proposed framework to be simple enough to be used by users with low programming skills.

The present work tries to fill this gap by presenting a hierarchical and modular agent-oriented framework for the energy transition which can be used in stand-alone configuration or co-simulation environments. For this reason, the main characteristics of the framework include (i) being time-step based, (ii) giving the possibility to use real data or single-purpose simulators such as a Photovoltaic (PV) panel simulator, (iii) giving the possibility to select the desired level of interest, i.e. analysing only the hierarchical levels needed to study certain phenomena, (iv) testing new strategies in the energy context.

The resulting framework wants to give high flexibility in the simulation, allowing the user to select the level of details, add a high degree of realism for the physical component derived from domain-specific simulators or real data, re-usage of existing models and inclusion of behavioural aspects that influence the outputs of the simulation.

The remainder of this paper is divided into five sections. “Related work” section gives a brief overview of literature solutions, stressing the gap. “Methodology” section  describes the proposed framework introducing the agents that will be used in different scenarios in “Case studies and results” section, which demonstrates the flexibility of the framework by presenting our experimental results. In “Discussion” section, we highlight the benefits of the proposed framework. Then, “Conclusion” section  discusses concluding remarks.

Related work

Among the available tools to develop ABM models, we find Netlogo (Uri 1999) which is intuitive, easy-to-use and, thus, suitable for non-programmers. However, “the simplicity of the programming language can limit the complexity of models” (VanDam et al. 2012). Other well-known tools are Repast (Argonne_National_Laboratory 2021) and Anylogic (The_AnyLogic_Company 2022) which allow more control over simulation, but require more programming skills. For even more complex descriptions of the agent, it is possible to create agents directly using programming languages (VanDam et al. 2012), e.g. in Java. Using existing tools for ABM, different works centered on the residential user can be found in the literature. A brief overview is provided in the following paragraphs.

Vellei et al. (2020) estimate the wet appliances’ flexibility potential under time-of-use (ToU) tariffs in France. They model a synthetic population, including a stochastic occupant behaviour model for the use of the washing machines and dishwashers and an ABM for the shift under a ToU tariff system. The whole household is an agent that takes probabilistic decisions for the shifting of the appliance based on socio-economic status, the time and the type of day. The appliance is shifted either at the beginning or at the end of the following off-peak hours according to a certain probability. Interactions between agents are not considered.

Walzberg et al. (2019) combined ABM to simulate human behaviours and their changes, i.e. shift in time of part of the electricity load, and Life-Cycle Assessment to assess environmental impacts. They establish four kinds of household agents characterised by different attitude levels. The decision to adopt pro-environmental behaviours follows a comparison with the best population score. To generate the electricity load profiles of agents, they used the stochastic model from Paatero et al. (2006). Instead, for the adoption of new environmentally-friendly behaviour, they exploit the model from Byrka et al. (2016). The ABM is implemented in NetLogo.

Micolier et al. (2019) proposed an ABM to simulate occupant-building interaction from the Building Information Modelling. The ABM is independent of a specific physical model and the framework is flexible allowing its usage in several civil engineering domains. The interpersonal relationships among occupants which affects the final behaviour are very detailed and take into account the dominance, the solidarity and the familiarity. From an extensive literature review, they also concluded that ABM for residential buildings have been designed for a specific use and none of the reviewed papers presented a systematic approach to managing inputs data.

The pivotal role of the end consumers and the need to model accurately the domestic electricity consumption at different resolutions has been underlined by the authors in Mahmood et al. (2020). They proposed a multi-resolution agent-based modelling and simulation framework in AnyLogic for estimating the electricity consumption of a given number of households by aggregating the consumption of major electrical appliances in the household. The framework focuses on three hierarchical levels: (i) household agents, (ii) occupant agents and (iii) individual appliance agents. Indeed, as Mahmood et al. (2020) claims, complex systems are made up of interrelated subsystems which in turn are composed of other subsystems, where each hierarchical level works without knowing the internal dynamics of its underlying components. The authors plan to utilise the model for the implementation of demand-response strategies. However, this has not been implemented yet. Moreover, it is not claimed that their framework can be used in co-simulation.

As observed by Micolier et al. (2019) and from the above-mentioned works, the majority of the models in the literature represent interesting but custom models with no focus on flexibility, modularity, and re-usage of the model. Thus, most of the proposed works lack two related and fundamental properties which could be regarded as re-usability in a broader context and modularity, which enable unlocking future and more complex simulations. Certain works re-use modules developed by others or mention the concept of flexibility, however only to a limited extent. Others, e.g. Mahmood et al. (2020), plan to re-use the model in a broader context, but do not claim that the proposed framework can be used in co-simulation.

Moreover, by comparing these studies, we can see that the concept of ABM is indeed “fuzzy”. Thus, the idea that we propose in this paper is a neutral agent-oriented programming framework that based on the final purpose can integrate new modules to test new strategies, MAS solutions or as an ABM to catch the socio-psychological behaviour of the user.

To sum up, the lack of a flexible, plug-and-play and re-usable agent-based framework was felt. Consequently, the novelty of this work lies in a framework that eases the test of new residential strategies and fosters the comparison among these latter by incrementally adding new models that tackle different aspects of the agent or easily changing simulators and strategies in a plug-and-play fashion.

Methodology

In this section, we provide a description of the framework first highlighting briefly the generic characteristics of an agent and then introducing the features of specific agents and specific modules that will be used in “Case studies and results” section for testing purposes.

The agents are created thanks to AIOMAS (Scherfke 2014), a Python library for request-reply channels, remote procedure calls and an agent layer. The logic of the agents needs to be implemented, thus we standardised each agent with some basic characteristics. The basic features of each agent include the agent_id and the capability to take track of time. Moreover, if the agent is part of a subsystem under study, it registers itself, i.e. connects, to the corresponding upper-level agent. The agent might know some “service agents”, which provide new functionalities, es. databases as done in Truong et al. (2016). Depending on the type of agent, it will have different attributes, it will perform different actions during the time step and it will have different methods for event-based behaviours. Thus, using the principle of inheritance, each agent will be characterised by the proper behaviour.

Therefore, on the one hand, AIOMAS provided us with high flexibility to customise the agents for our needs. On the other hand, being written in Python—one of the most easy-to-learn and easy-to-use programming languages, very rich in libraries—it can be easily used by others to add new behaviours or test new strategies. Moreover, it can spawn sub-process to parallelize the application and, thus, scale in size.

In what we called the “stand-alone mode”, at the creation of the scenario, it is possible to determine the number of agents per level. First, the uppermost level of interest is created. Then, the level below is built, receiving as input the desired upper agent address. Thus, the lower-level agents register with the upper-level agent and so on (see Fig. 1). Other attributes of the agents can be set through different configuration files, while new functionalities can be included by adding new Rule modules, i.e. separate pieces of code to model one specific aspect of the agent which substantially changes the outcome of the simulation. The scenario proceeds in time when all agents finish their tasks.

For the “co-simulation mode”, Mosaik (Schütte et al. 2011), a flexible and modular smart-grid co-simulation framework, has been coupled to our agent-oriented framework which remains almost unaltered, as already done in De Vizia et al. (2022). A more generic example that couples AIOMAS with Mosaik can be found at Scherfke (2018), where an AIOMAS agent acts as a gateway between the Mosaik API and the multi-agent system proposed. Mosaik allows the user to easily integrate existing simulators, representing a plus for the end-user. Thus, when the simulation begins both the simulators involved and our agent-oriented framework are started.

Fig. 1

Diagram of the scenario initialisation of the hierarchical agent-oriented framework

The agents

In this section, we introduce (i) the User Agent, (ii) the Household Agent and (iii) the Energy Provider Agent, i.e. the agents used for the scenarios presented in “Case studies and results” section, which correspond to three different level of detail. However, the framework does not limit itself to those agents. Indeed, it is expandable and capable of integrating new solutions and novel agents.

The User Agent and the Household Agent are based on an step-based implementation of the work done by Bottaccioli et al. (2019), enhanced in this work by using agent-oriented programming.

The User Agent is properly characterised through attributes, such as gender, age and employment—i.e. full-time worker, part-time worker, unemployed, retired, housewife, student, kid and infant—which would influence its behaviour. The socio-demographic characteristics are received through a configuration file at the beginning of the simulation.

It has to faithfully reproduce the behaviour of a single person in the energetic field. To this end, a sequence of actions generated using a semi-Markov model where the states are the daily activities, retrieved from large sample surveys on daily activities performed by the residential population, is used as done in Bottaccioli et al. (2019). The transition probability matrices created from survey data—i.e. the Time Use Survey—are implemented as a Service-provider agent, which retrieves the state based on the type of agent, time and type (week or weekend) of the day.

If the simulation does not involve other upper levels, the sequence of activities could be transformed directly into an energy consumption pattern. A possible future evolution of this scenario could involve the simulation of the travel pattern of the user and the corresponding consumption of hypothetical electric vehicles.

Fig. 2

Diagram explaining the two possible steps in the stand-alone configuration for the household agent and the user agent

If the scenario includes Household agents, the agents belonging to the same family will be associated with the corresponding Household agent through the initial registration process. Again, the Household agent will be characterised by certain attributes, such as geographical coordinates and type of appliances. The resulting behaviour of the whole family can be translated into energy consumption through a separate module that simulates the usage of owned appliances.

Thus, the behaviour of the household can be generated and retrieved at each time step. With the addition of the module that simulates the appliances, the activities of the user agents, i.e. the members of the family, can be translated into a consumption value for that time step, i.e. the “real-time” mode in Fig. 2.

However, we included a second mode which provides the initial future intentions of the agent, i.e. the energy pattern consumption before any exposure to dynamic tariffs or the actuation of an optimal strategy, to then test new DSM programs. This mode, i.e. the “initial-intention” mode in Fig. 2, represents one fundamental difference with respect to Bottaccioli et al. (2019).

Indeed, in the “initial-intention” mode, first n time-steps in the future—n defined by the user of the framework—are simulated by repetitively calling the same function of the “real-time” mode. Then, the corresponding electricity pattern can be saved in memory, acting as a baseline. Starting from the baseline, the strategy or the new behaviour can be simulated, obtaining a new electricity consumption pattern. Basically, the “change” block in Fig.2 can import a Rule module to implement the desired behaviour. In this way, it is possible to obtain a change in the consumption for the next n steps, testing the desired strategy or the effects of a new DSM program. Using this saved consumption vector, the simulation proceeds in time, returning at each time step the corresponding consumption value.

Finally, the Energy Provider Agent (i) communicates standard or time of use tariffs to the right agent and (ii) aggregates all the consumption of its customers, i.e. the household agents, at each time step, allowing the user of the framework to study macroscopic effects.

Fig. 3

Diagram explaining the configuration for future scenario, e.g. centralised control

Although it is not the focus of the current paper, we want to point out that the generic “change” module shown in Fig. 2 can be implemented also into an upper layer, e.g. into the Energy Provider Agent, which might, for example, implement a centralised optimisation for the benefits of an entire energy community or assume the role of a community manager (see Fig. 3).

The appliance module and the rule modules

In this section, we introduce the “Appliance module” and some Rule Modules used for the scenarios presented in “Case studies and results” section. These modules can be used by the agents for the implementation of the scenarios. The modules are described in the following:

• (i) The “Appliance module” simulates the appliances owned by each household. It contains the load profiles of real appliances and it is the one developed in Bottaccioli et al. (2019). It provides a one-to-one correspondence between the activities and the electricity consumption since it is the main element of interest in most case studies. For the sake of completeness, the match between the activities and the corresponding appliances is reported in Table 1. However, more details of this module can be found in Bottaccioli et al. (2019).

  Table 1 Mapping between activities and appliances

• (ii) The “Behavioural” model aims at simulating in a simple manner the behavioural change of a household which is exposed to dynamic tariffs. The behaviour of real people is not deterministic, thus when exposed to a dynamic tariff, this model introduces a change in the energy consumption according to a probability p. If the use of an appliance is anticipated or postponed, the appliance will be turned on during the lowest tariff. Again, since the user is not an optimiser, the load will be shifted at a random time during the lowest tariff.

• (iii) The “Optimisation” module instead contains a simple Mixed-integer linear programming (MILP) formulation for the optimal schedule of appliances that can be shifted in time. Being in a separate module, this strategy can be easily changed with more complex ones. This formulation is dynamically configured based on the resources considered. The objective function of the problem is to minimise costs:

  $$\begin{aligned} \text {min}\sum _{t=1}^{T} \delta \cdot [Cfrom_{t} \, Pfrom_{t} + Cpv_{t} \, PV_{t}- Cto_{t} \, Pto_{t}], \end{aligned}$$

  where $Cfrom_{\hbox {t}}$ are day-ahead prices in [€/ kWh] and, if installed, $Cpv_{\hbox {t}}$ is the Levelised Cost of Energy related to the PV systems. In the presence of PV panels, the surplus can be sold to the grid at a price $Cto_{\hbox {t}}$. $Pfrom_{\hbox {t}}$, $Pto_{\hbox {t}}$ are the amount of power imported from the main grid and the amount of power exported to the main grid at time t, respectively. $PV_{\hbox {t}}$ represents the PV generation at time t.

The constraints of the problem are defined as

$$\begin{aligned} \sum _{f=low}^{up} x_{f}^{i}= use\_appliance_{i} {\text { }}\forall {i} \end{aligned}$$

where Low and Up are the bounds for the shift in time of each single appliance defined by the user according to its preference. If the user does not want to use the appliance i (${\mathrm{i.e. use}}\_{\mathrm{appliance}}_{\hbox {i}}=0$), it is not turned on. The binary variable $x_{f}^{i}$ selects the daily load profile of the appliance i. Moreover, the power balance is respected by using

$$\begin{aligned} NoShift_{t} + \sum _{i=1}^{2} \sum _{f=low}^{up} x_{f}^{i} Lshift_{i} = Pfrom_{t} + PV_{t} -Pto_{t} \end{aligned}$$

where $Lshift_{i}$ contains the possible allocation of the consumption vector of the appliance i that can be shifted in time, i.e. the washing machine and the dishwasher in our scenarios. Instead, $NoShift_{t}$ represents the consumption of the appliances that cannot be shifted in time.

Case studies and results

In this section, we demonstrate the properties of the framework through three scenarios. The aim of these results is to demonstrate the versatility of the framework for future case studies. We focused our attention on the capability of the framework to address different scenario requirements and on its flexibility. Scenarios include a few households and results are shown for one single household to prove that the strategy or behaviour implementation works properly. In particular:

• (i) Case study 1 provides an example of the working principle of the framework, i.e. select the desired level of interest and test a simple strategy;

• (ii) Case study 2 proves the capability of the agent-oriented framework to work in a co-simulation environment and the re-usage of different modules;

• (iii) Case study 3 proves the flexibility of the framework, which can also be used to implement implicit demand response strategies. In other terms, the simplified behavioural model presented here can be substituted with more complex modules that model the change of behaviour of the user.

Case study 1

Fig. 4

Configuration of the Case study 1. The generic “change” block of Fig. 2 has been replaced by the “optimisation” module in red allowing the optimal scheduling of the shiftable appliances

Case study 1 considers a simple Home Energy Management System (HEMS) that optimally schedules certain appliances that can be shifted in time, i.e. washing machine and dishwasher (if present), under a variable tariff. Thus, the HEMS efficiently manages the usage of appliances inside the single house.

For this reason, Case study 1 requires a stand-alone configuration involving only two levels, i.e. four Household agents, ten User agents and as additional modules the “Appliance” and the ”Optimisation” modules (the block in red under the Household agent in Fig. 4). The “Optimisation” module acts as the HEMS. The simulation time of the scenario has been set to one day with a time resolution of 10 min, i.e. 144 time steps in a day.

Since the HEMS needs to know in advance the desire of the user for turning on an appliance the day after, we need to simulate the initial intention of the household for the next 24 h, i.e. n equal to 144 time-steps in the future (Fig. 2, “Initial-intention”). This consumption pattern is the baseline against which the change can be observed. Instead, the “change” block (Fig. 2) is implemented, as already explained, by the “Optimisation” module. Thus, the energy consumption of the appliances that can be shifted in time is optimally allocated. In this scenario, for demonstration purposes, the Household agents allow shifting the appliance at any time during the day.

Fig. 5

Day-ahead prices used in the Case study 1

Fig. 6

Example of optimal scheduling for a single household in the Case study 1

The mechanism underlying the model that generates the consumption was validated in Bottaccioli et al. (2019), thus we will only show the behaviour, i.e. the baseline and the modified consumption, for a single household with a housewife, a full-time worker and their kid. For each person, the corresponding transition probability matrices where obtained from the Italian Time Use Survey provided by the Italian National Institute of Statistics (ISTAT 2022).

For this scenario, the HEMS will determine the best time slots for the appliances considering the day-ahead prices. The day-ahead prices used in this scenario are shown in Fig. 5. We picked the day-ahead prices of a random day in 2013 provided by the Italian Power Exchange—(NORTH) (GME). We then added system and network charges (ARERA 2018) and fees (excise tax and VAT (Masci)).

However, to evaluate the implication for the Energy Provider Agent in presence of many customers equipped with a HEMS, we performed a second simulation with 2000 Household agents and 4000 User agents (random composition of families). We considered the same day-ahead prices, but we decided to limit the allowed shifting of the appliance to (i) $\pm {3}$ h and (ii) $\pm {5}$ h from the initial desired starting time, as long as the shift is performed inside the same day the users wanted to the appliance.

Results for Case study 1

For the first scenario, Fig. 6 shows when the Household agent wanted to turn on the washing machine (light-blue), i.e. when the user initially desired to use the appliance, and when it has been shifted according to the optimisation together with the rest of the consumption pattern (turquoise). The washing machine was initially scheduled around 1 p.m. However, since the electricity was cheaper for a longer period in the morning, the washing machine had been turned on at 9 a.m. Thus, we do not only observe the optimal schedule according to the HEMS but also the “initial-intention” for every single household. This allows us to make deeper considerations when considering a limited time window for the shift.

For the second scenario, in Fig.7 it is possible to observe the effects from the point of view of the Energy Provider Agent when $\pm {3}$ h of shift from the desired starting time is allowed. The shiftable appliances with an initial start around noon are anticipated and scheduled in the best time slots, as for the household example visible in Fig. 6. Those appliances scheduled around 8 p.m. are postponed after the highest prices since they cannot be shifted many hours in advance.

Running again the simulation for the case in which $\pm {5}$ h of shift is accepted, we observe the results shown in Fig. 8. Since more time is allowed for the shift, a secondary peak has been created. Indeed, more appliances have been shifted in the best time slots. The appliances scheduled in the evening are anticipated since it is more convenient.

Fig. 7

Example of optimal scheduling for 2000 households in Case study 1 with $\pm {3}$ h of shift allowed

Fig. 8

Example of optimal scheduling for 2000 households in Case study 1 with $\pm {5}$ h of shift allowed

Thus, this case study allows us to understand the benefits for a single household but also the possible threats for the Energy Provider Agent if no power limit is introduced.

Case study 2

To prove the capabilities of our solution in working in a co-simulation environment where the agent-oriented framework interacts with external and even third-party simulators, we developed the Case study 2. It considers a HEMS for prosumers by extending the previous Case study 1 including the PV generation. For this purpose, we exploited an external PV panel simulator developed in our previous work (Bottaccioli et al. 2018) and the Mosaik co-simulation framework, and its APIs, to couple this PV simulator with our agents (see Fig. 9) (De Vizia et al. 2022; Scherfke 2018). As for the Case study 1, Case study 2 considers the household and the user levels, the “Appliance” and the “Optimisation” modules, thus it uses the same configuration of the Case study 1 for the agents. The presence of the “PV panel simulator” slightly changes the MILP formulation, as described in section  “The appliance module and the rule modules”. The PV production for a hypothetical cloudy day used in this scenario is illustrated in Fig. 10.

Moreover, we performed a second simulation for an entire week (time resolution of 10 min and same household selected for demonstration purposes in Case study 1) since certain patterns can be appreciated only by considering weekly habits. The 1st of January was Tuesday. Saturday and Sunday are clearly distinguishable since these are the 2 days with a lower consumption (and no appliances that can be shifted in time). The households installed a 3 kW PV system each, thus PV production changes depending on the day. $Cpv_t$ has been set to 0.13 €/kWh according to IRENA (2018) and $Cto_t$ to plausibly 0.1 €/kWh.

Fig. 9

Configuration of the Case study 2. The generic “change” block of Fig. 2 has been replaced by the “optimisation” module with PV in red allowing the optimal scheduling of the shiftable appliances

Fig. 10

PV production in the Case study 2

Results for Case study 2

Fig. 11

Example of optimal scheduling for a single household in the Case study 2

Fig. 12

Power balance after the household agent optimisation in the Case study 2

Figure 11 shows an example of optimal scheduling for a single household in the Case study 2. The washing machine was initially scheduled during the evening when there is low or no PV generation. Thanks to the HEMS, the use of the appliance is anticipated in the morning, during high PV production, since the use of its solar energy is free of charge for the prosumer. For this scenario, we also check power balance in Fig. 12, which is clearly respected. In yellow, it is possible to observe the PV production. A good amount of it, i.e. the portion in green, is sold to the main grid, while the rest is used for self-consumption by the prosumer. During dark hours, instead, the electricity demand is satisfied by the main grid, i.e. the area in orange.

The results for the second simulations, visible in Fig. 13, highlight the tendency to anticipate the use of the appliances during daylight hours to save up. It can be also noticed that during nights the consumption of the household is very low as expected for a residential consumption pattern. Specifically, we simulated the first week of January 2013 in Fig. 13.

Fig. 13

Example of optimal scheduling for a single household in the Case study 2 for an entire week

Case study 3

Case study 3 simulates a group of simple customers under Time of Use tariffs. Therefore, for Case study 3 a stand-alone configuration with all the three levels, i.e. Energy Provider Agent, Household Agent, User Agent is needed. As additional modules, the “Appliance” and “Behavioural” are used (see Fig. 14). In this scenario, the Energy Provider Agent exposes its customers to Time of Use tariffs. These have been taken taken from Carmichael et al. (2014): (i) High tariff, i.e. 67.2 pence/kWh (ii) Normal tariff, i.e. 11.76 pence/kWh and (iii) Low tariff, i.e. 3.99 pence/kWh.

Again, it is important to know the baseline, i.e. the hypothetical electricity consumption under a flat tariff, and the change in consumption under Time of Use tariffs modelled thanks to the “Initial-intention” mode. This time the “change” block is implemented through the “Behavioural” module (the block in red under the Household agent in Fig. 14). Thus, as already explained, the shift in time of the load is performed according to a probability p and the turning on of the appliance occurs at a random time during the Low tariff communicated by the Energy Provider Agent.

For this case study, we also performed a second simulation with 150 Household agents and 334 User agents (random composition of families) and a probability p of a change in the energy consumption equal to 0.5. Indeed, also in this case it is interesting to observe the effects from the Energy Provider Agent perspective. We simulated consumption for 1 week and we plot the average electricity consumption over the 24 h comparing the initial consumption of the households and the one obtained after the households are exposed to ToU tariffs.

Fig. 14

Configuration of the Case study 3. The generic “change” block of Fig. 2 has been replaced by the “behavioural” module in red allowing a simulation of the behaviour of the users under ToU tariffs

Fig. 15

Example of a change in consumption of a single household under differentiated tariffs in the Case study 3

Results for Case study 3

Fig. 16

Aggregated load trends: Baseline (e.g. Flat tariff) vs ToU tariff with behavioural change vs “greedy” behaviour in Case study 3

The result for the Case study 3 is plotted in Fig. 15. The curve in light blue displays the intentional behaviour of the user, while the turquoise curve represents the behavioural change of the user after the communication of the tariffs as explained in “The appliance module and the rule modules” section. Thus, under a flat tariff, the user would have used the dishwasher during the evening. However, since under the ToU tariff the moment of the turning on corresponds to a high tariff period, the user decided to anticipate from 9 p.m. to 10 a.m. the use of the dishwasher when it is more convenient for it during a low-tariff period.

It is clear that since the appliances are randomly allocated inside the low tariff period, we do not observe secondary peaks. Instead, if all loads are moved just before the beginning of the High Tariff, we would create new peaks (blue curve in Fig. 16). Results are visible in Fig. 16 where the x-axis represents the daily time slots and the y-axis the mean aggregated value of household consumption, normalised using the peak value. An in-between scenario with a two-rate ToU tariff was implemented by Vellei et al. (2020), where the shift of the loads was not exactly at the beginning or at the end of the off-peak hours, but based on an exponential distribution.

Therefore, it is important to be able to forecast the consequences of a DSM strategy considering different scenarios with different technologies involved, e.g. random behaviour of people or coordination of HEMSs to smooth the peak as opposed to individual greedy HEMS or similar technologies.

However, independently of the behaviour, the proposed framework would be able to include the corresponding module which is the focus of this work.

Discussion

These simplified case studies demonstrate that starting from the architecture described in “Methodology” section with a little effort it is possible to study a good amount of scenarios and be able to compare them. In the presented simulations, we modelled and tested simple but nearly realistic scenarios that put the end customer at the centre of the energy transition. W.r.t. to the works proposed in the literature, we demonstrated the flexibility and re-usability of the framework by testing both optimisation strategies and simplified implicit DR focusing on different levels of aggregation with low effort thanks to the modularity of the developed code. Moreover, as opposed to Mahmood et al. (2020), we proved the framework to be capable of working in co-simulation, allowing us to add new simulators in the future, e.g. wind turbines. We considered the examples provided as a prerequisite and as a starting point with respect to the more complete definitions of ABM (and eventually MAS). In the Case study 3, the Behavioural module could be replaced with a more complex one that also includes interactions as done in Walzberg et al. (2019). However, new scenarios with complex interactions among agents would have required a validation part for all the proposed scenarios that is neither the scope of the proposed work nor feasible in a single paper. In any case, proof of the capability of including complex interactions and socio-psychological aspects can be found in De Vizia et al. (2022), where we used Small World Network, Theory of Planned Behaviour and Relative Agreement to model the adoption of a new DSM strategy. In future, these theories might be included as new modules integrated at the household level.

Conclusion

In this work, we presented a hierarchical agent-oriented framework to ease the re-usability of previously developed modules to readily test new strategies and allow the comparison of different optimisation and behaviours.

Starting from a model of the residential actors and adding new modules, the proposed framework might be used to test new strategies or to catch the behaviour of the user. Moreover, it can be used in co-simulation environments.

In this manner, it is possible to maintain the high precision of ad-hoc physical simulators without implementing everything from scratch.

To demonstrate the working principle, the flexibility and the modularity of the framework, we tested very simple strategies by reusing previously developed models and selecting the desired level of detail.

We started by observing the economic benefits of a HEMS inside a single household in Case study 1. However, when we introduced the Energy Provider Agent, we understood that at the aggregated level problems may arise if the household agents allow major shifts and HEMS are not properly coordinated. We then added PV panels in Case study 2, testing the framework in a co-simulation environment. With low effort, we moved from testing optimal strategies to the simulation of simplified behaviours of the agents in Case study 3. A random shift inside the low tariff period has been shown for a single household. Again, we underlined the importance of understanding the behaviour of the users. Indeed, depending on how people respond to the different tariffs, we could have significant advantages or major grid problems due to a higher peak.

In view of cooperation with researchers with different backgrounds or companies with more data, the framework can be used to understand the consequences of what-if scenarios based on the technological adoption or the behavioural changes observed or predicted.

The framework is in the early stages of its development. Indeed, existing agents need to be improved with more complex behavioural aspects and new actors should be introduced, such as industries.

In future works, we plan to extend the scenarios presented in this paper with new additional modules to realistically co-simulate future smart grids fostering the energy transition in a plug-and-play fashion. Thus, we plan to study more complex and advanced case studies. However, we believe it is important to first introduce the basic concept of our hierarchical and modular agent-oriented framework.