Agent-based bidirectional charging algorithms for battery electric vehicles in renewable energy communities

Abstract

This paper introduces a framework for agent based autonomous charging and discharging of Battery Electric Vehicle (BEV) at local energy communities. Agents are programmed to control the bidirectional charging according to green energy utilisation incentives, based on load and generation forecasts. The optimization is achieved within a group of independent prosumers following a fully distributed approach using multiple self-organising agents. No central instance is needed for communication, billing or decision making. To demonstrate the ecological benefits of the system, simulations for a car pool, with a Photovoltaic (PV) plant, in a residential neighborhood were performed. The simulations resulted in an increased community PV self-consumption value of 48% compared to 29% in case of uncontrolled charging processes.

Introduction

In the year 2010, as part of the 2050 carbon reduction goals, Germany pledged to reduce its greenhouse gas emissions by 40 % below 1990 levels by 2020 and by 80-85 percent by 2050 from 1990 levels. A feasible solution advised includes moving to 100 % renewable electricity by the year 2050 (Faulstich et al. 2011), with the binding target of at least realising 80 % (Bundestag 2017)[ §1(3)]. In addition to Germany, thanks to the global initiatives, many countries including India and China have already pledged additional ambitious goals (Mittal et al. 2016).

At the same time, the numbers of Electric Vehicle (EV) sales are steadily growing. The estimated worldwide sales of just passenger EVs is prospected to reach 56 million, with 90% being pure BEVs, by 2040 (McKerracher et al. 2019). This integration of an increased number of EVs is going to have a huge peak load stress on current electricity grids (Engel et al. 2018). Hildermeier et al. (2019) provides an up to date review on smart EV charging practises to solve those issues. However, the application of smart charging is still limited to a small number of pilot projects (DeBrey and Van Eijsden 2017; Dudek et al.Andersen et al. 2019).

To prevent overloading of electricity grids, another valuable solution proposed are self-sufficient communities. These communities produce and consume the energy within their geographical borders to prevent unnecessary energy transmission and distribution. In this context the European commission promoted the self-consumption of onsite produced renewable energy (Directive 2018) (§67-69) and encouraged member states to create opportunities for local citizens to participate in renewable energy communities (Directive 2018) (§70 and §71).

This paper promotes a self-organised multi-agent framework for autonomous charging and discharging of BEVs to maximize self-consumption within such energy communities. The system incorporates different ideas from previous research and applies them with specific modifications on the energy community concept. Diaz et al. (2018) implemented a Model Predictive Control (MPC) approach for smart charging using time variant prices and a central optimisation. With regard on the community aspect we follow a decentralised control approach but solve a similar optimisation problem. The decentralisation concept is inspired by Mohsenian-Rad et al. (2010), presenting a demand side management in a non-cooperative environment.

“System design & implementation” section gives the outline of the proposed system, giving information on the energy community setup, the pricing mechanism, the basic agent management and control framework and the algorithmic implementation of the optimisation. The algorithm is tested in a small micro grid simulation described in “Simulation setup” section. The performance of this simulation is evaluated in “Simulation results” section. Finally, in “Conclusion & outlook” section the major benefits and weaknesses of the agent based control are highlighted together with the prospected next project steps.

System design & implementation

An energy community typically consists of multiple independent residents, each trying to maximise their own benefit. Each resident can own multiple electrical devices, which build a cooperative team. To reflect such a cooperative environment on the individual tenant levels and competing interests among those teams, a decentralized approach is best suited. This way the tenant does not depend on a central player and the decision on whether the BEV shall be charged directly after arrival or with the generated PV remains an individual choice. Furthermore, with full decentralization on the communication layer, the multi agent approach is more resilient and new agents can easily be integrated in the running system. The 2-level system approach is visualised in Fig. 2b.

In the community we define three different available electrical energy sources:

• Privately owned PV within a team i.e. Internal Photovoltaic Energy (IPV)

• Public PV or private PV from other teams i.e. External Photovoltaic Energy (EPV)

• Electricity from the Grid i.e. Grid Energy (GE)

IPV is considered to be virtually free and gets the highest priority of consumption, with EPV coming in second, which has its own cost but is still cheaper than the price for GE.

System state and actions

An agents considers various user inputs such as, departure time, minimum State of Charge (SOC), which acts as an emergency buffer, and target SOC, representing the user expected SOC at the set departure time. Additionally, the user can choose from “Operation mode 1: maximum SOC”, “Operation mode 2: cost optimised” or “Operation mode 3: performance optimised” sections and a discharge enabled flag. The required user input can be given using a mobile app or a touch panel in proximity to the charging point. In case some or all information is missing for the current charging process, historical and default values are taken.

Apart from the user inputs, all agents communicate their energy forecast/plan for each timestep within the planning horizon. Furthermore, the state of the BEV and the Electric vehicle supply equipment (EVSE), containing the information about the current SOC and the current power flow, is communicated. Initially, the maximum battery capacity and the maximum charging and discharging power is communicated when ever an electric vehicle arrives at the charging point. In this context we assume that the EVSE and BEV can communicate the needed information using ISO15118 protocol.

Based on the given input information the algorithm communicates to the EVSE whether to charge, discharge or turn off the charging and how high the power flow shall be. The input state and action space is summarized in Fig. 2a.

Fair energy division

To guarantee a fair division of the available electricity, strict billing rules apply. The billing algorithm is publicly known and stored in a smart contract in an open distributed ledger. Every 15 minutes the contract gets executed and distributes the three energy sources in a fair way (see Fig. 1).

Fig. 1

Publicly known energy distribution and pricing mechanism

Fig. 2

Agent system design

In the first step production and consumption are matched internally. In step two, the production surplus of teams with an excess production (A & B) are summed up forming the EPV. The teams with a consumption surplus (C & D) get equal shares of the excess generation, while the remaining consumption is settled by grid energy (Step 3). Finally, according to the energy flows, grid and PV tokens are distributed (step 4). If after the first distribution of EPV (Step 3) a team is fully satisfied, the remaining excess energy is again distributed between all other non-satisfied teams. This loop continuous until all EPV is distributed or the demand of all teams is covered, in which case the remaining PV energy is fed into the grid. Using this billing strategy it is guaranteed that each team profits equally from the available PV plants.

Since the billing mechanism is publicly known, before going into the decision making, each agent first analyzes the communicated energy demand and production of the competing teams and the agents within its own team to evaluate its share of IPV, EPV and GE for each time step in the planning horizon. Due to its complexity, the algorithmic implementation is not described in here, instead the simple example as visualized in Fig 1 gives an idea of how the agents energy share calculation works.

Decision making

With the knowledge of the expected future energy shares the utilisation of electricity is first optimised at the team level following a cooperative approach, i.e agents decide to change their consumption to accomplish a cheaper overall electricity bill for the team. After the internal team optimisation, on the second level the teams compete over the remaining EPV by readjusting their consumption goals to target a global system optimum. The general control process of a controllable BEV agent is visualised in Fig. 2c.

The decision making algorithm consists of a Moving Horizon Control (MHC) and MPC based control. Depending on the selected charging modes and the current SOC the agent chooses the respective charging algorithm. Until the minimum SOC, full power is utilised to charge with priority. Subsequently the EV is charged according to the selected mode until the target SOC is reached. After the user target is satisfied the operation mode switches to a more economical mode, from SOC to performance and from performance to cost, to only charge with surplus PV energy. These operations are performed in conjunction to discharging, which is enabled separately by the user. The controller is designed to discharge energy only to its internal team members provided that its target SOC can still be met by future PV energy before the departure time.

Mathematical model BEV

The electrical model of the electric vehicle is determined as per equation.

$$ x^{t+1} \equiv x^{t}+ \frac{\Delta t \cdot P^{t} \cdot \eta}{E_{tot}} \quad \forall t \in T $$

(1)

with x^t being the battery SOC of the BEV at timestep t of the overall forecast horizon T. E_tot is the maximum capacity of the battery, P^t is the current power and η represents the charging efficiency.

$$ E^{t+1}_{dem} = E^{t}_{dem} - P^{t} \cdot \Delta t $$

(2)

where $E^{t}_{dem}$ represent the amount of energy still required to achieve the next SOC milestone (Minimum, target or full SOC). The BEV mathematical model is subjected to the hard constraints for power limits of the EVSE and the battery capacity as mentioned in (3):

$$ P^{min} \leq |P_{t}| \leq P^{max} \quad \land \quad 0 \leq x^{t} \leq 1 \quad \forall t \in T $$

(3)

Operation mode 1: maximum SOC

The SOC optimised mode can be imagined to be close to uncontrolled charging i.e. the EV charges with maximum power. This mode is used when the user does not want to participate in the program and when the EV battery SOC is below the minimum SOC

The objective function in SOC optimised mode is given by:

$$\begin{array}{@{}rcl@{}} \begin{aligned} &\hspace{50pt} \underset{\left\{x^{t}, P^{t}\right\}_{t \in T}}{\text{minimize}} \quad \sum_{t} -x^{t} \in T \\ &\text{subject to:} \quad x_{0} = x_{\text{init}} \quad (\text{\ref{2eq:evmodel}}), (\text{\ref{3eq:energychange}}), (\text{\ref{eq:constraints}}) \end{aligned} \end{array} $$

(4)

For the maximum SOC control (1) and (2) are repeated in a loop until the energy demand is zero or the target SOC is reached.

$$ \left(E^{t}_{dem} \neq 0 \lor x^{t} \neq X\right) \to P^{t} = P^{max} $$

(5)

where P^max is the maximum power limit of the EVSE and X is the next milestone SOC. The objective of max SOC is only fulfilled if there is no discharging and hence no discharging takes place in this mode.

Operation mode 2: cost optimised

Cost optimised mode uses IPV as much as possible and only moves to other sources if IPV is not sufficient. Since this mode differentiates between IPV and EPV, it is relevant for users with personal PV rooftop solar plants. The objective function in this mode is given by:

$$ \begin{aligned} &\hspace{30pt} \underset{\left\{x^{t}, P^{t}\right\}_{t \in T}} {\text{minimize}} \quad \sum_{t,Q} E^{t}_{Q} \cdot cost^{t}_{Q} \in T \\ &\text{subject to:} \quad x_{0} = x_{\text{init}} \quad (1), (2), (3) \end{aligned} $$

(6)

Q in (6) represents the forecast data of each electrical power source which in this case is given by (7). The cost factor in (6) represents the priority of energy source in terms of their cost.

$$ Q = {\{Internal\;{PV}, \; External\;{PV}, \; Grid\;Power}\} $$

(7)

The BEV agent is designed to only utilise the power according to the fixed cost optimal priority. Hence the controller first distributes the Internal PV throughout the control horizon according to the availability in corresponding time-steps. Equations 1 and (2) are repeated in a loop until $E^{t}_{dem}$ is fulfilled.

$$ \left(E^{t}_{dem} \neq 0 \lor x^{t} \neq X\right) \to P^{t} = P^{t}_{{IPV}} $$

(8)

If $E^{t}_{dem}$ is not fulfilled by just Internal PV, the controller distributes the External PV energy on top of the already distributed IPV to fulfil the remaining demand.

$$ \left(E^{t}_{dem} \neq 0 \lor x^{t} \neq X\right) \to P^{t} = P^{t}_{{IPV}} + P^{t}_{{EPV}} $$

(9)

If even after (9) the $E^{t}_{dem}$ is still not met, then the grid energy distribution is calculated using (10) and (11).

$$ E^{t, left}_{dem} = E^{t}_{dem} - E^{t, in}_{dem} - E^{t, ex}_{dem} $$

(10)

$E^{t, left}_{dem}$ is the total energy that the controller still needs to fulfil the total demand of energy from BEV. Since a BEV can be parked way longer than the control horizon, the controller delays the grid energy utilisation towards the end of BEV departure. Therefore it uses (11) to calculate the grid energy it needs to utilise in the current control step.

$$ E^{t}_{grid} = E^{t, left}_{dem} -\left(T^{depart} - T^{control}_{last}\right) \cdot P^{max} $$

(11)

where $T^{control}_{last}$ is the last timestamp of the control horizon. Grid energy, being an expensive but unlimited resource, is distributed from the other end of the control horizon. Eq. 12 is repeated in a reverse loop until the overall demand is met.

$$ \left(E^{t}_{dem} \neq 0 \lor x^{t} \neq X \lor |E^{t}_{grid}| \ne 0\right) \to P^{t} = P^{max} $$

(12)

Operation mode 3: performance optimised

In the performance optimised mode the agent tries to keep the battery SOC at maximum level at all times while keeping the cost to the minimum. Therefore it operates in between the Cost and SOC optimised modes. The objective function in this mode is the same as given in (6) for the Cost optimised mode but the controller achieves better performance by strategically modifying the input values. It uses the Total Photovoltaic Energy (TPV) instead of differentiating between IPV and EPV. Since equations and the decision making procedure for the performance optimised mode are almost the same as in the cost optimised version, they are not repeated in this section, but instead the modifications are mentioned in brief. Replacing Internal PV and External PV by Total PV in (7) the forecast data is modified. Eq. 8 is not needed anymore, instead in Eq. 9$P^{t}_{IPV}$ + $P^{t}_{EPV}$ is replaced by $P^{t}_{TPV}$. In Eq. 10$E^{t, in}_{dem}$ - $E^{t, ex}_{dem}$ are replaced by $E^{t, total}_{dem}$ while Eqs. 11 und 12 remain unchanged.

Simulation setup

To show the benefits of the previously described agent based control algorithms, a simulation is done for a multi family house with a PV plant and various BEVs. The simulation setup is shown in Table 1. The vehicle usage and uncontrolled charging profiles are generated using a BEV simulation tool described in Fischer et al. (2019). Electric profiles for different user groups on an aggregated apartment level are created using a stochastic bottom up simulation (Fischer et al. 2015). After the simulation, the household appliance profiles are re-scaled to have an overall (apartments + BEV) annual consumption of 20 MWh. PV electricity generation is obtained using a PV model based on Huld et al. (2010), King et al. (2004). PV modules are southwards oriented and optimally inclined as suggested in Calabrò (2009). We use measured weather and solar irradiation data for the city of Freiburg im Breigau (Germany). An in depth description of the decentralized bottom-up simulation framework is given in Surmann et al. (2019).

Table 1 Simulation setup

The PV plant is dimensioned to meet the combined annual energy demand of all apartments and BEVs (20 MWh). We chose different charging infrastructure and various types of BEVs together with different user groups to show a variety of factors involved in the flexibility potential. The applicable charging/ discharging minimum and maximum power limitations are listed in Table 1. The simulation is done with discharging enabled, the mode is set to cost optimal and the target SOC for all departure times is 80%, except for the Renault which always targets for 100% or unless a trip requires more energy, where the SOC target is also raised to 100%. The results are compared to a previously done uncontrolled simulation, in which the EV is always charged with full power upon arrival until 100%.

Simulation results

The simulation results for the previously described setup are presented in the following subsections. First, the overall performance of the controlled scenario is compared to the baseline. Afterwards, the different socio-economic factors, battery sizes and charging powers on the performance of the algorithm is analyzed. Finally, a section on the influence of discharging is provided.

Performance evaluation

The main target of the distributed control algorithms is to maximize PV self consumption of an overall energy community. Therefore this is the major performance indicator to analyze in the given simulation. Table 2 shows the self consumption shares for the non controllable loads, the EVs and the resulting overall performance. It can be observed, that for all teams with an electric vehicle the overall PV utilisation is significantly enhanced. For team 1 and team 5 the self consumption share was more than doubled. The lowest gain is noted for team 3. All efforts of the individual teams combined result in an increased community PV share from former 28.81% to 47.87% with the multi agent energy management system in place. This increased PV utilization can significantly lower the electricity bill of the community members and can encourage people to invest in additional PV plants.

Table 2 PV Self Consumption* in percentage (%)

Another benefit, that was not directly targeted for, but can still be observed is the annual peak load reduction of the community grid. The load smoothing effect can also be seen in the communities mean day plot Fig. 3f. This trend can further be enhanced by implementing grid limitations in the agents (see “Conclusion & outlook” section). It is a crucial factor for grid planning and also can influence the price of the community energy bill.

Fig. 3

Mean day PV utilisation for the different teams and the overall community

The influence of socio-economic status

As shown in Table 2 the agents performance highly depends on its environment. The largest impact can be traced back to the individual electricity consumption and EV usage pattern of the users. These patterns are highly individual but can also be clustered based on the socio-economic status of a user group. Though there could be multiple factors in socio-economics, this discussion is limited to the type of employment and driving patterns of the user. To elaborate the effect further, the mean day power curves of tenants 2 and 5 are presented in Fig. 3b and e. Both tenants have the same EV and charging infrastructure. The only difference is in the occupation with team 2 being a half time worker and team 5 consisting of two pensioners. This difference in employment status is expected to influence the general EV parking times. While a halftime worker has to leave for office at some point in time on workdays, a pensioner is assumed to stay at home more often. This means that the EV of the halftime worker is expected to be parked in office for a major part during the critical PV generation hours (7 a.m. to 8 p.m.) leading to some lost PV energy utilisation potential. It is witnessed that the morning peak is completely absent in the pensioner EV’s charging curve, in fact some discharging is observed during the early hours of the day (5 to 7 a.m.). This is because the car stays parked during the PV generation time and thus can decide to discharge during the morning time to eventually recover the energy from PV.

Impact of charger power limits

While running the simulation with different EV and apartment profiles, a strong correlation between the self-consumption ratio and the minimum power limit of the EV charger was observed. It is noticed that the self-consumption is inversely proportional to the minimum power limits of the charger. This can be explained by the fact that with a lower minimum limit the controller is able to cater to a larger number of discharge requests. To further investigate this correlation, loads from tenant 2 are simulated separately using 2 different chargers with all other parameters remaining constant. Table 3 compares the different energy usage of the two different chargers.

Table 3 Charging power settings and resulting energies for different chargers for EV2

It can be observed that by reducing the minimum power limits from 1.1 to 0.4 kW the discharged energy from the EV into the apartment has increased by a factor of more than 3. In addition, the PV self-consumption ratio of the tenant also increases by ≈ 11% from 40 to 51 percent. We conclude that a 0.4 KW charger, in this simulation setup, is able to better utilise the PV energy solely fuelled by more efficient discharge.

The minimum discharging power limits are only of relevance in cases where the energy demand of households is low. This is the case for the chosen simulation setup since discharging is only allowed into the loads of the corresponding team. If energy could be discharged into other teams as well, the power limitations are of less concern. Nevertheless, this large impact of the minimum discharging power limitations is of high interest for all single family houses with a PV - EV setup and a home energy management in place. In future such combinations shall be more common with increased EV shares and PV energy being a compatible cheap energy resource. Chargers that offer bidirectional charging are expected to be of high interest. Additionally, depending on the economical incentives for a high individual self consumption, chargers that can efficiently operate low powers further enhance the management possibilities for home charging infrastructure.

Impact of discharging

As explained in “The influence of socio-economic status” and “Impact of charger power limits” sections, the output numbers in Table 4 follow the presented argument. Highest discharging is observed for a halftime worker with the EV charger capable of catering to charge/ discharge demand from 0.4 KW i.e EV2. On the other end, smallest discharge numbers are obtained for EV3 which belongs to a fulltime worker and has the smallest battery size.

Table 4 Energy balances using discharging

To quantify the gains from an enabled discharging, another simulation was executed with exactly the same setup but discharging disabled. The self-consumption share of the community was lowered from 48% to 40%. Compared to the baseline (29%) a ≈11% gain is due to a shift in charging only and additional ≈8% are enabled with the bidirectional capability. Nevertheless, discharging comes with additional conversion losses. In the chosen setup the efficiency was set to be 93% for charging and discharging resulting in additional two-way energy losses of 14%. In case of EV2 the additional losses due to discharging accumulate to 135 kWh. From an economic perspective, this means that the saved money due to an increased PV consumption, should at least compensate for the lost energy. Without taking the qualitative battery degradation into consideration, the users still need to be incentivised with at least 15% monetary gains to make EV discharging economically feasible. From a broader energy system view, using already existing EVs as energy storages instead of installing new stationary batteries is beneficial. Nevertheless, before utilising discharging other means of demand side management should be exploited.

Conclusion & outlook

The multi agent energy management system was able to increase the PV self consumption share significantly while still meeting the energy demands of the car users. In a simulation of a small multi family house, the PV self consumption was increased by 19% to 48% compared to 29% in an uncontrolled state. 11% additional PV were harvested by only shifting charging processes while 8% can be traced back to bidirectional charging.

The executed work presents that battery electric vehicles offer a huge potential to shift electrical energy. However, the duration of BEV’s connection to the home charging station highly influence this potential. As a 9 to 5 full time worker, using the vehicle for the daily trip to work, even the most advanced PV optimisation is of limited use. It is expected that application of the developed multi agent framework in a mixed residential and commercial simulation should further enhance the overall PV integration and, therefore, should be followed up with further research.

An extension to the algorithm was developed to take into consideration the grid limitations following the approach in Mierau et al. (2017) but was not utilised for the current results. It is to be incorporated for the upcoming simulations. As each agent makes self-determined decisions, the communities can easily be scaled and new types of controllable devices like battery storage systems, heat pumps, μ-combined heat and power plants and other flexible electrical devices are to be added in future without any modification to the existing system. Further, incorporating SOC dependent charging power (saturation charging) in the agent’s decision making will be done in the next project steps. As a decentralised communication tool for agent to agent communication we used a bit torrent protocol called ZeroNet which, while performing good in the real time application, was found to be slow when used in simulations. We tried a more direct approach which did not speed up the communication, yet. This is still a matter of further investigation. Also, a test of different control algorithms (e.g. a central controller) against the same simulation setup shall further be done to quantify the system performance in relation to the presented algorithm. Finally, a real world implementation, test, monitoring and evaluation of the tenant electricity concept is scheduled to start in 2021 within the EnStadt:Pfaff project (Stadt Kaiserslautern 2020).