Learning capacity: predicting user decisions for vehicle-to-grid services

Abstract

The electric vehicles (EV) market is projected to continue its rapid growth, which will profoundly impact the demand on the electricity network requiring costly network reinforcements unless EV charging is properly managed. However, as well as importing electricity from the grid, EVs also have the potential to export electricity through vehicle-to-grid (V2G) technology, which can help balance supply and demand and stabilise the grid through participation in flexibility markets. Such a scenario requires a population of EVs to be pooled to provide a larger storage resource. Key to doing so effectively however is knowledge of the users, as they ultimately determine the availability of a vehicle. In this paper we introduce a machine learning model that aims to learn both a) the criteria influencing users when they decided whether to make their vehicle available and b) their reliability in following through on those decisions, with a view to more accurately predicting total available capacity from the pool of vehicles at a given time. Using a series of simplified simulations, we demonstrate that the learning model is able to adapt to both these factors, which allows the required capacity of a market event to be satisfied more reliably and using a smaller number of vehicles than would otherwise be the case. This in turn has the potential to support participation in larger and more numerous market events for the same user base and use of the technology for smaller groups of users such as individual communities.

Introduction

The global stock of electric cars surpassed 3 million in 2017 and is projected to grow to over 125 million by 2030 assuming the current policy environment (International Energy Agency 2018a). This growth promises to have a profound impact on our energy infrastructure requiring costly network reinforcements unless electric vehicle (EV) charging is properly managed. In the UK for example, it has been estimated that at least 30% of low voltage networks will requiring reinforcing at a cost of over £2bn (Cross and Hartshorn 2016). In parallel to this growth, the adoption of renewable energy has continued apace. In 2017, renewable energy provided 24% of global electricity demand with further strong growth expected (International Energy Agency 2018b). Much of this energy is supplied from inherently volatile sources that are weather dependent and this volatility places new challenges on balancing supply and demand in the power grid. The role of flexibility services, that enable modification of electricity production or consumption in response to a signal from the grid, is therefore increasingly important to ensure the stability of voltage, frequency and supply. This function has typically been provided by technologies such as pumped hydro storage systems and diesel generators, but the rapid response time and falling cost of batteries could see them playing an increasingly important role (Ofgem and BEIS 2017).

Whilst demand from EVs is a significant potential issue therefore, the opportunities they provide in terms of acting as battery stores when not being used for their primary purpose are also significant. Research has shown that in the UK for example, private vehicles are typically in use only 3–4% of the time (Bates and Leibling 2012), providing a huge opportunity for utilising EVs as energy storage. The technology to provide this capability is called Vehicle-to-Grid (V2G) and is under intense investigation in projects around the world (Reynolds et al. 2018). V2G allows energy to flow so the vehicle may be both charged and discharged depending upon grid demand and market opportunity (Kempton and Tomić 2005). This technology is a further development of smart chargers, which provide similar functionality but without the ability to discharge the vehicles. The need for such technology is apparent and its adoption is gaining momentum. in the UK for example, the government has legislated that all new electric vehicle chargers should be smart (Butcher and Edmonds 2018).

Whilst management of static storage assets is a relatively developed industry with many companies emerging to aggregate distributed loads, generation and storage to form Virtual Power Plants (Pudjianto et al. 2007), EVs present additional complexity due to their requirement to also deliver in their primary function – as a means of transport. Turning these disparate and moving assets into an aggregated energy store presents a challenge requiring a comprehensive software management system, which can manage power rates, delay charging, provide group power management and integrate with other signal requirements such as vehicle battery state of charge and network demand. Commercial organisations are beginning to emerge offering such capability attracted by the potential for significant return on investment (Nuvve Corporation n.d.). The flexibility markets providing this opportunity are complex, numerous and dynamic. However, several are appropriate for EV trading with smart charging or V2G including services to manage fluctuations in system frequency, reserve services to manage unforeseen demand increase or reduced generation and wholesale market trading in which energy is traded ahead of time (Payne 2019). Critical to the success of participation in these markets is prediction; given a set of mobile, geographically dispersed batteries it must be possible to guarantee contracted amounts of power or energy at a given time. This may be facilitated through Demand Side Management (Palensky and Dietrich 2011) in which end users are encouraged to change their demand profile to help meet the current requirements. For example, in the context of EVs, users may be incentivised by the aggregation service to shift their charging demand to off-peak periods where electricity price is lower and discharge during peak periods. Managing such an aggregation service is a complex problem and central to it are the vehicle users whose behaviour and decisions ultimately dictate the availability of a vehicle to participate.

Many factors will influence a user’s decision to make their vehicle available to participate in a V2G event, such as vehicle availability, reward opportunities and vehicle use (Gough et al. 2017; Geske and Schumann 2018; Will and Schuller 2016; Noel et al. 2018) and a successful service must therefore learn the key drivers of individual user decisions to help refine and optimise the operation of the overall system. This is particularly apparent for services that seek to recruit individual electric vehicle owners rather than fleets of vehicles, which are typically more predictable. However, while there has been much focus on the required technical infrastructure for V2G, sociotechnical considerations have been less will represented in the work to date (Sovacool et al. 2018). Of 50 worldwide projects recently reviewed (Reynolds et al. 2018), very few included a significant focus on user behaviour. One exception was the Parker project in which the customer had access to an app to indicate the required state of charge at each point of the day and individual user profiles were explored that allowed an aggregator to consider minimum energy thresholds while meeting target energy requirements at target times (Andersen et al. 2018).

This paper aims to help address this knowledge gap by focussing on the impact of user decisions on the viability of V2G services and the need for such a service to adapt to these decisions in order to maximise market opportunity. We present a model that utilises linear regression to learn the key factors driving a user’s decision. Each user is represented by a node that takes input from simulated data for vehicle locations and user-specified preferences and presents as output the probability of that user participating in a future market event. The model learns the motivating factors of each user driven by their response to previous offers to participate in, and follow through on, future energy trades. This knowledge allows the overall available capacity from the population of users to be predicted and thus supports market participation decisions made by the aggregation service.

The paper is organised as follows; the following section describes the design of the simulations that are used in this paper, the learning model used as the basis of the prediction engine is then introduced, the simulation results are then presented and discussed before the paper concludes.

V2G service simulation

For future V2G scenarios it is envisaged that aggregation services will bring together populations of electric vehicles to pool their available storage and provide a larger scale capacity that can be profitably utilised for flexibility services. In this work we consider such a service with the components and interfaces shown in Fig. 1.

Fig. 1

High-level interfaces for the proposed V2G aggregator service

The function of the key components of this service are highlighted below:

• User app; a smartphone app downloaded by each participating user with the following key features:

  • Location tracking using Global Navigation Satellite Systems (GNSS) capabilities of the phone and communication of this data to the vehicle location component.

  • Ability for the user to specify their preferences of when their vehicle is likely to be available to participate in a market opportunity.

  • Ability for the service to present an offer to users to participate in a market opportunity at a given time.

  • Ability for the user to accept or decline that offer and inform the service.

• Vehicle location data; the location of participating vehicles is monitored either through on-board vehicle telemetry and/or location data collected from the user app.

• Energy markets interface; identification of potential market opportunities that the service may wish to participate in and the execution of trades.

• Prediction engine; algorithms to predict available capacity and available vehicles by learning patterns of vehicle usage and user behaviour.

Simulation overview

The goal of the V2G service is to generate profit by providing flexibility services. This may involve provision of power to support ancillary services or energy for wholesale trading markets. The service would therefore maintain an interface with the energy markets to identify potential opportunities in which to participate and to execute any agreed trades. A common requirement for all these market opportunities is to reliably predict available capacity from the population of participating vehicles at a given time. In this work the focus is on the provision of an agreed amount of energy at an agreed future time as may be the case for trading on the wholesale energy markets for example. To align with the needs of this market, we consider the provision of energy within each hour period, which is a typical trading interval (Nord Pool Group 2019).

Input probability distributions

To help assess the available capacity for a future energy-trading market opportunity it is necessary to predict the likelihood of each vehicle being available to the service within a future hour period. We therefore define 2 input probability distributions with probabilities defined for each hour of each day of the week. These distributions are generated from 2 sources; firstly, the user’s own interpretation of vehicle availability specified in the service app and secondly actual vehicle location determined through vehicle telemetry or GNSS data from the user app.

User preferences

After signing up to the service, users download the service app to their smartphone. During initial configuration of the app it is envisaged that users will be asked to describe the likely availability of their vehicle. This input would reflect the users own understanding of when their vehicles are likely to be plugged into a suitable charging point and the routine constraints and preferences that may impact its use by the service. It is important to note that this method does not attempt to directly model user behaviour but rather model the process by which users would be asked to provide their own view of vehicle availability. Besides location, true availability may be influenced by a number of factors including the users’ immediate travel plans, the likelihood of unplanned journeys and typical available battery capacity. Users would be encouraged to reflect these factors in the preferences that they specify. In this way, the service capitalises on the user’s own knowledge to help bootstrap initial performance.

In this work we consider an app that allows the user to specify the likely availability of their vehicle for each hour of a typical weekday or weekend. This could be achieved in a number of ways. For example, a slider could be presented to allow users to specify a continuous probability for each hourly period. However, in this work we consider an app in which these likelihoods are specified as either low, medium or high as shown in Fig. 2(a). These are then translated to discrete probabilities 0.2, 0.65 and 0.85 respectively for use in the service as shown in Fig. 2(b). This has the advantages of being a simpler interface for a user and allowing clear demonstration of the behaviour of the learning algorithms in the simulations that follow. The learning approach adopted however, is applicable to both scenarios and other methods of capturing probabilities of vehicle availability from the user.

Fig. 2

Example probability distributions (b) reflecting likely vehicle availability from user-specified preferences (a)

Vehicle locations

The location of each participating vehicle is tracked by the service and location data made available either via the user app or via telemetry from a device installed in the vehicles. In a live scenario, it is envisaged that the service would access such data from vehicle manufacturers or fleet operators (Trakm8 Limited n.d.). Where smartphone data is utilised, it would be necessary to identify vehicle movement as opposed to the independent movement of users. This could be achieved in several ways. Some existing services such as Google Maps infer travel type (walking, car, train etc.) through GNSS-derived speed and on-board gyroscopic data (Reddy et al. 2010). This approach could be achieved without the need for additional hardware but is likely to suffer from accuracy issues and the inability to distinguish which vehicle is being used for a given journey. Alternatively, the user’s vehicle could be uniquely identified using a local wireless signal such as a Bluetooth Beacon placed within the vehicle as in (Naylor et al. 2019) or an existing Bluetooth system (radio etc.) within the vehicle. Any journeys made while this signal is detected by the user app could be assumed to be made with the user’s vehicle.

In this work, we are principally concerned with proximity of a vehicle to a V2G charge point and therefore stationary vehicle location must be cross-referenced with known locations of compatible charge points to allow a distribution to be generated giving the probability of the vehicle being near a suitable charging point for each period of the day. To demonstrate the creation of such a probability distribution, data was utilised from a fleet of 48 vehicles operated by the University of Nottingham and tracked using a commercial vehicle tracking service (Trakm8 Limited n.d.). Each time a vehicle made a journey, a record was received providing the time and GPS co-ordinates at that start and end of that journey. The vehicle was assumed to be stationary for any period between journeys. The haversine formula, shown in Eq. 1, was used to calculate the great-circle distance, d, between the location at the end of each journey, i.e. where the vehicle was stopped, and the locations of each of 6 defined charge points on the University campuses.

Equation 1 Haversine formula to calculate great-circle distance, d, of a parked vehicle from a charge point where lat1, lng1 and lat2, lng2 are the latitude and longitudes of the 2 locations and r is the radius of the earth (6371 km)

$$ d=2.r.\arcsin \left(\sqrt{\sin^2\left(\frac{lat_2-{lat}_1}{2}\right)+\cos \left({lat}_1\right).\cos \left({lat}_2\right).{\sin}^2\left(\frac{lng_2-{lng}_1}{2}\right)}\right) $$

If the shortest distance fell within 100 m, the vehicle was considered to be in proximity to a compatible charge point and therefore its battery could potentially be utilised for an energy-trading event. Each day was divided into hour periods and a vehicle considered to be available to the service if it was stationary and close to a charge point for the whole of that hour. A nightly web service was developed to analyse trip data for the previous day and update a cumulative moving average for the availability of each vehicle for each hour of the day. Figure 3(a) shows example probability distributions following the analysis of 34 weeks of data for a weekday for 2 sample vehicles within the fleet.

Fig. 3

(a) Probability distributions showing the likelihood of proximity to defined locations of V2G charge points for each hour during a typical weekday for 2 sample vehicles tracked using a commercial vehicle tracking service, (b) generalised probability distributions derived from these profiles that were used in the simulations

The figure reveals 2 distinct patterns of usage. Both vehicles are typically parked in the proximity of a charge point overnight. However, vehicle A has a much lower probability of availability during the day in contrast to vehicle B which retains a relatively high probability. Generalised and simplified distributions based on these profiles were generated as shown in Fig. 3(b) and were used in this work to facilitate clear demonstration of the behaviour of the learning algorithms. It should be noted however that the approach is equally valid for the continuous (and continually updated) distributions derived from real vehicle location data.

User simulation

Once a potential market opportunity has been identified and the prediction engine has suggested a set of vehicles to participate in that market event, offers must be made to the users to determine whether they wish to participate. In a live service, these offers would be delivered to the users via the app who are able to accept or decline. Their response provides a key learning input for the service validating the accuracy of the predicted response for this user. In this work we simulate this decision-making process by assuming that they are driven by a combination of user specified preferences and vehicle location data. In many cases the former may be a good predictor of actual user decisions however in cases where the usage pattern of the vehicle differs from the users originally defined estimates, location data may more strongly influence their decision. The probability of user n accepting an offer, P (A_n), and the result of that offer, A_n, is defined as in Eq. 2:

Equation 2 Probability of user n accepting an offer (a) and offer result (c) given a uniformly distributed random number (b)

$$ (a)\kern0.5em P\left({A}_n\right)=\frac{\left({W}_{pn}{P}_n+{W}_{ln}{L}_n\right)}{\left({W}_{pn}+{W}_{ln}\right)} $$

$$ (b)\kern0.5em R\sim U\left(\left[0,1\right]\right) $$

$$ (c)\kern0.5em \left(R\le P\left({A}_n\right)\to {A}_n=1\right)\bigwedge \left(R>P\left({A}_n\right)\to {A}_n=0\right) $$

Where P_n and L_n are the probabilities of vehicle availability at the time of the market opportunity as defined in the user preferences and vehicle location probability distributions respectively. W_pn and W_ln are configurable weightings specified as real numbers ranging from 0 to 1. The relative difference between the 2 weightings thus defines the contribution of each probability to the overall probability.

The user decision is thus simulated as follows; P (A_n) is calculated for the hour of the proposed market opportunity, a uniformly random number is generated (R) falling between 0 and 1, if R < = P (A_n) the offer is deemed to be accepted otherwise the offer is deemed to be rejected. This method allows changes in the driving factors behind a user’s decision to be simulated through modification of the weights W_pn and W_ln. The impact of such changes on predicted capacity and the corresponding response of the learning algorithms can then be assessed in the simulations. The key aim of these simulated decisions is to provide a training input to the learning models with known drivers so as to allow adaptation to this training stimulus to be demonstrated. In a live system, a training stimulus would be received in the same way but the drivers of that user decision would of course be much more multifaceted and nuanced.

Several other simplifying assumptions are made in this calculation. Firstly, it is assumed that the user responds immediately to the offer and therefore if refused another offer can be made immediately to other potential users. In a real system, offers would likely need to be made for a time window such that other users could be targeted if a response hadn’t been received within that window. Secondly, it is assumed that offers need to be accepted for every market event. In reality, it will likely be the case that users would be allowed to specify times that their vehicles can be used if available without explicitly accepting individual offers. Thirdly, a uniform offer is made to all users i.e. all users are offered the same financial incentive for providing the same capacity. It is likely that in a real service, different offers would be made to different users depending on a number of factors including battery size, available capacity and propensity to accept an offer.

Once an offer has been accepted there is no guarantee that the vehicle will actually be available to the service for a number of reasons. For example, the user may change their plans resulting in the vehicle being inaccessible, the vehicle may not have enough available capacity, or the user may disconnect during an event due to a pressing need. These eventualities are represented in the simulation through a single specifiable reliability parameter for each user, P (Dn), which is a real number ranging from 0 to 1. A value of 1 would thus represent a user who always follows through on an offer making all the offered capacity available and a value of 0 a user who never does so.

Equation 3 Actual capacity available from user n (c) given the probability of delivering the capacity (a) and the uniformly randomly generated number (b)

$$ (a)\kern0.5em \left\{\ P\left({D}_n\right)\in \mathbb{R}\ |\ 0\le P\left({D}_n\right)\le 1\ \right\} $$

$$ (b)\kern0.5em R\sim U\left(\left[0,1\right]\right) $$

$$ (c)\kern0.5em \left(R\le P\left({D}_n\right)\to {D}_n={C}_{veh}\right)\bigwedge \left(R>P\left({D}_n\right)\to {D}_n=0\right) $$

The final capacity contributed by user n (D_n) is thus simulated as in Eq. 3; for each user that has accepted an offer a uniformly random number (R) between 0 and 1 is generated, if R < = P (D_n) the offered capacity (C_veh) is deemed to have been made available otherwise no capacity is deemed to have been made available.

There are again several simplifying assumptions in this calculation. Firstly, it is assumed that the service knows immediately whether a user has followed through on an offer and therefore other users can immediately be targeted where this in not the case. In a real system this would not be known until the time of the market event and thus a larger buffer of users would likely need to be maintained. Secondly, a uniform reliability is assigned to the user independently of the specific day and time, which may not be the case in real-life scenarios. Thirdly, it is assumed that no capacity is available if a user does not follow through on an offer. However, there is a possibility that the vehicle may be disconnected after contributing some but not all the offered capacity. While these points are important to consider in a live service, they do not materially impact the operation of the learning algorithms that are the focus of this paper.

To alleviate the need to specify the probability distributions, decision making and reliability parameters for each simulated user individually, user types were defined to group users that share the same characteristics. In this paper, 2 such types (type A and type B) were used based on the profiles for the 2 sample fleet vehicles (vehicle A and vehicle B, respectively) shown in Fig. 3. For non-fleet users type A may correspond to day workers who have a charge point at home but tend to be away from compatible charge points during the day. Type B may correspond to those who have a more uniform pattern throughout the day and tend to remain in proximity of their home charge point such as home workers or retired people. The simplified default input probability distributions for both types were used in the simulations as shown in Fig. 3(b). Note that a single representative day was used in the simulations and therefore changes due to weekends for example are not considered. To demonstrate the impact of changes to the distributions, the “inverse” of a type A profile was also used for some simulations in which all low probability hours became high probability and vice versa. This had the result of there being a low probability of the vehicle being available overnight and a high probability during the day.

Prediction engine

The probability distributions defined in the previous section provide 2 key factors influencing the availability of a vehicle to participate in a V2G event. In some cases these distributions may be aligned, for example the user may indicate a high probability of availability in their app-specified preferences at a time when the historical location data indicates that there is also a high probability of that vehicle being in proximity to a compatible charge point. However, there are also likely to be situations in which the distributions deviate from each other. This may be due to a number of reasons, such as changing usage patterns since the vehicle availability was specified in the app or changes in parking location for example. An aim of this work is to develop a prediction engine that can learn the strength of influence of each of these factors for each user to better predict total available capacity from the pool of vehicles at the time of a proposed V2G event. This foundation can then be built upon to incorporate further influencing factors as the service matures.

Learning model architecture

A network of learning nodes was used as the basis of a prediction engine that aimed to learn and predict both the decisions made by users and the likelihood of them successfully delivering the agreed capacity at the agreed time. Each user was represented by a single node with inputs drawn from the user preferences and vehicle location probability distributions for the hour of a potential market trade. Each of these inputs was weighted to allow their relative importance to the user’s decision to be learned. The outputs of each “user” node were combined by a “capacity” node that predicted the cumulative available capacity given the predicted decisions of each user. Each input to the capacity node was also weighted, which allowed the reliability of each user to be learned i.e. a low weight would represent a user that rarely followed through on an offer and vice versa. The final output represented the predicted cumulative capacity available from the entire user base for the proposed market trade. The resulting architecture is shown in Fig. 4:

Fig. 4

Architecture of the prediction engine. Probabilities from an individual’s user preferences (yellow) and vehicle locations (blue) at the time of the potential market opportunity are input to a layer of nodes for each of the N users (U₁ to U_N). The capacity node predicts overall capacity (C_cum) given predicted outputs for each user weighted by the user’s reliability

The TensorFlow machine learning library (Abadi et al. 2015) was used to construct the network. Each user node performed a weighted sum of its inputs to produce its output as show in Eq. 4:

Equation 4 Output calculation for each user node, n

$$ {U}_n={P}_n{W}_{Pn}+{L}_n{W}_{Ln} $$

A high output thus corresponded to a prediction that the user was likely to participate in the market opportunity and vice versa. The capacity neuron also performed a weighted sum of its inputs and scaled each input by the offered vehicle capacity (C_veh), which was the same for all users, to yield the cumulative predicted capacity (C_cum) as shown in Eq. 5:

Equation 5 predicted capacity for user n (a) and cumulative capacity from the N users (b)

$$ (a)\kern0.5em {C}_n={U}_n{W}_{Un}\ {C}_{veh} $$

$$ (b)\kern0.5em {C}_{cum}=\sum \limits_{n=1}^N{C}_n $$

The predicted cumulative capacity aimed to inform the decision as to whether to participate in a given market opportunity through comparison to the required capacity. The list of users was also sorted in accordance with the output of the corresponding node. This list was then traversed in descending order to preferentially make offers to the users most likely to accept until the required capacity was reached.

Learning algorithm

User decisions and reliability were simulated as defined in Eq. 2 and Eq. 3 respectively, which provided two key outputs. Firstly, whether the user accepted the offer and secondly whether they followed through on the offer i.e. delivered the available capacity. These outputs could then be compared with the predicted outputs to provide the required learning signals for the model. Stochastic gradient descent (SGD) (Bottou 2010) with a learning rate of 0.05 was used to update the weights using the process described below:

Initialisation

Generate a network for n users and initialise the weights as follows:

• For each user n set W_Pn = 1 and W_Ln = 0. This seeded the model to emphasise user-specified preferences over vehicle location on the assumption that these are likely to be more representative initially given the limited time to profile vehicle movements.

• For each user n set W_Un = 1. This seeded the model to initially assume all users are 100% reliable and always deliver the offered capacity, taking an “innocent until proven guilty” approach.

Learning phase

• 1. Identify a potential market trade (through the interface to the energy markets) and look up the current user preferences (P_n) and vehicle locations (L_n) for each user n for that day and time.

• 2. Feed forward these inputs through the network to yield the predicted responses (U_n) for each user n and the predicted cumulative capacity, C_cum.

• 3. Traverse through the ordered list of users and determine the actual response of each user, A_n, to the simulated offer using Eq. 2

• 4. Calculate the squared error (A_n – U_n)² between the actual and predicted responses for each user n and update the weights W_Pn and W_Ln using SGD.

• 5. For each user n determine the actual available capacity, D_n, using Eq. 3

• 6. Calculate the squared error (D_n – C_n)² between the actual and predicted capacity from Eq. 5(a) for each user n and update the weights W_Un using SGD.

• 7. Repeat from step 1

In this way, the model predictions are continually refined based on what has been learned from previous market events. Updates to W_Pn and W_Ln allow the relative importance of these inputs to be learned for each user and updates to W_Un allow the reliability of each user to be learned.

Results

A number of simulations were performed using the following default configuration:

• 125 Type A and 125 Type B users with the probability distributions as shown in Fig. 3(b) for both user specified preferences (P_n) and vehicle locations (L_n).

• A fixed offer (C_veh) of 10 kWh for each user i.e. each participating user agreed to deliver 10 kWh of energy at the agreed time.

• A market trade requiring 1000 kWh at the agreed time.

• Learning model initialised as described above.

• Simulated user decisions configured to mirror the initialisation of the prediction engine i.e.:

  • User preferences (P_n) and vehicle locations (L_n) in Eq. 2 used the same default probability distributions shown in Fig. 3(b).

  • Decisions driven by user preferences i.e. W_pn = 1 and W_ln = 0

  • User reliability of 100% i.e. P (D_n) = 1

In the following sections we present results demonstrating the learning of user reliability and changes in the user’s decision-making criteria.

User reliability

Default configuration

Given the correspondence between the initialisation of the prediction engine and simulated user decisions, we would expect the output of the prediction engine to correlate closely with the actual results. To confirm this expectation, a series of 100 market events were simulated for each hour with no learning enabled. Figure 5(a) shows the predicted and actual capacity from the entire user base. Figure 5(b) shows the predicted and actual number of users required to satisfy the market opportunity and whether the trade was a success i.e. all required capacity was successfully delivered.

Fig. 5

Predicted vs actual capacity from all users (a), predicted vs actual users required for the trade and proportion of successful trades (b) averaged over 100 trades for each hour of a weekday. Dark column areas show contribution of type A users and light column areas type B users. Error bars show +/− 1 standard deviation

A close correlation between predicted and actual capacity and required users is evident from the figure. Given the surplus of capacity during the morning and evening all trades were successfully completed. However, during the day from hour 10 to 18 the predicted capacity was only just sufficient to meet the required capacity and thus a small proportion of trades failed even after making offers to all 250 users. The relative contribution of type A and type B users reflect the default probability distributions. During the morning and evening the probabilities were the same for both types however during the day type B probabilities were higher resulting in a greater proportion being used for successful trades and a greater contribution to overall capacity.

Impact of reduced reliability

To demonstrate the impact on prediction errors of changes in the user’s reliability, the above simulations were repeated with no learning enabled and the reliability parameter, P (D_n), for all type B users set to 0.5. Thus, there was a 50% chance of each of these users not delivering the offered capacity. The results are shown in Fig. 6:

Fig. 6

Simulation results (description as for Fig. 5) with type B user reliability, P (Dn), set to 0.5 showing prediction errors for type B (light column areas)

The impact of less reliable type B users is clear from the prediction errors that are now being made. Figure 6(a) shows that far more capacity is predicted for type B users than is ever delivered and therefore more users than predicted are required to satisfy the demand as show in Fig. 6(b). Although there remained just enough capacity predicted during the day from hour 10 to 18, every trade now failed during this period. These prediction errors could significantly impact the profitability and viability of the service.

Learning reliability

To demonstrate the ability of the learning algorithm to reduce the prediction errors a series of 10 simulations were performed consisting of 100 iterations of the same trade, which was arbitrarily chosen to be at 09:00. The first 20 iterations of each simulation were as for the default configuration, the next 20 iterations with user type B reliability at 50% and the final 60 iterations with learning enabled. The results are shown in Fig. 7, which reveals the impact of the change to user reliability followed by a gradual adaptation to this change resulting in a greatly reduced prediction error.

Fig. 7

Results averaged over 10 simulations of 100 iterations of the same trade. The area to the left of the dashed lines is using the default configuration, the area between the dashed lines is with user type B reliability P (D_n) at 0.5 and the area to the right of the dashed lines is with learning enabled. Dotted lines show + 1 standard deviation

As for previous simulations, 100 trades were simulated for each hour using the prediction engine following training. The results are shown in Fig. 8, which reveals that the prediction errors have been greatly reduced. Predicted and actual overall capacity are again closely aligned as are the predicted and actual users required to satisfy a trade. Although, all trades still fail during the day this is now successfully predicted and the possibility of potentially costly errors has therefore been reduced.

Fig. 8

Simulation results (description as for Fig. 5) after 60 trade iterations with learning enabled and user type B reliability, P (D_n) = 0.5

Decision criteria

In this section we demonstrate the impact of a change in a user’s decision-making criteria by using a different probability distribution for vehicle locations (L_n) for type A users. In reality, any changes to these distributions will often be relatively gradual, however, to demonstrate the impact and the response of the learning algorithms we consider a more abrupt change by “inversing” the distribution such that there was now a low probability of the vehicle being available overnight and a high probability during the day as previosuly discussed.

Overfitting

A series of 10 simulations were performed each consisting of 100 iterations of the same trade, which was arbitrarily chosen to be at 12:00. The first 20 iterations used the default configuration, the decision criteria for type A users was then set to be dominated by vehicle locations by setting W_Pn = 0 and W_Ln = 1 and after 40 iterations learning was enabled. The results are shown in Fig. 9:

Fig. 9

Learning a change in decision criteria for a trade at 12:00; iterations to the left of the dashed lines used the default configuration, the decision criteria for user type A was then changed to use vehicle locations with no learning enabled, which was enabled for iterations to the right of the dashed lines. Results are averaged over 10 simulations. Dotted lines show + 1 standard deviation

It can be seen from the figure that predicted and actual capacity are closely matched during the initial 20 iterations. The decision criteria for type A users is then switched to vehicle locations, which for a 12:00 trade has the effect of changing a low probability from the user preferences to a high probability from the vehicle location distribution. The actual capacity thus rises accordingly. With learning enabled, the prediction error gradually reduces until the predicted capacity is again closely matched to the actual capacity.

To visualise the impact of this learning on the full day. A series of 100 trades were simulated for each hour of the day and the results are shown in Fig. 10. Predicted vs actual capacity (a) and predicted vs actual users (b) are closely aligned for the period from hour 10 to 18. This reflects the fact that all hours during this period underwent the same change in vehicle location probability as for the 12:00 trade that was used to train the network. However, the predicted capacity for type A users was much higher than the actual capacity during other periods, which resulted in type A users being incorrectly preferred by the prediction engine and for type A users predicted capacity to be higher than the 10kWh maximum. The solution was therefore overfitted to the 12:00 trade; the output for type A was scaled up to account for the increased available capacity with no differentiation between the inputs from user preferences and vehicle locations. This is a reasonable solution given the limited breadth of the training data, however, a viable solution for the service must be generalised across the whole day.

Fig. 10

Results for 100 simulated trades for each hour following learning of changed decision criteria from 60 iterations of the same trade. Description as for Fig. 5

Generalised learning

To compensate for overfitting, the simulations were repeated using a randomly chosen hour of the day for each of the 100 iterations. This had the result of exposing the network to a broader range of trades during learning. As predicted capacities varied during the day, prediction error was used to show the learning progress as defined in Eq. 6.

Equation 6 Prediction error given a predicted and actual capacity

$$ {P}_{err}=100\ast \frac{\left|\ Predicted\ Capacity- Actual\ Capacity\ \right|}{Predicted\ Capacity} $$

As in the previous section, a series of 10 simulations were performed. The results in Fig. 11 show 20 iterations using the default configuration, followed by a change to use of vehicle location probabilities and the final 60 iterations with learning enabled. A gradual adaptation is evident from the figure resulting in an average prediction error of approximately 12% at the end of the simulation.

Fig. 11

Prediction error (P_err) using a random trade time for each iteration. Area to the left of the dashed lines with the default configuration, followed by a change in decision criteria that is gradually adapted to in the final 60 iterations to the right of the dashed lines. Dotted line shows + 1 standard deviation

Following learning, 100 trade iterations were again performed for each hour of the day. The results shown in Fig. 12 show that although the prediction errors are higher than previously, they are relatively consistent throughout the day. The general shape of the updated vehicle probability distribution has now been adapted to and the solution is now therefore much more generalised.

Fig. 12

Results from 100 simulated trades for each hour following generalised learning. Description as for Fig. 5

To validate the impact of the learning model on capacity and user predictions, these results were compared to those prior to learning being enabled at iteration 40 of the experiment. It can be seen from Fig. 13 that while the available capacity from the pool of vehicles was sufficient to meet the needs of the trade, the predicted capacity and the predicted number of users required to satisfy the trade were significantly different to the actual values. In the morning and evening periods, capacity was predicted to be much higher and the required number of users much lower than actual values. Conversely, from hours 10 to 18, capacity was predicted to be much lower and the number of required users much higher than actual values. These prediction errors are likely to have a deleterious impact on the operation of the aggregation service. For example, a trade during the morning or evening could result in fewer users than required receiving offers to participate and hence a failed trade. A trade during hours 10 to 18 may be refused due to low margins of available capacity and where a trade was pursued many more users would be involved, which may impact profitability and the possibility of making use of those users for other trades. Prior to learning, the mean error across the day was 745 kWh for capacity predictions and 67 for user predictions. This contrasted to 153 kWh and 5.5 users following learning; on order of magnitude improvement that would significantly benefit decisions made by the aggregation service.

Fig. 13

Results from 100 simulated trades at iteration 40 of the experiment i.e. prior to any learning. Description as for Fig. 5

Discussion

The simulation results presented above demonstrate the ability of the model introduced in this work to learn about the factors influencing a user’s decision to participate in a V2G event and their reliability in following through on that decision. The aim was to present a model that could form the basis of a live service while supporting extensions and enhancements to account for additional complexities as the service matures. User specified vehicle availability captured in the service app and historical vehicle location data provide key building blocks for such a live service, which have the potential to provide reasonable initial predictions of vehicle availability that can be beneficially utilised. For example, use of the continuous probability distributions derived from 34 weeks of historical data for the University of Nottingham fleet of vehicles described previously allowed the number of vehicles parked at defined charge point locations to be predicted for each hour period within the subsequent 4 weeks with an average error rate of only 5% measured as the percentage difference between predicted and actual available vehicles. Sustained deviations from this potential availability resulting from user or fleet manager preferences that may have been defined a priori in the app or from users not making a vehicle available to the service at these times despite a favourable location can then be learned by the model presented in this paper. This allows total vehicle availability for a future trading event to be more accurately predicted, which can be translated into actual available capacity by assuming a fixed amount of available energy from each vehicle for example. Such an assumption may be enforced in a number of ways, for example a fleet could introduce a policy of requiring vehicles to always retain that amount of charge plus an operating minimum or to only make the vehicle available if this condition holds. Total predicted available capacity would then inform the decision made by the service operator as to whether to participate in the trading event.

There are many ways in which such a service can be extended and enhanced to help improve predictions. One key method is through the use of battery data. As EVs are connected and disconnected from charge points, valuable data is generated that can be made use of by the service. In particular, the times of connection and disconnection can be used to help refine location estimates. This data is of high quality and has the potential to significantly enhance the estimates made for vehicles tracked using GNSS data from the user app as it provides a reliable indication of vehicle presence at a known fixed location, which can be used to correct misidentified vehicle movements. The increasing adoption of international standards (Observations et al. 2014) will also facilitate availability of other information including state of charge at the start and end of the charging event, battery status and condition. This data could be used as further input to the learning models to better predict the likelihood of a vehicle participating and the energy that could be contributed by considering likely state of charge at the time of a trading event.

The simulations in this work build profiles of user preferences and vehicle location for each hour during a week to facilitate the prediction of vehicle availability for the hour of a potential trading event. This method works well for users and vehicles with regular routines. However, in many cases there are likely to be deviations from typical routines that are predictable but are not a function of the specific day of the week. Examples include holidays, special events and seasonal effects. It may not always be the case that the presence and impact of such deviations are known when the model is constructed and thus an iterative process is likely to be required in which the model is enhanced in response to observed performance. This process would involve identifying the root cause of any prediction errors and updating the model and input data to accommodate it. Taking two examples from the University fleet; public holidays have the same profile as weekends and there are significantly fewer vehicles available from Friday evening to Monday morning when there is a University open weekend. Having identified the root cause, the occurrence of these events can be built into the training data set and inputs added to the model to facilitate learning and help reduce the observed prediction errors for future similar events.

A more complex, multifaceted set of inputs would necessitate a more complex learning model to predict user decisions. In this work linear regression with stochastic gradient descent was appropriate to learn about each user, as online learning was required and the task was linearly separable. However in a live service with additional inputs, the decision-making process is likely to be significantly non-linear and therefore each user would need to be represented by a more complex learning model to allow this non-linearity to be represented. The prediction engine was designed however to be extensible to these scenarios so that different learning models can be used within individual nodes while retaining the same overall architecture. The use of stochastic gradient descent is also transferable to other approaches such as the use of a neural network to represent each user.

There are likely to be many other factors influencing a user decision in addition to those considered in this work. These will include constraints such as available capacity at the start of the event window and required capacity at the end of the event window. They will also include softer constraints such as a user’s personal risk profile, a desire to ensure enough capacity is always available for unforeseen journey requirements or more general range anxiety. While interfaces with V2G charge point data sources could provide richer data sets for use by the service and the user app could help better profile users, other techniques will be of value to help identify the drivers of user decisions and reflect them in the predictions that are made. A key aspect of this, are the offers made to users via the app. For example, these offers could be differentiated based on offer price or incentive and differing emphasis around cost and carbon saving messages. This opens up the opportunity of utilising findings from recommender system research that has developed from modelling user preferences for e-commerce and advertising for example (Jung et al. 2005; Wei et al. 2007; Liu 2015).

Although this paper was principally concerned with predicting user decisions, these techniques could also be used to influence that behaviour through targeted offers to support participation in profitable trading opportunities for which the capacity may not otherwise have been available. Such offers may be made to users for which the service predicts a relatively low likelihood of participation so as to increase this likelihood and hence the cumulative predicted capacity. As an example, consider the situation in which a vehicle is typically parked at a charge point for an hour during a lunch break and a market opportunity exists for the following hour. A targeted offer could incentivise the user to change their routine and take a later lunch to allow participation. Such variability in user offers could also be employed more widely by targeting users taking into account the size of their vehicle battery and user profiling data for example. Similarly, a baseline of users could be established through an agreement that allows a user’s vehicle to be used when available during certain periods without a confirmatory acceptance from that user. Such offer optimisation adds a new layer to the learning model but has the potential to significantly benefit the performance of the service and allow a greater degree of confidence predicting available capacity given the inherent delays in receiving a user response to an offer.

Success of any learning system requires sufficient training data. In this work, stochastic gradient descent was used that updated the model following every new training signal i.e. a user decision to offer their vehicle for a given market trade and whether it was actually made available at the agreed time. In simulation, it was possible to generate as many market events as required to ensure that sufficient training data was available. This may not always be the case however in a live service, where the market opportunities may be more limited. In this work 60 market events were used during the training phase and more limited exposure would have limited the ability of the system to adapt to changing circumstances. The changes made in the simulation were however relatively abrupt and significant. This is representative of sudden changes in routine for participants for example. In some real-life scenarios however, changes may be more gradual and thus require less adaptation.

The simulation results also emphasised that the breadth of market event exposure was as important as the volume. Use of the same market event for learning resulted in a solution that was overfitted and thus inaccurate for different events. Exposure to a more diverse range of market events allowed a more general solution to be learned. This is an important point to consider in a real system. The choice of market events to participate in will likely need to be influenced not only be economics but also this need for diversity; it may be necessary to sacrifice some short-term profit for longer term gain.

Another important point to consider is to ensure that sufficient training data is acquired for each individual user. If users are not selected as candidates to participate in a market event, it is not possible to learn from their response to it. In this work an offer was made to all users for every market event thus ensuring continued learning. For any live service it will also be necessary to ensure enough offers are made to each user to ensure that they remain engaged with the service, are able to generate a return for their participation and that they provide the necessary training stimulus to the service so that overfitting to sets of users does not occur. There are a number of ways in which this could be achieved. For example, offers could be made to a random sampling of users who would otherwise not have been chosen or offers could be made to all users on the understanding that acceptance of an offer does not guarantee participation in that event. In this way, a training signal is still received based on their response to the offer.

In order to further enhance the profitability of the service it will also be important to consider multiple market events in relatively close proximity. For example, there may be an opportunity to participate in 2 events within the same or consecutive hour periods. In such a scenario the prediction engine must consider the previous activity of users such that the same users aren’t identified as candidates to participate in both events where this is not possible. Data captured from charge points would help ensure this is not the case and other techniques could also be used such as a refractory period following each participation. It is also the case that some market events may run for multiple hours and thus the same set of users may not be used for the whole period. Such a scenario would require a subset of users to be identified for each sub-period. Exploration of such complicating factors will be the subject of future work.

Conclusions

In this work we have considered a proposed future service that pools EV batteries to form a larger storage resource that can be used for trade in the energy markets. A learning model was introduced that, in simplified simulations, learned the criteria influencing a user when deciding whether to participate in a market event and their reliability in delivering any offered capacity at the time of that event. We demonstrated that by learning about users it was possible to better predict the available capacity at a given time and the number of users that are required to satisfy the required capacity. It is proposed that such predictive capability enhances the viability and profitability of such a service by helping to ensure market trades are only pursued if they can be satisfied by the user base and are not unnecessarily refused due to an incorrect assessment of available capacity. A more accurate prediction of required users also places less requirement on the maintenance of a “buffer” of users by the service and thus supports participation in larger and more numerous market trades. In addition, this predictive capability has the potential to better support deployment of the service in scenarios with an inherently restricted user base such as individual communities.