The analysis of electrical load signatures is an enabling technology for many applications, such as ambient assisted living or energy-saving recommendations. Through the digitalization of electricity metering infrastructure, meter reading intervals are gradually becoming more frequent than the traditional once-per-year reporting. In fact, across smart meter generations, samples were initially reported in 15-min intervals, more recently once per second, and even newer devices capture readings at rates on the order of several kilohertz. The advantages of using such high sampling rates have, however, not been unambiguously demonstrated in literature. We thus choose a widely considered application scenario of energy data analytics, event detection, and assess the impact of the sampling rate choice on the correct event recognition rate. More specifically, we compare the accuracy of two event detection algorithms with respect to the resolution of their input data. The results of our analysis hint at a non-linear relation between accuracy and data resolution, yet also indicate that most event occurrences can be correctly determined when using a sampling rate of approximately 1 kHz, with only minimal improvements achievable through higher rates.
Introduction
In recent years, a global trend towards the roll-out of smart metering infrastructure can be observed. A widely researched application domain for the data collected through smart meters is Non-Intrusive Appliance Load Monitoring (NIALM) (Hart 1992). It follows the objective of determining the operative appliances and their modes of operation from a household’s aggregate power consumption. An important component in NIALM algorithms is the recognition of state or mode of operation changes, also referred to as event detection in literature (Chang et al. 2011; Bijker et al. 2009). Event detection approaches range from the detection of step changes of an appliance’s power intake to more sophisticated solutions able to detect minor changes or gradual state transitions. One fundamental aspect of event detection has seen virtually no scientific consideration yet. The temporal resolution at which data is provided to an event detection algorithm can differ widely. Using the lowest possible data resolution is, however, an important objective to minimize the demand for storage space and computation time. We thus investigate in this work what impact the temporal resolution of the input data has on the achievable event detection accuracy, as well as on the algorithm’s execution times.
First, we survey existing work on event recognition methods and data sets suitable to accomplish this objective in “Related work”. Subsequently, we formulate a definition for the notion of “events” in load signatures in “Detecting events in load signatures” and discuss necessary data preprocessing steps to find events in smart meter data in “Data preparation”. Next, we conduct an in-depth analysis of the accuracy levels and execution times of two state-of-the-art event detection algorithms in “Event detection accuracy analysis”. Finally, we summarize the insights gained during our investigation in “Conclusions”.
Related work
Electrical load signatures have been demonstrated to contain a great information content. This bears promising potential for the application of signal processing algorithms to extract relevant high-level (i.e., abstract) features from the possibly large volume of consumption data. One prominent example for load signature analysis is NIALM, first introduced by George Hart in (Hart 1985; 1992). A range of approaches to detect activities and identify the causing appliances have been presented in literature, e.g., in (Zeifman and Roth 2011), and numerous companies have added disaggregation products to their portfolio in recent years. The process of inferring appliance activity through NIALM is composed of three major steps: Data acquisition, feature extraction, and load identification (Zoha et al. 2012). All of which have been extensively investigated in research, resulting in a large set of proposed algorithms, methods, and features, e.g., in (Jin et al. 2011; De Baets et al. 2017; Leeb et al. 1995; Bergés et al. 2011; Kahl et al. 2017). Event detection is commonly a part of the feature extraction step and used to detect changes in appliance operation from the data.
Event detection algorithms can be categorized by their analysis of steady state or transient information. Algorithms relying on steady state information, such as power consumption readings during the periods before and after a state transition, are well-suited to detect events of appliances with a constant power consumption in each of their modes of operation. The second option is to operate on transient signatures, i.e., the power consumption changes that can be observed during an event. They allow for the characterization of a device and its mode of operation by the unique shape of its power consumption during state changes (Zoha et al. 2012; Anderson et al. 2012).
A secondary aspect by which event detection methods can be categorized and which can be combined with either steady or transient information usage, is their analytic methodology. Algorithms can be predominantly attributed to one of the following classes (Anderson et al. 2012):
Expert Heuristics, which determine events by means of pre-defined expert rules. The first event detection approaches for NIALM defined such rules by setting a threshold for event recognition (Hart 1992; 1985).
Probabilistic Models are based on statistical methods to detect sudden power consumption changes. Different approaches have been proposed, e.g., the use of the General Likelihood Ratio (GLR) (Luo et al. 2002), its adaptation presented in (Bergés et al. 2011), or a chi square (χ2) Goodness-of-Fit Test which was introduced by Jin et al. in (Jin et al. 2011) and has since been widely adopted (Yang et al. 2015; De Baets et al. 2017).
Matched Filters are used to detect the characteristic features of a known appliance’s power signal in another appliance’s data (Leeb et al. 1995). While this leads to good detection rates for previously observed events, matched filters cannot correlate sections with unknown or non-recurrent characteristics.
A number of energy consumption data sets have been presented for research in NIALM. Many of these data sets (such as REDD (Kolter and Johnson 2011), tracebase (Reinhardt et al. 2012), or SMART* (Barker et al. 2012)), however, exclusively contain electric power consumption data or voltage and current readings. The absence of ground truth annotations (i.e., metadata on what user actions took place during the data collection, and how they are reflected in the data) complicates their use in conjunction with event detection. Fortunately, a few trace collections provide metadata along with the power consumption readings. Most notably, the BLUED data set (Anderson et al. 2012) overcomes this issue. BLUED is composed of one week of voltage and current measurements, collected at 12 kHz, in a family home in Pittsburgh, Pennsylvania. It furthermore features annotation files, timestamped at millisecond resolution, that contain information on events happening inside the monitored dwelling. An excerpt of BLUED data is shown in Fig. 1, side-by-side with the annotations of appliance activity.
Detecting events in load signatures
Events are defined in literature as “state changes of devices” (Anderson et al. 2012). Most commonly these changes constitute transitions between the device’s on and off modes. Some appliances also follow an internal state machine (e.g., washing machines that follow a pre-defined sequence of operational steps) that determines their mode of operation. This temporal sequence also results in events happening at all internal state changes (Anderson et al. 2012). Jin et al. formulate an event to be a change in the (electrical power) data stream, which occurs at the onset of the change (Jin et al. 2011). In this work, we slightly extend the aforementioned event definition:
An event is a change in a stream of sequential power consumption data, that is caused by an appliance changing its state. The event time is defined as the time of the onset of this change. Each event is followed by a transition. This transition has a duration, measured from the time of the onset of change until the time when the power consumption reaches a steady state again.
Fig. 1
Excerpt of the BLUED data set, showing the actual data (phase A, collected on 20 October 2011) and the corresponding appliance events from the provided annotations file
An example of the occurrence of two events in actual data is shown in Fig. 2. The diagram shows an appliance turning on first, and off again after approximately 3 s, with the corresponding transitions.
Event detection methods
In order to be usable in the context of NIALM, event detection methods must reliably recognize changes. Simultaneously the measurement noise and changes induced by transitions of preceding events should ideally be ignored. Event detection hence not only faces the challenge to correctly identify load changes as events, but also to correctly ignore load variations that do not result from events.
Fig. 2
Excerpt of a power consumption trace with event and transition annotations. Events are marked as solid lines, transitions are marked as grey boxes
The successful detection of events strongly relies on the temporal resolution of the input data. Low sampling rates induce much less fluctuations and in turn reduce the likeliness of erroneous event detections. However, at the same time the reporting rate might not be high enough to ensure that every event is detected at the correct time offset, and the switch continuity principle (Hart 1992; Makonin 2016) may thus no longer be assumed to hold true. As opposed to using very low sampling rates, the sampling of data at frequencies on the order of tens of kHz provides a greater data resolution for event detection, yet at the same time measurement noise is more likely to be falsely identified as events. We thus analyze the achievable accuracy of event detection mechanisms when providing them with input data of varying temporal resolutions. To avoid unintentional bias, we comparatively analyze a variation of the chi square (χ2) algorithm (Jin et al. 2011) as well as the threshold-based event detection (Hart 1985).
Event detection using the chi square method
The chi square algorithm relies on the comparison of two continuous sets of samples, a so-called “pre-event window” x and a “detection window” y, both of duration n. A goodness-of-fit level lGOF is computed between the two according to Eq. 1.
The decision whether an event has occurred is made by comparing the obtained value of lGOF to the χ2 distribution by checking whether \(l_{GOF} > \chi _{\alpha, n-1}^{2}\). The parameter α is the relevance level for which χ2 has been computed. If the condition holds (indicating the two samples do not originate from the same distribution), an event is detected at the last entry of the pre-event window, with a confidence of (1−α). To achieve a high level of confidence, we set α to 0.01. We have furthermore set n=1000, according to the recommendations for window sizes made in (Jin et al. 2011).
Event detection using threshold analysis
In early work on NIALM (Hart 1985), events are reported whenever the difference between two subsequent values exceeds a threshold value Pth. The temporal resolution of load signature data has a strong impact on the this method because higher sampling rates generally lead to smaller changes between successive samples. To encounter this on the high sampling rate BLUED provides, we calculate and compare the average of two subsequent analysis windows, each of length n. Each event detection is followed by a back-off period, during which any event detections caused by the event’s transition are ignored. As a result of our investigation of suitable parameter values, n=1000 has been selected because it yielded accurate event detection results. For the remaining parameters Pth and the back-off period between allowed event detections, we rely on information provided along with the BLUED data set (Anderson et al. 2012). In BLUED changes are considered as events if they introduce a change of more than 30W and have a duration of at least 3 s. Given that no duration is stated across which a 30W change must occur, however, we have selected our threshold and back-off time windows slightly more sensitive. The threshold was set to 15W, and the back-off time to 1 s.
Events in the BLUED data set
In the BLUED data set, a total number of 2355 events are present in more than 50GB of power consumption data. As the goal of this paper is a comparative evaluation of the impact of data resolution on event detection performance across a large parameter space, processing the complete data set would have required excessive computational resources. Instead, we have confined our analysis to a subset of BLUED data, namely the first snippet (20 October 2011, starting at 11:58 and ending at 22:47) of data and a weekend snippet (23 October 2011 starting at 14:09 and ending at 00:37 on the following day). Devices without events logged in this period were thus implicitly excluded. Albeit some of the excluded devices have significant power consumption, the chosen subset can still be considered representative, considering that devices with similar consumption are included. We have extracted events from both phases (A and B) of the BLUED data set, and used 169 events occurring on phase A and 288 events on phase B in our analysis. Phase A mostly contains events caused by the refrigerator and different lights, as well as household appliances. It has a very low noise level. In contrast, events on phase B mostly originate from entertainment devices (TV, monitors, A/V receiver, computer) and some higher consuming household appliances, e.g., a microwave oven. More random noise is present, as well as periodic spikes without any corresponding annotations.
Data preparation
In the following parts we describe the chosen approach for lowering data resolution, before elaborating on its impact on the parameters we supply to the event detection methods.
Lowering data resolution
As both considered event detection algorithms require macroscopic (Zeifman and Roth 2011) power data to work on, we have computed the Root Mean Squared (RMS) values of the input data. To this end, we multiply voltage and current signals first, and then shift a sliding window of the length of one period of the mains voltage (n=200 at full resolution) across the results. The root mean square value of the window is subsequently computed and reported. By shifting the sliding window sample-by-sample across the full length of the power data, RMS power values are available at the same temporal resolution as the input data, i.e., 12kHz. To compute representations of the data set with reduced resolutions, we slide non-overlapping aggregation windows of lengths 4, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 32 across the data and compute the arithmetic mean values of their contained RMS values.
Parameter adaptation
The reduction in temporal resolution effectively changes the temporal distance between two samples. As all considered algorithms rely on windows of fixed lengths, a reduction of the input data requires the window sizes to be scaled proportionally. We hence reduce the size of the detection window of the chi square method and the threshold-based approach by the same factor as the power data, such that the windows capture the same duration (rather than the same number of samples).
Event detection accuracy analysis
In this section we will describe the setup of our event detection evaluation, followed by the presentation and discussion of accomplished results.
Metrics for event detection
We analyze traces of all considered temporal resolutions in conjunction with both event detection algorithms described in “Event detection methods”. To compare their event detection results, we determine the number of true/false positives/negatives and consider their combination in the F1 score as the metric for our evaluations. This choice is further supported by the fact that true negative events do not impact the F1 score. This is an important property as for electrical load signatures, the high sampling rate and the sparseness of event occurrences leads to a very low ratio between events and samples without events (often below 1:1,000,000).
For the calculation of the metric, the typical definition of a true positive needed to be adapted. Many algorithms detect events throughout the whole duration of the transition period. We thus add a one second window around each annotated event, and only one correct event detection within this period will be considered a true positive. In all presented evaluations, we visualize the results of the event detection by adding an event timeline to the power consumption graphs, as shown in Fig. 3. This visual aid was found to be of great help in visualizing the effects of the reduction on the event detection that the F1 score cannot fully express. It should furthermore be noted that the achieved F1 scores are not used to measure an absolute goodness of the method. Instead they are compared in relation to each other to enable a comparison of the effects of the lower input data resolutions.
Fig. 3
Excerpts of results obtained from executing the chi square algorithm on data of reduced resolution, showing the detected events on both phases
First we evaluate the chi square method’s reaction to reduced data resolution. To this end, we plot the detected events alongside the actual power consumption. The results are shown in Fig. 3a for phase A and Fig. 3b for phase B. Less events are correctly recognized on phase B, which can be attributed to the greater presence of noise on this phase. Supplementally, Table 1 shows how the resolution reduction steps affect the F1 score of the chi square event detection.
Table 1 Values of the F1 score for the chi square algorithm
The F1 scores on phase A consistently range below the highest value achieved for data of full resolution. However, the event detection accuracy only experiences minor fluctuations unless the data resolution is reduced by a factor of 14 or greater. Best F1 scores are observed at full resolution and reduction factors of 4, 13, and 14 (i.e., effective sampling rates of 3kHz, 923Hz, and 857Hz). Further reductions show a higher detection of small power changes until the detection deteriorates such that events are not recognizable anymore at factor 32 (as visible in Fig. 3a). The F1 score on phase B is generally lower, yet slightly increasing F1 scores are observed for data with reduced resolution. This is consistent with the higher number of false positives observed; the F1 score varies consistently between 0.072 and 0.087 until reducing the data by a factor of 14 (at this point the score drops to 0.017). For phase B, the best event detection performance was achieved on data reduced by factor between 9 and 11 (i.e., effective sampling rates between 1.091kHz and 1.33kHz).
To further visualize the event recognition rates, Fig. 4 shows the distribution of time differences from detected events to the closest ground truth event. On phase A (shown in Fig. 4a), a positive trend can be seen for the first three plotted reduction factors. While the median remains constant across all reduction factors, the upper quantile shrinks until a factor 10 and then only slowly varies. For reduction factors between 8 and 12 the shrinking does not result in more outliers, whereas the factors 14 and 16 show a high number of outliers. These findings are consistent with the falling F1 score starting at factors of 15, which had shown a higher number of false positives far away from real events.
Fig. 4
Time difference between detected events and the temporally closest ground truth event for the chi square method
The outliers show a similar trend on phase B (shown in Fig. 4b), though their increase is more strongly correlated with the reduction factor. The quantiles and mean on phase B are also rising. Three distinct steps for factors up to 10, between 10 and 14, and at 16 can be seen, with each step rising in mean, quantiles, and number of outliers. Again the boxplot illustrates the deterioration after a reduction by a factor of 14, on the one hand through a general rise of the time differences between real and detected events, and on the other hand through the higher number of outliers.
Results for the threshold analysis
Overall the threshold analysis results exhibit a similar trend as observed for the chi square method across all different resolution reduction steps. The achieved F1 scores are tabulated in Table 2. Again, the score on the full resolution is higher on phase A than on phase B, with the highest F1 score of 0.636 and following scores around 0.475. This observation can be attributed to the higher precision of the full resolution data. On phase B, the algorithm picked up more false positives, resulting in a lowered precision and the F1 score reaching only 0.07. For thresholding, the F1 score is highest for the input data that has not undergone a resolution reduction. A slight falling trend can be observed, caused by registering more false positives.
Table 2 Values of the F1 score for the threshold analysis
A stark change can be seen at the highest reduction factor. At factor 32 on phase B, the algorithm suddenly detected nearly 600% more events than before, on phase A about 300% more. This rise was caused by the variations being aggregated over the extended time of one sample, such that some samples passed the detection threshold. The effect can be observed in Fig. 5 from timestamp 18:40:00 onwards.
Fig. 5
Events detected by the threshold aon input data reduced by a factor of 32
In a supplementary study, we have investigated the occurrences of false positives. Upon closer inspection of events and their transition times, it has become apparent that on phases A and B events are often already detected before the ground truth event is taking place. The impact of these occurrences could be mitigated by ignoring all events detected a short time before an actual (as per the ground truth) event in the evaluation. On phase B an additional cause for false positives can be attributed to the high amount of noise. Part of this noise appears in the form of small outbursts or spikes. This can also be seen in Fig. 3b, which moreover shows that further reduction of the input data leads to higher event detection rates at the spikes. For the threshold analysis, most of the false positive are the result of variations introduced by running appliances. Figure 5 displays these variations which are most prominent at the times after 18:32:00.
Execution time evaluation
Another important aspect when assessing the impact of reduced data resolutions is their effect on the execution times of the algorithms. The execution time evaluation was conducted on a 64-bit windows system with 8GB of RAM and an Intel i7-4790 processor unit. For this test, all algorithms were run on a trace of 10min length, which contains 2 events, including a 5min duration without any event occurrences. For all tests the overall execution time (which includes loading the data from a file) was measured. The results are shown in Table 3. The table confirms that the execution times of both algorithms scale inversely proportional with the reduction factor.
Table 3 Algorithm execution time (in seconds) on a trace of 10 min length
We have analyzed the event detection accuracy with regard to the resolution of the traces. Our evaluations have shown that only small differences of the F1 score can be observed when comparing event detection results on data at their native resolution (12kHz) to input data that have been reduced by a factor of approximately 10. On phase B, the use of data at lower resolutions even yielded slightly higher F1 scores, i.e., a better event detection accuracy. Even though results for the threshold analysis do not fully corroborate this trend, comparably constant F1 scores can be observed even when the data resolution is reduced. The observed potential of using lower-resolution data confirms the assumption that future research activities into this domain are needed.
Besides that, both compared event detection methods unambiguously hint at the existence of a changing point, at which the detection rate begins to drastically deteriorate. While this point lies at factors between 16 and 32 for the threshold-based approach, Table 1 in conjunction with Fig. 4 shows this changing point at a reduction factor of 14 (i.e., an effective sampling rate of 857Hz). This not only reveals a non-linear correlation between sampling rate and detection accuracy, but also poses a lower bound for practically usable data sampling rates for the given configuration.
Conclusions
For the realization of smart homes, effective NIALM methods are required. Event detection is an important aspect in the creation of such methods. So far it has been assumed that the information value in load signatures is positively correlated with the temporal resolution at which the data is collected. We have refuted this assumption for the use case of event detection, and shown that the chi square algorithm and a threshold analysis can well tolerate a lowered temporal resolution of the input data. The F1 score improvements of the chi square method after resolution reductions indicate that higher resolutions are more likely to introduce measurement noise and other erratic fluctuations, which have increased chances of being falsely detected as events. The in-depth analysis has shown that a lowered resolution by a factor of up to 13 had almost no perceptible effect on the event detection rate, pointing at sampling rates between 925Hz and 1.2kHz as suitable temporal resolutions for event detection. The F1 score only starts to be affected at a resolution of 857Hz and below, i.e., at a reduction of BLUED’s resolution by a factor of 14. These results indicate that the relation between data resolution and information content is non-linear. Further investigations are hence required to determine the optimum resolution choice that caters to both low data rates for their efficient transmissions while ensuring a high information content.
References
Anderson, K, Ocneanu A, Benitez D, Carlson D, Rowe A, Bergés M (2012) BLUED: A fully labeled public dataset for event-based non-intrusive load monitoring research In: Proceedings of the 2nd KDD Workshop on Data Mining Applications in Sustainability (SustKDD).. ACM, New York.
Anderson, KD, Bergés ME, Ocneanu A, Benitez D, Moura JM (2012) Event Detection for Non Intrusive Load Monitoring In: Proceedings of the 38th Annual Conference on IEEE Industrial Electronics Society (IECON).
Barker, S, Mishra A, Irwin D, Cecchet E, Shenoy P, Albrecht J (2012) SMART*: An Open Data Set and Tools for Enabling Research in Sustainable Homes In: Proceedings of the 2nd KDD Workshop on Data Mining Applications in Sustainability (SustKDD).. ACM, New York.
Bergés, ME, Goldman E, Matthews HS, Soibelman L, Anderson K (2011) User-Centered Nonintrusive Electricity Load Monitoring for Residential Buildings. J Comput Civ Eng 25(6):471–481.
Bijker, AJ, Xia X, Zhang J (2009) Active Power Residential Non-Intrusive Appliance Load Monitoring System In: Proceedings of the IEEE AFRICON Conference.. IEEE, Piscataway.
Chang, H-H, Chen K-L, Tsai Y-P, Lee W-J (2011) A New Measurement Method for Power Signatures of Non-Intrusive Demand Monitoring and Load Identification In: Proceedings of the 46th IEEE Industry Applications Society Annual Meeting (IAS).. IEEE, Piscataway.
De Baets, L, Ruyssinck J, Develder C, Dhaene T, Deschrijver D (2017) On the Bayesian Optimization and Robustness of Event Detection Methods in NILM. Energy Build 145:57–66.
Jin, Y, Tebekaemi E, Berges M, Soibelman L (2011) Robust Adaptive Event Detection in Non-Intrusive Load Monitoring for Energy Aware Smart Facilities In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).. IEEE, Piscataway.
Kahl, M, Ul Haq A, Kriechbaumer T, Jacobsen H. -A. (2017) A Comprehensive Feature Study for Appliance Recognition on High Frequency Energy Data In: Proceedings of the 8th International Conference on Future Energy Systems (e-Energy).. ACM, New York.
Kolter, Z, Johnson M (2011) REDD: A Public Data Set for Energy Disaggregation Research In: Proceedings of the SustKDD Workshop on Data Mining Applications in Sustainability (KDDSUST).. Elsevier, New York.
Luo, D, Norford L, Shaw S, Leeb SB, Danks R, Wichenko G (2002) Monitoring HVAC Equipment Electrical Loads from a Centralized Location - Methods and Field Test Results. ASHRAE Trans 108:841–860.
Makonin, S (2016) Investigating the Switch Continuity Principle Assumed in Non-Intrusive Load Monitoring (NILM) In: Proceedings of the 29th Annual IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).. IEEE, Piscataway.
Reinhardt, A, Baumann P, Burgstahler D, Hollick M, Chonov H, Werner M, Steinmetz R (2012) On the Accuracy of Appliance Identification based on Distributed Load Metering Data In: Proceedings of the 2nd IFIP Conference on Sustainable Internet and ICT for Sustainability (SustainIT).. IEEE, Piscataway.
Zoha, A, Gluhak A, Imran MA, Rajasegarar S (2012) Non-intrusive Load Monitoring Approaches for Disaggregated Energy Sensing: A Survey. MDPI, Basel 12(12).
This research was supported by Deutsche Forschungsgemeinschaft (DFG) grant no. RE 3857/2-1 and the Simulationswissenschaftliches Zentrum Clausthal-Göttingen (SWZ). Publication of this supplement was funded by Austrian Federal Ministry for Transport, Innovation and Technology.
The idea for this paper was conjointly conceived by both authors. The paper was written by JH (75%) and AR (25%). Both authors read and approved the final manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Competing interests
The authors declare that they have no competing interests.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Huchtkoetter, J., Reinhardt, A. A study on the impact of data sampling rates on load signature event detection.
Energy Inform2
(Suppl 1), 24 (2019). https://doi.org/10.1186/s42162-019-0096-9
