My name is Oliver Parson, and I'm currently employed as a Product Data Science Lead at Octopus. I'm interested in investigating the ways in which machine learning can be used to break down household energy consumption data into individual appliances, also known as Non-intrusive Appliance Load Monitoring (NILM) or energy disaggregation.
José and I have been working on a project to apply NILM methods to the health monitoring domain, specifically to help monitor the activity of elderly people living independently in their own homes. The approach we pursued aims to disaggregate the kettle reliably from smart meter data, without requiring any training in each home. We chose to use the kettle as the appliance of interest as it is an appliance common to almost all UK homes, is used regularly as part of most elderly people’s daily routine, and also has a signature which varies little between houses.
The core novelty of the our work is that deviations in kettle usage from the normal routine can be recognised from the disaggregated smart meter data, allowing interventions in households to occur as soon as possible. Crucially, such routines are learned individually for each household, rather than using a static routine for all households. As such, kettle usage is used as a proxy for health, since no additional sensors were installed to directly measure health parameters (e.g. heart rate).
An example of a routine for a single household is shown below. The top graph shows the probability that the kettle would be used at least once within a half hour interval, while the bottom graph shows the cumulative probability that the kettle would have been used at least once by that time of the day. It can be seen that for this household, a highly repeatable routine is followed, where the kettle is used by midday on about 80% of days.
Full details can be found in our paper to be presented at BuildSys later this year:
The Dataport database is the world's largest source of disaggregated customer energy data. The database contains electricity data collected from 722 houses in the US; 631 in Texas, 49 in Colorado and 42 in California. The houses monitored include 501 single-family homes, 183 apartments, 35 town homes and 3 mobile homes. Access to the portal is free for members of universities, while commercial access is limited to members of Pecan Street's Industry Advisory Council.
To date, the Dataport data has been available via direct access to the database. While this provides an efficient means of querying a small amount of data, large amounts of data can take a long amount of time to download since the data is transferred in an uncompressed format. Furthermore, as with most other data sets, the data set is described in a custom format, requiring researchers to parse the data and metadata before making use of the data set.
For these reasons, we've been working with Pecan Street Inc to release a subset of the Dataport database in NILMTK HDF5 format. The HDF5 file is 1.09 GB in size, and contains one month of data from 669 of the Dataport houses, which were selected as they contain at least 8 meters. In each house, the circuit name has been converted from the Dataport names to the NILM Metadata controlled vocabulary. The produced dataset can be easily analysed using tools described in the NILMTK documentation. The HDF5 file is available via the Dataport portal under the same access control as the Dataport database.
Below is a boxplot showing the proportion of energy consumed by each circuit category across all households in the HDF5 data.
Oliver Parson, Grant Fisher, April Hersey, Nipun Batra, Jack Kelly, Amarjeet Singh, William Knottenbelt, Alex Rogers. Dataport and NILMTK: A Building Data Set Designed for Non-intrusive Load Monitoring. In: 1st International Symposium on Signal Processing Applications in Smart Buildings at 3rd IEEE Global Conference on Signal & Information Processing, Orlando, FL, USA, 14-16 December, 2015.
Overfitting is a well-cited problem in the field of Non-intrusive Load Monitoring. Overfitting refers to the high accuracy of an algorithm on one small data set, while the same algorithm generalises poorly to other data sets. In NILM, this often corresponds to algorithms which work well on data covering a short period of time or a small number of houses, but performs poorly on data covering longer time periods or other houses.
One solution to this problem is through the use of a competition, in which the organiser releases part of the data set for training, while holding out the remaining data for testing. This approach works fine for one-off competitions where each participant can only submit one solution, but it weakens when participants are allowed to submit multiple entries. The reason for this is that participants can use information learned from their accuracy score on the test set to inform their algorithm choice.
Belkin organised exactly this type of competition for NILM via the Kaggle platform a few years ago. The competition allowed two entries per day per participant, each of were evaluated using half of a private hold out data set and displayed on a public leaderboard. However, the competition ended after 4 months, at which point the final standings were determined by each solution's performance on the other half of the private hold out data set which had never previously been released. Through such a format, the final standings can only be calculated once and the competition cannot be re-run, as the final standings convey information about the private hold out data set which could inform the design of future algorithms.
A recent Google Research blog post describes this exact problem in a much more general sense than NILM. Most interestingly, their recent paper even proposes a solution to this problem through the use of a reusable holdout set. The approach is that the reusable hold out set is only accessed through a differentially private algorithm, which effectively samples the holdout set in order to produce a different sample each time it is accessed.
At the 2015 European NILM workshop, an MSc group from Imperial supervised by Jack Kellypresented a platform which could potentially be used to host NILM competitions in the future. I’d be really interested to see whether such a platform could use such a reusable hold out set in order to allow the competition to run for much longer without compromising the results relative to the classical method of evaluation.
During July 2015, Imperial College London played host to nearly 70 attendees from all over the world for the European Non-Intrusive Load Monitoring (NILM) workshop, bringing together energy disaggregation researchers and professionals for this 2-day event.
The purpose of the workshop was to provide a forum for energy disaggregation enthusiasts to learn about recent developments in the field, as well as network and discuss projects for future collaboration. The workshop was attended by academics, employees of energy disaggregation companies, multinational utility companies and a few hobbyists.
Prof Mario Bergés, Assistant Professor at Carnegie Mellon University gave the keynote speech which focussed on the relevance of NILM within the emerging domain of the Internet of Things (IoT). Mario covered recent trends in energy disaggregation, as well as his projection of the field’s relevance into the future. His talk also proposed four ‘million dollar’ ideas which he believes will have significant impact on the domain of energy disaggregation. You can watch Mario’s full talk below.
Workshop attendees also enjoyed talks from both academic and industry aspects, with speakers including Mingjun Zhong from the University of Edinburgh and Stephen Makonin from Simon Fraser University representing academic findings, and focussed on models for energy disaggregation, socioeconomic concerns and accuracy evaluation. From an industry perspective the workshop welcomed Bruno Charbonnier from EDF R&D, and Hjalmar Nilsonne from Watty who cemented the importance and benefits of disaggregating electricity and announced the release of a new dataset.
Delegates were invited to bring a poster to present at a dedicated ‘lightning talk’ session, giving each presenter a chance to disseminate and discuss a NILM related topic of their choice for 1 minute. During the lunch and break sessions, posters were displayed on the walls, giving the presenter an opportunity to engage in one-to-one discussions with other attendees. The posters are available via a dropbox folder, while the lightning talk session can be watched below.
An MSc group from Imperial College London presented a tool for evaluating NILM algorithms without requiring the NILM algorithm’s code to be released. There was a real buzz of excitement around such an initiative, and a number of improvements were suggested around the need for a real-world private data set.
As the NILM Workshop came to a close, an agenda was decided for topics to discuss the following morning at an informal user group designed to encourage collaboration and potential projects among attendees, which included funding applications and data sharing. The afternoon session explored NILMTK; an open source toolkit for non-intrusive load monitoring and included an overview of the toolkit as well as discussions on how to encourage contributions from the community. The need for a collaborative knowledge base, where items such as public data sets can be described in an easily comparable and searchable format was also discussed, with the result being a web based wiki which will be available on the nilm.eu website soon.
The most obvious learning from the workshop was the increasing momentum in this domain. The 2014 European Workshop was attended by around 20 people while this year saw nearly 70 attendees from around the globe. In addition, the diversity of the problems being studied by each of the attendees was also clear from the poster session, as each start up or academic project has a subtle but significantly different take on the problem of energy disaggregation. Lastly, the problem of evaluation accuracy cropped up regularly throughout the workshop. The need for standard data sets, metrics and methodologies is now more important than ever.
The workshop was streamed live on YouTube, and videos of all talks can be seen via our YouTube playlist, while each presenter's slides can be downloaded from our dropbox folder.
The two day workshop finished with a discussion of plans for the 2016 European workshop. While the location of the workshop is yet to be decided, it was clear that there was sufficient demand for a future meeting. We will soon be announcing a call for hosts, with the aim of hosting the workshop in a city with convenient international transport links with the rest of Europe.
10.00 Welcome and workshop overview - Oliver Parson
10.30 Keynote talk - Mario Bergés
11.30 Poster lightning talks - All poster presenters
12.00 Lunch & poster session (not streamed)
13:40 Academic talks - Mingun Zhong & Stephen Makonin
14.30 NILM evaluation tool - Imperial MSc group
15.00 Coffee & networking (not streamed)
15.40 Industry talks - Bruno Charbonnier & Hjalmar Nilsonne
16.30 Roundup and discussion for future workshops
17.00 Stream ends
Please note all times are local London time (BST = GMT+1).
We might need to restart the Hangout if we encounter technical problems on the day, so please keep an eye on NILM_Workshop on twitter for links to new streams.
Videos of the talks should be available on YouTube shortly after the event so long as everything goes smoothly on the day.
We have also set up a LinkedIn group to allow people to introduce themselves, continue workshop discussions, or catch up on anything they've missed.
I've noticed a fair amount of disagreement regarding exactly what type of learning is being used by a specific energy disaggregation method. I think the confusion arises from a discrepancy between the definition of supervised learning in the general machine learning literature and the practical assumptions of energy disaggregation methods:
Machine learning definition
General purpose machine learning defines supervised learning methods as those which require labelled training data to train a model. Labelled data refers to both the input and answers to the problem, which in the case of energy disaggregation corresponds to both household aggregate and individual appliance energy consumption. Conversely, unsupervised learning refers the use of only unlabelled data (household-level) data to construct models.
Practical energy disaggregation
In the energy disaggregation field, a fundamental problem exists due to the variation in appliances between different houses. As a result, scalable methods must not require appliance-level data from the houses in which disaggregation is to be performed (test houses). As such, practical approaches can apply supervised learning to appliance-level data from houses other than the test house, but can only apply unsupervised learning to aggregate-level data from the test house.
Semi-supervised learning
General purpose machine learning defines semi-supervised learning as the combination of a small amount of labelled training data with a large amount of unlabelled training data. Although this sounds similar to the scenario described above, the crucial difference is that energy disaggregation requires that the supervised and unsupervised learning takes place on data from difference domains (buildings), while general purpose machine learning assumes both the labelled and unlabelled training data are drawn from the same domain. Furthermore, energy disaggregation training methods could even make use of a large amount of labelled training data from non-test houses, and only a small amount of unlabelled training data from the test house.
Summary
I've been apprehensive to use the term semi-supervised learning to describe practical energy disaggregation methods due to the domain-specific requirements of the field. Instead, I generally refer to methods as unsupervised if they use appliance-level data from only non-test houses, which often leads to confusion. I'd be interested to hear other people's opinions on the matter, and hopefully we can reach some consensus!
