What even is supervised/unsupervised disaggregation?

I've noticed a fair amount of disagreement regarding exactly what type of learning is being used by a specific energy disaggregation method. I think the confusion arises from a discrepancy between the definition of supervised learning in the general machine learning literature and the practical assumptions of energy disaggregation methods:

Machine learning definition

General purpose machine learning defines supervised learning methods as those which require labelled training data to train a model. Labelled data refers to both the input and answers to the problem, which in the case of energy disaggregation corresponds to both household aggregate and individual appliance energy consumption. Conversely, unsupervised learning refers the use of only unlabelled data (household-level) data to construct models.

Practical energy disaggregation

In the energy disaggregation field, a fundamental problem exists due to the variation in appliances between different houses. As a result, scalable methods must not require appliance-level data from the houses in which disaggregation is to be performed (test houses). As such, practical approaches can apply supervised learning to appliance-level data from houses other than the test house, but can only apply unsupervised learning to aggregate-level data from the test house.

Semi-supervised learning

General purpose machine learning defines semi-supervised learning as the combination of a small amount of labelled training data with a large amount of unlabelled training data. Although this sounds similar to the scenario described above, the crucial difference is that energy disaggregation requires that the supervised and unsupervised learning takes place on data from difference domains (buildings), while general purpose machine learning assumes both the labelled and unlabelled training data are drawn from the same domain. Furthermore, energy disaggregation training methods could even make use of a large amount of labelled training data from non-test houses, and only a small amount of unlabelled training data from the test house.

Summary

I've been apprehensive to use the term semi-supervised learning to describe practical energy disaggregation methods due to the domain-specific requirements of the field. Instead, I generally refer to methods as unsupervised if they use appliance-level data from only non-test houses, which often leads to confusion. I'd be interested to hear other people's opinions on the matter, and hopefully we can reach some consensus!

Announcing the 2015 European NILM workshop

I'm really excited to announce that the Second European Workshop on Non-intrusive Load Monitoring will be held on 8th July 2015 at Imperial College London. The workshop is the follow up to last year's NILM @ London workshop, which provided the first European venue which brought together both academics and companies with an interest in energy disaggregation. Updates and registration information can be found at the new website: www.nilm.eu

Some important information:

• When: 8th July 2015
• Where: Imperial College London, UK
• Cost: Free
• Objective: To provide a European venue for disaggregation researchers to discuss recent developments in the field and fuel future collaborations

This workshop will have a more technical focus than the first workshop, and will feature a keynote from Mario Bergés in the morning and a technical session of invited talks in the afternoon. Furthermore, we're inviting all attendees to bring a poster on a topic of their choice, which could be a recent piece of work, their company's current direction, or even an invitation for collaboration on a joint project. Last, we're hoping to live stream the event online for anyone who can't be there in person, though this is a little experimental!

EEme release disaggregation accuracy calculated by Pecan Street

I recently came across EEme, a spin-out from Carnegie Mellon University, who apply energy disaggregation to 15-minute smart meter data to provide demand side management analytics. EEme are particularly interesting, since they’re the first company (to the best of my knowledge) to go completely public regarding the accuracy of their product, as calculated by a third party. EEme used Pecan Street’s 3rd party evaluation tool, which provided EEme with 15-minute aggregate smart meter data and weather data from hundreds of homes, and required EEme to return monthly totals for four of the largest energy consuming appliances. Since Pecan Street also measured the appliance-level energy consumption as part of their deployment, they are able to calculate the exact level of accuracy of EEme’s disaggregation. The full 7 page report is available for request from EEme’s website. Although EEme focus on lower resolution data than many other energy disaggregation companies, I’m interested to see whether this report sparks the beginning of a more public competition between energy disaggregation companies to release accuracy statistics as confirmed by a third party.

Overview of the NILM field

This post aims to use trends in recent publications to provide an overview of the field of non-intrusive load monitoring / energy disaggregation.

Recent Growth

Researchers often refer to a recent explosion in the number of NILM publications. The graph below shows the number of papers published per year, from which the upward trend since 2010 is clearly visible. This renewed interest is likely due to recent country-wide roll outs of smart meters.

Significant Publications

In such a rapidly growing field, it's often hard to understand which publications are the most significant. The graph below shows the number of citations of the most cited papers. Unsurprisingly, Hart's 1992 seminal paper is the most cited by far, with a number of other papers from the 90s also appearing high up the list.

Since older papers have had more time to accumulate citations, it's also interesting to look at citations per year to get a better idea of recent trends in the field, as shown by the graph below. Unlike before, there is no stand-out paper, with recent review papers and data set papers receiving the greatest citation velocity. Besides these papers, a number of the remaining highly cited papers propose techniques based upon principled machine learning models.

Publishers

I am often asked which are the most popular venues for NILM research. The graph below shows the most popular publishers, with ieeexplore clearly publishing the most papers in this field. Unfortunately, I couldn't get hold of high quality data for conferences/journals, which I'm sure would have been useful.

Keywords

Finally, it's also interesting to analyse common keywords in existing publications. The graph below shows the most commonly occurring words in paper titles, with words such as 'a' and 'the' omitted. Besides the obvious terms such as 'nonintrusive', 'appliance', 'load', 'monitoring', 'energy' and 'disaggregation', other interesting terms pop up, such as 'smart', 'identification', 'residential' and 'home'.

Update 12.09.2016: cleaned up graph appearance.

Update 12.09.2016: added data and notebook used to generate these graphs to the nilm-papers github repository.

GridCarbon app updated to Android 5

Yesterday I pushed v2.1 of the GridCarbon app to the Google Play store. GridCarbon is an Android and iOS app that lets you track the carbon intensity of the UK electricity grid on your smartphone or tablet.

This latest release includes the following changes:

• Updated to Android 5 material design
• Improved robustness when generation mix data received from server contains missing or repeated elements
• New about page

The app also allows you track carbon intensity and generation sources over the past 24 hours via the landscape view.

If you've you have any comments, I'd love to hear them!

Collaboration with José Alcalá on using NILM for health monitoring

José has recently been visiting our lab in Southampton as part his PhD programme at the University of Alcalá. He's interested in how energy disaggregation can be applied within the healthcare domain, specifically to support elderly people living independently in their own homes. I've really enjoyed the collaboration so far, with José making great use of the data sets we've collected over the past few years. Despite only arriving in September last year, we've achieved a lot and recently co-authored a paper together. I hope José has found the visit useful, with him gaining experience from other members of our lab way beyond the domain of energy disaggregation, in fields such as Gaussian processes.

If you're also studying towards a PhD within the energy disaggregation domain and are interested in visiting our lab as part of your programme, please get in touch!

British Gas to acquire AlertMe

Today, both British Gas and AlertMe pushed out press releases announcing the acquisition of AlertMe by British Gas, for an estimated total of £65m. As a fresh member of British Gas' connected homes team and a long-term customer of AlertMe, I was really excited to hear the news and am eager to be part of the team to build the next generation of domestic energy feedback products. This move makes British Gas' position clear within the Internet of things / connected homes market, and will undoubtedly have a big impact on the field on energy disaggregation in the UK.