Wednesday, 16 April 2014

Introducing NILMTK: an open source toolkit for non-intrusive load monitoring

Today, Nipun Batra, Jack Kelly and Oliver Parson are really pleased to announce the release of NILMTK: an open source toolkit for non-intrusive load monitoring. The toolkit will allow researchers to easily develop algorithms which disaggregate a household’s total electricity consumption into individual appliances.

Specifically, the toolkit includes:

  • a number of parsers to read public data sets into a common format
  • a suite of statistical functions to analyse such data sets and identify potential problems
  • two benchmark energy disaggregation algorithms
  • a suite of evaluation metrics to compare disaggregation algorithms

Further details can be found in the accompanying paper recently accepted at e-Energy 2014 available via arXiv and Soton ePrints:

  • Batra, N., Kelly, J., Parson, O., Dutta, H., Knottenbelt, W., Rogers, A., Singh, A., Srivastava, M. (2014). NILMTK: An Open Source Toolkit for Non-intrusive Load Monitoring. In Fifth International Conference on Future Energy Systems (ACM e-Energy). Cambridge, UK. arXiv:1404.3878

This release is hopefully just the beginning of the toolkit’s contribution to energy disaggregation, and as such we welcome feedback and contributions to all aspects of the project.

This has been cross posted via Nipun Batra’s blog, Jack Kelly’s blog and the ORCHID project blog.

Thursday, 10 April 2014

Thesis available online

Today I can finally say that the finished version of my thesis has been submitted and is now available online. Here's the full reference:

Parson, Oliver (2014) Unsupervised Training Methods for Non-intrusive Appliance Load Monitoring from Smart Meter Data. University of Southampton, Electronics and Computer Science, Doctoral Thesis.

Here's a Wordle of my thesis:



And also here's the abstract:

Non-intrusive appliance load monitoring (NIALM) is the process of disaggregating a household’s total electricity consumption into its contributing appliances. Smart meters are currently being deployed on national scales, providing a platform to collect aggregate household electricity consumption data. Existing approaches to NIALM require a manual training phase in which either sub-metered appliance data is collected or appliance usage is manually labelled. This training data is used to build models of the house- hold appliances, which are subsequently used to disaggregate the household’s electricity data. Due to the requirement of such a training phase, existing approaches do not scale automatically to the national scales of smart meter data currently being collected.

In this thesis we propose an unsupervised training method which, unlike existing approaches, does not require a manual training phase. Instead, our approach combines general appliance knowledge with just aggregate smart meter data from the household to perform disaggregation. To do so, we address the following three problems: (i) how to generalise the behaviour of multiple appliances of the same type, (ii) how to tune general knowledge of appliances to the specific appliances within a single household using only smart meter data, and (iii) how to provide actionable energy saving advice based on the tuned appliance knowledge.

First, we propose an approach to the appliance generalisation problem, which uses the Tracebase data set to build probabilistic models of household appliances. We take a Bayesian approach to modelling appliances using hidden Markov models, and empirically evaluate the extent to which they generalise to previously unseen appliances through cross validation. We show that learning using multiple appliances vastly outperforms learning from a single appliance by 61–99% when attempting to generalise to a previously unseen appliance, and furthermore that such general models can be learned from only 2–6 appliances.

Second, we propose an unsupervised solution to the model tuning problem, which uses only smart meter data to learn the behaviour of the specific appliances in a given house-hold. Our approach uses general appliance models to extract appliance signatures from a household’s smart meter data, which are then used to refine the general appliance models. We evaluate the benefit of this process using the Reference Energy Disaggregation Data set, and show that the tuned appliance models more accurately represent the energy consumption behaviour of a given household’s appliances compared to when general appliance models are used, and furthermore that such general models can per- form comparably to when sub-metered data is used for model training. We also show that our tuning approach outperforms the current state of the art, which uses a factorial hidden Markov model to tune the general appliance models.

Third, we apply both of these approaches to infer the energy efficiency of refrigerators and freezers in a data set of 117 households. We evaluate the accuracy of our approach, and show that it is able to successfully infer the energy efficiency of combined fridge freezers. We then propose an extension to our model tuning process using factorial hidden semi-Markov models to model households with a separate fridge and freezer. Finally, we show that through this extension our approach is able to simultaneously tune the appliance models of both appliances.

The above contributions provide a solution which satisfies the requirements of a NIALM training method which is both unsupervised (no manual interaction required during training) and uses only smart meter data (no installation of additional hardware is required). When combined, the contributions presented in this thesis represent an advancement in the state of the art in the field of non-intrusive appliance load monitoring, and a step towards increasing the efficiency of energy consumption within households.

Tuesday, 8 April 2014

International WikiEnergy Data Conference 2014

The WikiEnergy Data Conference has just been announced, and will be co-hosted by Carnegie Mellon University and Pecan Street Research Institute. The conference will feature presentations from computer science, public policy, and engineering graduate students selected as finalists for the Pike Powers energy research fellowship.

Some important information:

  • When: 4-5 June 2014
  • Austin, TX, USA
  • Objective: To convene the WikiEnergy researcher community, highlighting the results of research conducted by graduate students selected as finalists for the Pike Powers Energy Research Fellowship.
The first day of the workshop will be dedicated to the Pike Powers Fellowships, while the second day is scheduled as an industry day. Attendance is free, although attendees should  register via the conference website. The workshop will also be co-located with the NILM 2014 workshop on 3 June.

Wednesday, 19 March 2014

Visit to Telekom Malaysia R&D

I’ve just returned from a visit to Telekom Malaysia Research & Development during which I delivered a 5 day course on non-intrusive appliance load monitoring. The course aimed to provide a broad overview of the range of research happening in both academia and industry as well as an in-depth description of the current state of the art. Below is a photo of the group taken outside their research facility:




With a roughly 50/50 split between presentations and practical exercises, the course covered the following areas:

  • Introduction & history
  • Data sets & preprocessing
  • Event based & non-event based methods
  • Data sets & accuracy metrics
  • Disaggregation exercise using factorial hidden Markov models


I really feel that although many of the attendees started the course with little or no background in energy disaggregation, everyone left the course with a broad understanding and appreciation of the successes and limitations within the field. It was truly a pleasure to be invited by TM R&D and I hope they are able to invest their newly found knowledge in NIALM into practical solutions.

If you’re interested in a similar course for your company, please feel free to drop me an email.

Thursday, 13 February 2014

GridCarbon Android app v2 released

I'm very happy to announce the release of an update to the GridCarbon Android app. The app allows you to track the carbon intensity of the UK electricity grid on your phone or tablet.
The demand for electricity in the UK varies throughout the day, and thus, the mix of generators supplying this electricity continually changes. As a result, the carbon intensity of the electricity – the quantity of CO2 produced for 1 kWh of electricity consumed – also varies continually. Deferring your use of electricity to off-peak times, when the carbon intensity is low, can help reduce your carbon footprint.

The app features a breakdown of the the current electricity generation sources:


The landscape view shows how both the carbon intensity of the grid and the generation sources varied over the previous 24 hours:


There is also an iOS version of GridCarbon which shares the same functionality as the Android app.

Friday, 7 February 2014

NILM in the New Scientist

A couple of weeks ago, Google acquired Nest, a smart thermostat manufacturer. The Nest thermostat aims to combine manually set temperature preferences with automatically learned household occupancy schedules. The acquisition seemed to prompt a few privacy concerns, given the unification of household occupancy data with the information Google currently stores about its customers. This is where NILM got its mention in the New Scientist, given that private information can also be inferred from household aggregate electricity data. A summary article even referenced research which showed that the film being watched on a plasma TV can be inferred from 2 Hz smart meter data, assuming two 5 minute periods of data can be extracted when no other appliances are changing state. Unfortunately though, none of these articles seemed to mention that such methods would require higher frequency data than will be automatically uploaded to utilities by smart meters, and therefore would require explicit consent from the customer to opt-in to such a system.

Wednesday, 29 January 2014

NILM 2014: Second International Workshop on Non-intrusive Load Monitoring

The Second International Workshop on Non-intrusive Load Monitoring is being organised by Mario Bergés and Zico Kolter in partnership with Pecan Street. The workshop is a follow up to the 2012 NILM Workshop held in Pittsburgh, which brought academics and vendors interested in energy disaggregation together for the first time. Full information about the upcoming workshop can be found at nilmworkshop.org.

Some important information:

  • When: 3 June 2014
  • Where: Austin, TX, USA
  • Objective: To review the main types of approaches that have been explored to date to solve the problem of electricity disaggregation, and to then discuss possible paths forward knowing what has been tried and what has yet to be experimented.
The workshop will feature talks from invited speakers, paper presentations and a poster session. Attendees are be able to register for the workshop at nilmworkshop.org. Authors will also be able to submit papers via the same website, for which submission system is live and will close on 28 March 2014. The workshop will also be co-located with a two day workshop hosted by Pecan Street on 4-5 June.