A while ago, I wrote a post about the Belkin Energy Disaggregation Competition, which concluded a couple of weeks ago. The competition drew entries from 165 participating teams, who each provided up to 169 submissions. The top 3 participants shared prizes from a combined pot of $24,000, while a separate data visualisation competition had a prize of $1,000.
It seemed like most entrants were regular Kagglers, with little participation from NIALM companies or academics with the NIALM field. Although I understand that many companies are likely unwilling to participate to prevent their secrets being divulged, I wonder if greater participation from the existing NIALM academic field could have been achieved by hosting the competition at a relevant conference or workshop. I would love to see the winners of the competition invited give a talk about their approaches!
The data used in the competition consisted of a public (training) and a private (test) data set, collected from 4 households. The training data included both household aggregate data and individual appliance sub-metered data. However, only the aggregate test data was released, while the sub-metered data was kept private for the evaluation of each submission. As a result, although a cross-validation evaluation technique was used, the participants were crucially not required to generalise to new households since sub-metered data was available from each test household for training.
With each submission, the public leaderboard was updated showing the best performance for each user over half of the private test data, while their undisclosed performance over the other half of the test data was used to calculate the final standings. Interestingly, the winner of the competition shown by the final standings was actually only ranked 6th on the public leaderboard. This suggests that many participants might have been overfitting their algorithms to the half of the test data for which the performance was disclosed, while the competition winner had not optimised their approach in such a way.
An interesting forum thread seems to show that most successful participants used an approach based on only low-frequency data, despite the fact that high-frequency data was also provided. This seems to contradict most academic research, which generally shows that high-frequency based approaches will outperform low-frequency methods. A reason for this could be that, although high-frequency based approaches perform well in laboratory test environments, their features do not generalise well over time, and as a result algorithm training quickly becomes outdated. However, another reason could have been that the processing of the high-frequency features was simply too time consuming, and better performance could be achieved by concentrating on the low-frequency data given the deadline of the competition.
Overall, I think the competition was very successful in provoking interest in energy disaggregation from a new community, and I hope that any follow up competitions follow a similar format. Furthermore, I think that hosting a prize giving and presentation forum at a relevant conference and workshop would inspire greater participation from academics already working in the field in NIALM.
Tuesday, 12 November 2013
Tuesday, 5 November 2013
Thesis Finished!
Today I finally finished my thesis titled: 'Unsupervised Training Methods for Non-intrusive Appliance Load Monitoring from Smart Meter Data'. My viva (defence) is scheduled for December, and I plan to upload a final version upon addressing any revisions it brings up. In the meantime, I've included the thesis abstract below:
Non-intrusive appliance load monitoring (NIALM) is the process of disaggregating a household's total electricity consumption into its contributing appliances. Smart meters are currently being deployed on national scales, providing a platform to collect aggregate household electricity consumption data. Existing approaches to NIALM require a manual training phase in which either sub-metered appliance data is collected or appliance usage is manually labelled. This training data is used to build models of the household appliances, which are subsequently used to disaggregate the household's electricity data. Due to the requirement of such a training phase, existing approaches do not scale automatically to the national scales of smart meter data currently being collected.
In this thesis we propose an unsupervised training method which, unlike existing approaches, does not require a manual training phase. Instead, our approach combines general appliance knowledge with just aggregate smart meter data from the household to perform disaggregation. To do so, we address the following three problems: (i) how to generalise the behaviour of multiple appliances of the same type, (ii) how to tune general knowledge of appliances to the specific appliances within a single household using only smart meter data, and (iii) how to provide actionable energy saving advice based on the tuned appliance knowledge.
First, we propose an approach to the appliance generalisation problem, which uses the Tracebase data set to build probabilistic models of household appliances. We take a Bayesian approach to modelling appliances using hidden Markov models, and empirically evaluate the extent to which they generalise to previously unseen appliances through cross validation. We show that learning using multiple appliances vastly outperforms learning from a single appliance by 61-99% when attempting to generalise to a previously unseen appliance, and furthermore that such general models can be learned from only 2-6 appliances.
Second, we propose an unsupervised solution to the model tuning problem, which uses only smart meter data to learn the behaviour of the specific appliances in a given household. Our approach uses general appliance models to extract appliance signatures from a household's smart meter data, which are then used to refine the general appliance models. We evaluate the benefit of this process using the Reference Energy Disaggregation Data set, and show that the tuned appliance models more accurately represent the energy consumption behaviour of a given household's appliances compared to when general appliance models are used, and furthermore that such general models can perform comparably to when sub-metered data is used for model training. We also show that our tuning approach outperforms the current state of the art, which uses a factorial hidden Markov model to tune the general appliance models.
Third, we apply both of these approaches to infer the energy efficiency of refrigerators and freezers in a data set of \117 households. We evaluate the accuracy of our approach, and show that it is able to successfully infer the energy efficiency of combined fridge freezers. We then propose an extension to our model tuning process using factorial hidden semi-Markov models to model households with a separate fridge and freezer. Finally, we show that through this extension our approach is able to simultaneously tune the appliance models of both appliances.
The above contributions provide a solution which satisfies the requirements of a NIALM training method which is both unsupervised (no manual interaction required during training) and uses only smart meter data (no installation of additional hardware is required). When combined, the contributions presented in this thesis represent an advancement in the state of the art in the field of non-intrusive appliance load monitoring, and a step towards increasing the efficiency of energy consumption within households.
Non-intrusive appliance load monitoring (NIALM) is the process of disaggregating a household's total electricity consumption into its contributing appliances. Smart meters are currently being deployed on national scales, providing a platform to collect aggregate household electricity consumption data. Existing approaches to NIALM require a manual training phase in which either sub-metered appliance data is collected or appliance usage is manually labelled. This training data is used to build models of the household appliances, which are subsequently used to disaggregate the household's electricity data. Due to the requirement of such a training phase, existing approaches do not scale automatically to the national scales of smart meter data currently being collected.
In this thesis we propose an unsupervised training method which, unlike existing approaches, does not require a manual training phase. Instead, our approach combines general appliance knowledge with just aggregate smart meter data from the household to perform disaggregation. To do so, we address the following three problems: (i) how to generalise the behaviour of multiple appliances of the same type, (ii) how to tune general knowledge of appliances to the specific appliances within a single household using only smart meter data, and (iii) how to provide actionable energy saving advice based on the tuned appliance knowledge.
First, we propose an approach to the appliance generalisation problem, which uses the Tracebase data set to build probabilistic models of household appliances. We take a Bayesian approach to modelling appliances using hidden Markov models, and empirically evaluate the extent to which they generalise to previously unseen appliances through cross validation. We show that learning using multiple appliances vastly outperforms learning from a single appliance by 61-99% when attempting to generalise to a previously unseen appliance, and furthermore that such general models can be learned from only 2-6 appliances.
Second, we propose an unsupervised solution to the model tuning problem, which uses only smart meter data to learn the behaviour of the specific appliances in a given household. Our approach uses general appliance models to extract appliance signatures from a household's smart meter data, which are then used to refine the general appliance models. We evaluate the benefit of this process using the Reference Energy Disaggregation Data set, and show that the tuned appliance models more accurately represent the energy consumption behaviour of a given household's appliances compared to when general appliance models are used, and furthermore that such general models can perform comparably to when sub-metered data is used for model training. We also show that our tuning approach outperforms the current state of the art, which uses a factorial hidden Markov model to tune the general appliance models.
Third, we apply both of these approaches to infer the energy efficiency of refrigerators and freezers in a data set of \117 households. We evaluate the accuracy of our approach, and show that it is able to successfully infer the energy efficiency of combined fridge freezers. We then propose an extension to our model tuning process using factorial hidden semi-Markov models to model households with a separate fridge and freezer. Finally, we show that through this extension our approach is able to simultaneously tune the appliance models of both appliances.
The above contributions provide a solution which satisfies the requirements of a NIALM training method which is both unsupervised (no manual interaction required during training) and uses only smart meter data (no installation of additional hardware is required). When combined, the contributions presented in this thesis represent an advancement in the state of the art in the field of non-intrusive appliance load monitoring, and a step towards increasing the efficiency of energy consumption within households.
Monday, 21 October 2013
Neurio - a new energy disaggregation product looking for Kickstarter funding
Jack Kelly recently linked me to an exciting new company called Energy Aware. Energy Aware are currently seeking Kickstarter funding to develop their electricity disaggregation technology Neurio, which consists of:
Two things strike me that set Neurio apart from the competition:
- a hardware sensor featuring two CT clamps, capable of reporting voltage, current, real power and power factor at 1 second intervals
- a set of cloud-based disaggregation algorithms which breakdown your household electricity usage into individual appliances
Two things strike me that set Neurio apart from the competition:
- Interconnectivity with third party services - Neurio are keen to connect the detected appliance switch events (e.g. lights turned on) to any third party services through their open RESTful API.
- Real-time notifications - The company will also provide a mobile app, with the aim of notifying their users when an appliance (e.g. oven) is left on.
Monday, 23 September 2013
BuildSys 2013 Interesting Papers
The 5th ACM Workshop On Embedded Systems For Energy-Efficient Buildings (BuildSys) is coming up in November and I wanted to share a few papers which have recently been accepted there. Although the camera ready submissions weren't due at the time of writing, I've managed to get hold of a pre-print of a few interesting papers, which the authors are happy for me to share.
- Towards a Smart Home Framework, Muddasser Alam, Alper T. Alan, Alex Rogers, and Sarvapali D. Ramchurn. Agents, Interaction and Complexity Research Group, University of Southampton, UK.
- This paper presents the Smart Home Framework simulation platform for modelling smart homes. The platform provides extendable building blocks for smart households, such as micro-generation, energy storage, in addition to the components of more traditional homes, such as household electronics and heating. The framework has been designed to easily enable the simulation of different household environments in order to test the potential for different smart technologies.
- It’s Different: Insights into home energy consumption in India, Nipun Batra, Manoj Gulati, Amarjeet Singh, Mani B. Srivastava. Indraprastha Institute of Information Technology, Delhi, India & University of California Los Angeles, United States.
- This paper presents a new data set called Home Deployment, collected from a single household in Delhi. The authors describe many factors which distinguish the data set from other data sets collected from developed countries, such as the unreliability of the electrical grid and Internet connectivity. The data spans 73 days, collected from household-level, circuit-level, and appliance-level meters.
- A Scalable Low-Cost Solution to Provide Personalised Home Heating Advice to Households, Alex Rogers, Siddhartha Ghosh, Reuben Wilcock and Nicholas R. Jennings. University of Southampton, UK.
- This paper presents MyJoulo, a low-cost hardware solution which provides personalised home heating advice to households. The system consists of a single temperature logger which is placed on top of a household's thermostat, which is able to learn the thermal properties of a household. The thermal model is then used to provide feedback to the household occupants by comparing your learned thermostat set point to the national average, in addition to estimating the the potential savings should the set point be reduced or the timer settings changed.
Thursday, 29 August 2013
Disaggregation in the UK
I was recently asked about smart meter legislation in UK, and its direction with respect to energy disaggregation. Here was my response:
As far as I'm aware, UK smart meters are not required to provide appliance specific electricity breakdowns. Instead, they're being installed primarily for automatic billing purposes, but also to provide total household consumption information and potentially real-time pricing data via in home displays. Some general information about UK smart meters is available from the UK Government, and the latest technical specification for smart meters is the Smart Metering Equipment Technical Specification v2.
In the UK, most consumers are signed up to electricity contracts with one of the big 6 energy suppliers. However, British Gas, the largest of these suppliers, recently started running a TV advert in which their smart meters were shown to break down their households' electricity usage into heating and lighting. In my opinion, this shows that although appliance specific breakdowns are not required by government directives, energy suppliers are keen to provide such services in order to incentivise consumers. I think breaking down electricity usage into heating and lighting is only the beginning of what electricity disaggregation can offer to consumers, and we're likely to see some interesting competition in this domain between some of the major players in the UK energy sector.
As far as I'm aware, UK smart meters are not required to provide appliance specific electricity breakdowns. Instead, they're being installed primarily for automatic billing purposes, but also to provide total household consumption information and potentially real-time pricing data via in home displays. Some general information about UK smart meters is available from the UK Government, and the latest technical specification for smart meters is the Smart Metering Equipment Technical Specification v2.
In the UK, most consumers are signed up to electricity contracts with one of the big 6 energy suppliers. However, British Gas, the largest of these suppliers, recently started running a TV advert in which their smart meters were shown to break down their households' electricity usage into heating and lighting. In my opinion, this shows that although appliance specific breakdowns are not required by government directives, energy suppliers are keen to provide such services in order to incentivise consumers. I think breaking down electricity usage into heating and lighting is only the beginning of what electricity disaggregation can offer to consumers, and we're likely to see some interesting competition in this domain between some of the major players in the UK energy sector.
Friday, 23 August 2013
Incorporating general appliance knowledge into disaggregation algorithms
I've recently been thinking a lot about how to incorporate prior knowledge into disaggregation training algorithms. By prior knowledge, I mean general information about how appliance types operate, i.e. what makes a fridge different from a washing machine. However, this prior knowledge should not be specific to a single household, and should therefore generalise to previously unseen households. Clearly, such prior knowledge is required if a NIALM system is to operate without manual intervention. To date, I have seen two categories of approaches which incorporate prior information into the learning process, which I have described below.
This approach first aims to identify the characteristics of the appliances in a household without any prior knowledge. This produces a list of appliances (e.g. appliance 1, appliance 2) with their corresponding behaviour. Next, this category of approaches use a second step to assign labels to the learned appliances (e.g. appliance 1 = fridge, appliance 2 = washing machine). A diagram of this approach is given below:
This approach has been adopted in some recent state-of-the-art disaggregation papers. Both Kim et al. (2011) and Kolter et al. (2012) both use an unsupervised learning approach to the first step, while the manual labelling of learned appliance models is required by the second step.
This approach aims to use both aggregate data and prior knowledge in order to simultaneously learn appliance models, as shown below:
This seems to be a more principled approach, in that prior knowledge is not ignored when the learning algorithm identifies distinct appliances within the aggregate load. We demonstrated such an approach in a recent paper (Parson et al., 2012). Furthermore, this type of approach lends itself well to Bayesian learning, whereby the learned appliances models constitute a weighted combination of information extracted from aggregate data and the general appliance models. Such an approach is detailed in Johnson and Willsky (2013).
In summary, I believe it is essential to incorporate such general appliance knowledge into the learning algorithms of energy disaggregation systems to allow them to scale to large numbers of previously unseen households. Only in the past few years has published work come close to Hart's vision of a Manual Setup NIALM (Hart, 1992), but the problem is still far from solved.
Sequential learning and labelling
This approach first aims to identify the characteristics of the appliances in a household without any prior knowledge. This produces a list of appliances (e.g. appliance 1, appliance 2) with their corresponding behaviour. Next, this category of approaches use a second step to assign labels to the learned appliances (e.g. appliance 1 = fridge, appliance 2 = washing machine). A diagram of this approach is given below:
This approach has been adopted in some recent state-of-the-art disaggregation papers. Both Kim et al. (2011) and Kolter et al. (2012) both use an unsupervised learning approach to the first step, while the manual labelling of learned appliance models is required by the second step.
Simultaneous learning and labelling
This approach aims to use both aggregate data and prior knowledge in order to simultaneously learn appliance models, as shown below:
This seems to be a more principled approach, in that prior knowledge is not ignored when the learning algorithm identifies distinct appliances within the aggregate load. We demonstrated such an approach in a recent paper (Parson et al., 2012). Furthermore, this type of approach lends itself well to Bayesian learning, whereby the learned appliances models constitute a weighted combination of information extracted from aggregate data and the general appliance models. Such an approach is detailed in Johnson and Willsky (2013).
In summary, I believe it is essential to incorporate such general appliance knowledge into the learning algorithms of energy disaggregation systems to allow them to scale to large numbers of previously unseen households. Only in the past few years has published work come close to Hart's vision of a Manual Setup NIALM (Hart, 1992), but the problem is still far from solved.
Friday, 9 August 2013
Outstanding student paper award at IJCAI-13
A while ago, I wrote a blog post about our paper which aims to detect the New Forest Cicada from smartphone audio recordings. Davide Zilli recently presented this work in Beijing, and I'm very happy to announce that paper was awarded the outstanding student paper award. Although we're yet to rediscover the cicada native to the UK, the app has collected thousands of audio recordings worldwide and located similar species of cicada in nearby countries. There are still a few more days left of the cicada season in the UK, so check out the New Forest Cicada Project website for more details if you're thinking of heading to the New Forest.
 |
| The New Forest Cicada |
Subscribe to:
Posts (Atom)