Is energy disaggregation a solved problem?

Every week I seem to read about a new start-up company, or a new academic paper, or a new patent, which claims to solve the problem of energy disaggregation. This is probably what makes it so exciting to be working in this field, given its recent explosion in size and attraction of much funding across the world. However, as a lonely PhD student, it's also a little intimidating to try to make an impact in this increasingly crowded field. I therefore decided to write this post which will hopefully persuade others that there are still a lot more problems just waiting to be tackled.

The best way to decide whether the problem has been solved is clearly to first unravel what we mean by the term solved. The two most important factors in my opinion have got to be the scenario and and the accuracy, which I've picked apart below.

Scenario

The most compelling scenarios are the ones in which the software does as much of the hard work as possible, allowing the hardware to remain simple and inexpensive. Furthermore, the installation process should also be free from any dedicated training phase in which the temporary control of appliances is required or additional monitoring equipment is deployed. This style of approach clearly affords the maximum scalability, which is essential if disaggregated data is ever to reach the masses. While approaches are still being proposed that don't address this scenario, there's still quite a way to go yet.

Accuracy

It's very easy to get carried away when reading papers how one approach has achieved X% accuracy, while another achieved Y% accuracy. However, I'm fairly sure that aiming for 100% disaggregation accuracy is not only unrealistic, but also unnecessary too. I think it's far more important that energy disaggregation is used as a platform from which personalised, actionable, energy-saving suggestions can be derived. Therefore, disaggregation only needs to be accurate enough to convincingly determine the bigger picture, and any effort beyond this is likely to be a waste of time.


On a final note, even if there exists an approach that's managed to address the above scenario to a suitable level of accuracy, this is still only only piece of the disaggregation puzzle. The diversity of electricity consumption means that different countries and buildings require different approaches, and there will always be a niche sub-area of disaggregation just waiting to be found.

NIALM companies in France

I've recently come across two (fairly) new French companies working on energy disaggregation, so I thought I'd share a short summary of each. I've also updated my post on NIALM in industry.

Fludia, Paris, France

Fludia is a French company specialising in energy management, who aim to provide their customers with technology to increase the energy efficiency of their homes. They have developed a device to retrofit non-smart meters, called Fludiameter, such that 1 minute resolution energy data can be collected without installing a whole new meter. Fludia also provide a tool to break down electricity consumption into its end uses, called Beluso, which makes use of household aggregate data and also information entered from a household survey.

Wattseeker, Nice, France

Wattseeker offer a datalogger, which includes a number of current clamps and the ability to upload data via 3G, Ethernet or Wi-Fi. These current clamps sample the current and voltage at a kHz rate since real and reactive power are reported, along with harmonics etc. However, the installation does require a short shut down of the building's power. Their disaggregation system, LYNX, then disaggregates the electricity consumption to provide actionable energy saving suggestions. Their website indicates that each current clamp can disaggregate up to 12 appliances, with an accuracy of +/- 2%.

Thanks to Jack Kelly for the link to Fludia!

Differences between disaggregation in the UK and US

I was asked recently what I thought the differences were between disaggregation in UK households in comparison to those in the US. My first reaction was that a UK household is just a simplification of a US household, but I've been thinking it over since and come up with quite a few key differences. This is my attempt to categorise them:

Load diversity

There are quite a few differences in the loads that can be found in American households compared to their British counterparts. The most significant of which is the presence of heating, ventilation and air-conditioning (HVAC) systems in American households. In comparison, UK households are far more likely to be heated by gas, and not require air-conditioning at all. Since the HVAC system often constitutes the largest electrical load in American households, the problem of disaggregating the remaining appliances is clearly simpler for UK households. Furthermore, in my experience, American households contain not only a wider range of loads (pool pumps, etc.) but also contain more duplicate appliances (2 or 3 fridge/freezers, etc.). This again contributes to a harder disaggregation problem for US households.

Split-phase power

American and British households also vary in the way that electricity is supplied to the properties. UK households typically receive single-phase (1 electricity input wire) power at 230 volts, while US households receive split-phase (2 electricity input wires) power at 120 volts. The case of split-phase power provides a convenient opportunity to install two current clamps instead of one. Through this small additional installation cost, the complexity of the disaggregation is more than halved, since most appliances are connected to only one of the input cables.

Smart meters

To the best of my knowledge, the capabilities of smart meters in both countries are yet to be finalised. However, the UK provides an interesting extension to the government mandated smart meter rollout, in which households will also receive an in-home display (IHD). IHDs will primarily provide a household's occupants with real-time information, such as their current power demand and cost of electricity. However, since these devices are likely to have access to electricity data at a higher granularity than is transmitted to the energy provider, they provide a convenient platform on which disaggregation can be performed. Furthermore, performing on higher granularity data within each household even circumvents any privacy concerns related to transmitting private data outside of the home. However, IHDs will clearly have limited resources in terms of processing power etc., which raises the interesting field of resource constrained energy disaggregation.

Postdoc position through Doctoral Prize award

Recently I've been thinking a lot about my career beyond my PhD, for which the funding runs out at the end of this year. Having experienced energy disaggregation from both sides of the academia/industry fence, I've learned a lot about both getting algorithms to scale to huge amounts of data and presenting my work at international venues. However, as I entered the final year of my PhD, I was enjoying my PhD so much that I decided to apply for the Doctoral Prize award; a grant for a one year postdoc position designed specifically to increase the impact of technology developed during a PhD.

Today I'm excited to announce that my application has been accepted, and assuming that all goes to plan while writing up my thesis, I'll start work as a postdoc in the new year. I'm hoping this position will give me the opportunity to put my PhD work into the hands of real users and show that energy disaggregation really can provide actionable suggestions and lead to real energy savings. I'm excited about the new challenges this will bring and the insight it will provide to this growing community.

Of course this also means that I'll continue to blog about energy disaggregation for at least another year.

NIALM and limited sub-metering

I've read a couple of papers recently about using minimal sub-metering to enhance the accuracy of NIALM. They are:

• Yongcai Wang, Xiaohong Hao, Lei Song, Chenye Wu, Yuexuan Wang, Changjian Hu, and Lu Yu. Tracking states of massive electrical appliances by lightweight metering and sequence decoding. In Proceedings of the Sixth International Workshop on Knowledge Discovery from Sensor Data (SensorKDD '12). ACM, New York, NY, USA, 34-42, 2012.
• Nambi, S. N., Akshay, U., Papaioannou, Thanasis G., Chakraborty, Dipanjan, Aberer, Karl. Sustainable Energy Consumption Monitoring in Residential Settings. 2nd IEEE INFOCOM Workshop on Communications and Control for Smart Energy Systems, Turin, Italy, April 14-19, 2013.

This problem introduces an interesting trade-off between disaggregation accuracy and physical intrusion. I've definitely wondered where I should be installing appliance plug monitors in my home, and this type of approach potentially provides such answers.

However, I can't help but feel that any intrusion beyond the installation of a household-level monitor prevents the scalability of this technology. I accept that 100% disaggregation accuracy for a previously unseen household with a single aggregate meter is likely be to impossible. However, I find software solutions far more compelling than hardware solutions, due to their ability to scale to any data set containing a single stream of aggregate readings. Furthermore, I think the state of the art in unsupervised NIALM is still far from the best that can be achieved, and there are still many algorithmic improvements to be made.

Residential vs Commercial/Industrial NIALM

I was recently asked how energy disaggregation in the residential sector compares to the commercial and industrial sectors, since the majority of academic research focuses on domestic energy consumption. This got me thinking about the major differences, and why academic research focuses on disaggregating household appliances, so I decided to collect my thoughts here.

Total energy consumption

As a general rule of thumb, I think of a commercial premises (e.g. a shop, office or restaurant) as consuming roughly an order of magnitude more energy than an average household. Clearly there's a huge variability here, but I think it's a fair assumption given that, in general, there are more people spending more time in these premises. I think industrial premises are bound to consume even more energy than commercial premises, given that a single type of machinery could quickly consume more energy than a household or shop could in a day. This would surely make energy disaggregation more attractive in these settings, given the greater potential for energy savings.

Variation of loads

I'd expect commercial and industrial premises to contain many more electrical loads compared to a household. As a result, it's likely that more loads will be run simultaneously, and also more duplicate appliances will exist within a single premises. This clearly increases the complexity of the disaggregation problem. However, I'd also expect these types of premises to show a higher correlation over daily and weekly cycles, and further similarities are likely to exist between chains of commercial premises. These differences clearly change the focus of the disaggregation problem, and would likely require different approaches to feature selection and parameter learning.

Cost/benefit ratio

Given the greater potential for energy savings, the scenario becomes quite different to that of household energy disaggregation. There's clearly an argument for greater investment in monitoring equipment, such as higher frequency sampling or sub-metering. It might even be financially viable to monitor each load individually in an industrial environment, since the cost of the energy monitor is much lower in comparison to the size of the load. This begs the question of whether NIALM is even necessary in such situations.

Motivation and access

In my opinion, the domestic scenario is made far more compelling by the smart meter deployments mandated by many governments around the world. This means that household electricity data will soon be available at huge scales, and therefore there will be huge potential for advances in disaggregation algorithms. Conversely, commercial and industrial premises owners probably care relatively little about the electricity costs, as a result of the small cost to their business compared to other factors. I think these points help to explain why academic research has so far been concentrated around energy disaggregation in residential settings.

NIALM interest worldwide

Without attending an international conference, it can be hard to get an idea for how localised the interest in a technology might be. To this end, I've shared the following map of the visits to this blog over the last two years:

where dark green countries represent the most hits and white countries represent no hits.

The two darkest countries are the United States and the United Kingdom by a long a way, which doesn't particularly surprise me. The next band of interest seems to be from western Europe, Canada, Russia, India and Australia. I think this represents the nationalities of most of the people I've met and that of authors of papers I've read fairly well.

I know that colouring countries isn't an ideal representation of interest over a geographical area, but I didn't want this to turn into an exercise in statistics.