Why NIALM shouldn't be modelled as the knapsack or subset-sum problem

In his seminal work, Hart (1992) highlighted the similarities between the appliance disaggregation problem and some well-studied combinatorial optimisation problems. He stated that if the power demand of each appliance is known, the disaggregation problem can be modelled as an instance of the subset sum problem. This approach selects the set of appliances whose power demands sum to the household aggregate power, applied independently to each aggregate power reading. However, Hart identified a core problem of this approach:

• Small fluctuations in the aggregate power lead to solutions with unrealistically high numbers of appliance switch events.

To confirm this in practice, I evaluated this formulation using simulated data. My simulations showed that this approach performs poorly for realistic numbers of appliances even with ideal conditions of no noise and only on-off appliances. Clearly, this is not a good model for appliance disaggregation.

More recently Egarter et al. (2013) extended this model, stating that if both the power demand and duration of usage of each appliance are known, the disaggregation problem can be modelled as a two-dimensional knapsack problem. This approach is similar to Hart's proposed method, with the exception that each aggregate measurement is not considered as an individual optimisation problem. Instead, appliance operations must persist for the exact given duration. However, I see two major problems with this approach:

1. Optimal solutions might give unrealistic results. As the baseload power increases, I'd expect the number of solutions (those that produce a small difference between the sum of appliance power and the aggregate power) to increase rapidly. As a result, I seriously doubt that the actual optimal solution would give the best disaggregation accuracy.
2. Appliance durations are often highly variable. Unlike power demand, their duration of usage is often determined by the user. However, extending this formalism such that appliances could have a variable duration would hugely increase the size of the solution space.

Although these approaches show promise in simple scenarios, I don't believe they capture the flexibility required by the energy disaggregation scenario. Instead, I believe probabilistic approaches are more appropriate, given their ability to reason over different solutions by estimating the likelihood of different event sequences.

Paper accepted at IJCAI on Biodiversity Monitoring

We recently had a paper accepted at the International Joint Conference on Artificial Intelligence 2013 based on the detection of the New Forest Cicada from audio recordings.

Abstract:

Automated acoustic recognition of species aims to provide a cost-effective method for biodiversity monitoring. This is particularly appealing for detecting endangered animals with a distinctive call, such as the New Forest cicada. To this end, we pursue a crowdsourcing approach, whereby the millions of visitors to the New Forest will help to monitor the presence of this cicada by means of a smartphone app that can detect its mating call. However, current systems for acoustic insect classification are aimed at batch processing and not suited to a real-time approach as required by this system, because they are too computationally expensive and not robust to environmental noise. To address this shortcoming we propose a novel insect detection algorithm based on a hidden Markov model to which we feed as a single feature vector the ratio of two key frequencies extracted through the Goertzel algorithm. Our results show that this novel approach, compared to the state of the art for batch insect classification, is much more robust to noise while also reducing the computational cost.

Reference:

Davide Zilli, Oliver Parson, Geoff V Merrett, Alex Rogers. A Hidden Markov Model-Based Acoustic Cicada Detector for Crowdsourced Smartphone Biodiversity Monitoring. In: 23rd International Joint Conference on Artificial Intelligence. Beijing, China. 2013.

Releasing code used in academic publications

In academia, it's good scientific practice to release code after producing a publication. This allows other researchers to replicate the publication's findings, benchmark against the published approach, and even extend the work in new directions. However, it's actually quite rare to find a given paper's code online, and even more so to find the documentation required to make use of the code. Although having said this, many academics are happy to provide their code when requested, especially if they're no longer pursuing that field of research.

Unfortunately, I seem to have fallen into the habit of providing my code upon request, instead of releasing it online. I guess the reason for this is that the task of releasing code always seems to be superseded by other upcoming deadlines, and consequently remains at the bottom of my to do list. As a result, I end up giving out an undocumented archive of my code, which I can't believe is particularly useful to many people.

Therefore, I've decided to release all my code at the end of my PhD. I'm hoping that after submitting my thesis, I'll have a window of time to tidy up these kind of loose ends. However, if I get to the end of the year (2013) without releasing anything, please remind me of this post!

Is energy disaggregation a solved problem?

Every week I seem to read about a new start-up company, or a new academic paper, or a new patent, which claims to solve the problem of energy disaggregation. This is probably what makes it so exciting to be working in this field, given its recent explosion in size and attraction of much funding across the world. However, as a lonely PhD student, it's also a little intimidating to try to make an impact in this increasingly crowded field. I therefore decided to write this post which will hopefully persuade others that there are still a lot more problems just waiting to be tackled.

The best way to decide whether the problem has been solved is clearly to first unravel what we mean by the term solved. The two most important factors in my opinion have got to be the scenario and and the accuracy, which I've picked apart below.

Scenario

The most compelling scenarios are the ones in which the software does as much of the hard work as possible, allowing the hardware to remain simple and inexpensive. Furthermore, the installation process should also be free from any dedicated training phase in which the temporary control of appliances is required or additional monitoring equipment is deployed. This style of approach clearly affords the maximum scalability, which is essential if disaggregated data is ever to reach the masses. While approaches are still being proposed that don't address this scenario, there's still quite a way to go yet.

Accuracy

It's very easy to get carried away when reading papers how one approach has achieved X% accuracy, while another achieved Y% accuracy. However, I'm fairly sure that aiming for 100% disaggregation accuracy is not only unrealistic, but also unnecessary too. I think it's far more important that energy disaggregation is used as a platform from which personalised, actionable, energy-saving suggestions can be derived. Therefore, disaggregation only needs to be accurate enough to convincingly determine the bigger picture, and any effort beyond this is likely to be a waste of time.


On a final note, even if there exists an approach that's managed to address the above scenario to a suitable level of accuracy, this is still only only piece of the disaggregation puzzle. The diversity of electricity consumption means that different countries and buildings require different approaches, and there will always be a niche sub-area of disaggregation just waiting to be found.

NIALM companies in France

I've recently come across two (fairly) new French companies working on energy disaggregation, so I thought I'd share a short summary of each. I've also updated my post on NIALM in industry.

Fludia, Paris, France

Fludia is a French company specialising in energy management, who aim to provide their customers with technology to increase the energy efficiency of their homes. They have developed a device to retrofit non-smart meters, called Fludiameter, such that 1 minute resolution energy data can be collected without installing a whole new meter. Fludia also provide a tool to break down electricity consumption into its end uses, called Beluso, which makes use of household aggregate data and also information entered from a household survey.

Wattseeker, Nice, France

Wattseeker offer a datalogger, which includes a number of current clamps and the ability to upload data via 3G, Ethernet or Wi-Fi. These current clamps sample the current and voltage at a kHz rate since real and reactive power are reported, along with harmonics etc. However, the installation does require a short shut down of the building's power. Their disaggregation system, LYNX, then disaggregates the electricity consumption to provide actionable energy saving suggestions. Their website indicates that each current clamp can disaggregate up to 12 appliances, with an accuracy of +/- 2%.

Thanks to Jack Kelly for the link to Fludia!

Differences between disaggregation in the UK and US

I was asked recently what I thought the differences were between disaggregation in UK households in comparison to those in the US. My first reaction was that a UK household is just a simplification of a US household, but I've been thinking it over since and come up with quite a few key differences. This is my attempt to categorise them:

Load diversity

There are quite a few differences in the loads that can be found in American households compared to their British counterparts. The most significant of which is the presence of heating, ventilation and air-conditioning (HVAC) systems in American households. In comparison, UK households are far more likely to be heated by gas, and not require air-conditioning at all. Since the HVAC system often constitutes the largest electrical load in American households, the problem of disaggregating the remaining appliances is clearly simpler for UK households. Furthermore, in my experience, American households contain not only a wider range of loads (pool pumps, etc.) but also contain more duplicate appliances (2 or 3 fridge/freezers, etc.). This again contributes to a harder disaggregation problem for US households.

Split-phase power

American and British households also vary in the way that electricity is supplied to the properties. UK households typically receive single-phase (1 electricity input wire) power at 230 volts, while US households receive split-phase (2 electricity input wires) power at 120 volts. The case of split-phase power provides a convenient opportunity to install two current clamps instead of one. Through this small additional installation cost, the complexity of the disaggregation is more than halved, since most appliances are connected to only one of the input cables.

Smart meters

To the best of my knowledge, the capabilities of smart meters in both countries are yet to be finalised. However, the UK provides an interesting extension to the government mandated smart meter rollout, in which households will also receive an in-home display (IHD). IHDs will primarily provide a household's occupants with real-time information, such as their current power demand and cost of electricity. However, since these devices are likely to have access to electricity data at a higher granularity than is transmitted to the energy provider, they provide a convenient platform on which disaggregation can be performed. Furthermore, performing on higher granularity data within each household even circumvents any privacy concerns related to transmitting private data outside of the home. However, IHDs will clearly have limited resources in terms of processing power etc., which raises the interesting field of resource constrained energy disaggregation.

Postdoc position through Doctoral Prize award

Recently I've been thinking a lot about my career beyond my PhD, for which the funding runs out at the end of this year. Having experienced energy disaggregation from both sides of the academia/industry fence, I've learned a lot about both getting algorithms to scale to huge amounts of data and presenting my work at international venues. However, as I entered the final year of my PhD, I was enjoying my PhD so much that I decided to apply for the Doctoral Prize award; a grant for a one year postdoc position designed specifically to increase the impact of technology developed during a PhD.

Today I'm excited to announce that my application has been accepted, and assuming that all goes to plan while writing up my thesis, I'll start work as a postdoc in the new year. I'm hoping this position will give me the opportunity to put my PhD work into the hands of real users and show that energy disaggregation really can provide actionable suggestions and lead to real energy savings. I'm excited about the new challenges this will bring and the insight it will provide to this growing community.

Of course this also means that I'll continue to blog about energy disaggregation for at least another year.