Comparison of public disaggregation data sets

Over the last year I've maintained a list of the public data sets which have been useful in my disaggregation research. However, I've found it's still quite time consuming to compare the finer details of the data sets, and I often end up trawling through papers or sifting through the data itself. For this reason, I've attempted to build a table which allows easy comparison between important attributes of each data set. As always, please leave a comment if you notice any errors!

Data set	Institution	Location	Duration	# houses	# sub-meters per house	Features	Resolution
REDD	MIT	Boston, MA, USA	3-19 days	6	9-24	V, P aggregate, P sub-metered	15 kHz aggregate, 3 second sub-metered
BLUED	CMU	Pittsburgh, PA, USA	8 days	1	0 (manual switch event labels available)	I, V	12 kHz
Smart*	UMass	Western Massachusetts, MA, USA	3 months	1	25 circuits, 29 appliance monitors	P, S circuits, P appliance monitors	1 second circuits, various appliance monitors
Tracebase	Darmstadt	Germany	N/A	N/A	N/A	P	1-10 second
Sample data set	Pecan Street	Austin, TX, USA	7 days	10	12	S	1 minute
IHEPCDS	EDF R&D	France	4 years	1	3	I, V, P, Q	1 minute
HES	UK DECC	UK	1 month - 1 year	250	13-51	P	2 minute
AMPds	Simon Fraser University	Greater Vancouver, BC, Canada	1 year	1	19	I, V, pf, F, P, Q, S	1 minute

N.B. I don't maintain this table of comparison, so please see this post for an up-to-date list!

AMPds data set released

Stephen Makonin recently released the first version of the Almanac of Minutely Power Data set. The data set contains 1 minute aggregate meter readings as well as sub-metered readings from 19 individual circuits. Each reading includes measurements of voltage, current, frequency, power factor, real power, reactive power and apparent power. Furthermore, the aggregate gas and water consumption was also measured at 1 minute intervals, in addition to 1 individual usage for each utility. The data set spans an entire year from April 2012 to March 2013 from a single household in the greater Vancouver area, BC, Canada. The data set is available to anyone for free, although the authors require a username and password to be requested for the purposes of usage tracking.

The authors of the data set have described collection process in more detail in the accompanying paper, as well as showing benchmark results of a method based on independent time slice combinatorial optimisation:

Stephen Makonin, Fred Popowich, Lyn Bartram, Bob Gill, and Ivan V. Bajic, AMPds: A Public Dataset for Load Disaggregation and Eco-Feedback Research, in Electrical Power and Energy Conference (EPEC), The Annual, pp. 1-6, 2013.

Appliances are not as smart as you think

Designing suitable energy disaggregation algorithms requires a fair amount of knowledge about household appliances. It's important to understand which features make it possible to differentiate between appliances, such as average power demand or duration of use. However, collecting suitably varied appliance data takes a lot of time and resources, while using existing data sets often neglects the study of how the appliances were actually used. This post collects a few examples of appliances which have lead me to conclude that many appliances are really not that smart.

Refrigerator

Originally, I expected a fridge's power demand to vary depending on its temperature set point, i.e. the cooler you set the temperature, the more power required to maintain that temperature. In fact, a fridge's cooling motor is only turned on or off according to a thermostat, and as a result always draws the same level of power while it is on, and therefore spends a larger proportion of each day cooling as opposed to idle.

Kettle

Similar to the fridge, I expected the power demand of a kettle to vary depending on the amount of water in the kettle. However, the power demand is always constant while it's boiling, and it's actually the time taken to boil the water which increases with the volume of water.

Oven

Following the same trend, electric ovens are also thermostatically controlled. As a result, they draw the same power irrelevant of the temperature set point. This means the heating element spends more time heating for higher temperatures, and leaves longer gaps between heating periods for lower temperatures.

Others

Spotted the trend yet? Other examples in this category include air conditioners, microwaves, electric hobs and irons. This is in addition the even dumber category of completely manually controlled appliances, such a lamp or fan.

Exceptions

Unfortunately, all appliances aren't this dumb. Some appliances have a power demand that slowly ramps up or down at the start or end of its usage. One appliance which is quite the opposite of those described above is the plasma television. These appliances draw a power demand which is proportional to the brightness of the screen, with a fully white screen requiring the maximum power draw and a black screen requiring the minimum power draw. Such appliances are not only hard to disaggregate, but also significantly complicate the process of disaggregating other simpler appliances.

Data set collected by UK EST, DECC and DEFRA

In 2012, the UK Energy Savings Trust, Department of Energy and Climate Change, and Department for Environment, Food and Rural Affairs published a 15 page report called Powering the Nation. This report summarises the full 600 page Household Electricity Use Study, which aimed to better understand how electricity is consumed in UK households. As part of this study, 251 owner-occupier households were monitored across England between April 2010 and April 2011. Of these households, 26 were monitored for 12 months, and 225 were monitored for 1 month. For each household, the energy consumption of 13-51 appliances was monitored at 2 minute intervals. A software portal is currently under development to provide access to the data set, although in the meantime the data can individually requested from ICF International by contacting efficient.products@icfi.com.