PNNL to use data-driven protocol for NILM vendor evaluation

PNNL recently held their fourth NILM Protocol Development Advisory Group conference call, in which it was decided that they would use a data driven protocol to evaluate the accuracy of NILM products. To give a bit of background to this decision, the choice was between the following two options:

Data driven - use (potentially existing) data collected from real homes to evaluate the accuracy of NILM products

Lab test - collect new data from an artificial lab home, in which the schedule of appliances is programmed rather than operated by humans

In my opinion, this is definitely the right decision given the diversity of loads and schedules of use in real homes. Although theoretically this is possible to simulate in an artificial lab home, in practice I would still be concerned that some reality gap might exist between the data collected from lab homes and real homes. However, monitoring real homes is more difficult than lab homes given the inherent intrusion into people's homes, and clearly a careful approach to data collection will be required to ensure the integrity and usefulness of the resulting data set.

A summary of the meeting is available via the advisory group's Conduit community.