What if new technology could take a single measure of household power use -- say, smart meter data, or measurements from a single home sensor -- and tell us exactly how much power our air conditioning, water heating, kitchen appliances, household lighting and all our consumer electronics were using?

That’s the goal of a host of “energy disaggregation” technologies, from startups like Bidgely, PlotWatt, Navetas and Energy Aware, original technology developers like Enetics, and big companies including Belkin and Intel, in various stages of development and deployment around the world. Their goal is to capture the value from being able to break out discrete energy loads within homes and businesses, or even diagnose inefficiencies or imminent equipment failures, through the magic of big data, algorithms and analytics.

But there are a lot of questions for these technologies, also known as “non-intrusive load monitoring,” or NILM. The first one is how accurate they can be. The second one is how accurate they need to be, and at what cost, to meet the various use cases being contemplated for them.

Last year, the Electric Power Research Institute (EPRI) set up a mock home in its Knoxville, Tenn. lab to see if it could answer some of these questions. The test pitted four separate NILM technologies -- two of them software-based, and two of them hardware-based -- against 57 separate real-world loads, from air conditioning down to cellphone charging, which EPRI monitored with submetering systems to get the real figures to use as a basis for comparison.

Last week, EPRI and the Department of Energy co-sponsored a workshop in Palo Alto to reveal some of the results. The presentation broke down each technology in terms of price over a five-year deployment, as well as accuracy in discerning and measuring loads, from easily recognizable HVAC and water heating systems, to harder-to-distinguish refrigerators, clothes washers, electric ranges or even household lights.

In simple terms, EPRI found “a cost versus accuracy tradeoff in some of [the] initial research,” Chris Holmes, senior project manager at EPRI, said at the workshop. Software-based systems that use available energy data from smart meters, circuit panel monitors or other sources were cheaper, but less discerning, while technologies that included specialized sensing hardware were far more accurate, but came at greater cost.

Balancing this cost-benefit equation will be an interesting challenge for companies interested in applying technologies like these, whether they’re companies trying to help property owners engage with and manage their energy use, or utilities that are hungry for data to help optimize load planning, energy pricing, and efficiency and demand response programs.  

While EPRI isn’t disclosing the names of the vendors and didn’t delve into the technical details of how they go about their energy disaggregation task, there’s still a wealth of data to draw from the test results. Here’s a snapshot of what was on offer.

The Software Route: Hands-Off Intelligence, Less Accuracy

The first two technologies EPRI tested fell into the category of software-based, meaning that they don’t rely on their own hardware to do their work. Instead, they compare home energy data from other sources against a range of existing libraries of data about typical home and building energy use and configurations, and use various algorithms to yield estimates of which systems are using which shares of power at different times.

While Holmes didn’t name the vendors, he did say that “a lot of what you see being offered to utilities in the cloud, software-based, are of this nature” -- a description that would appear to match the offerings from Bidgely and PlotWatt, two startups that have been landing partnerships with home energy management companies of late. It’s also useful for utilities that aren’t often able to install equipment inside customers’ homes.

Here are some results from “Product A,” the more accurate of the two software-based technologies EPRI tested. As the chart shows, it provided its results in daily and weekly increments, and was able to isolate only the four largest loads in the house: HVAC, water heater, refrigerator and clothes washer.

At first glance, this level of detail might not seem that impressive -- but compared to the existing tools available to the industry, it can still be quite useful. For example, EPRI conducted a test last year (PDF) to see how hourly smart meter data could be used to verify whether specific appliances had actually turned off after receiving a demand response signal. The results weren’t so great: 67 percent accuracy for HVAC loads, 58 percent for water heater load, 24 percent for pool pump load and only 18 percent for refrigerator loads.

Disaggregation technologies that could provide more accurate measurement and verification (M&V) could be quite valuable for utilities seeking more certainty from their demand response projects, Holmes noted. It could also help in designing better energy efficiency programs, building more accurate load profiles that guide utility asset planning processes, and integrating new end loads like electric vehicle chargers, which haven’t been around long enough to build the historical assessments that utilities rely on today to forecast future demand.

The Hardware Route: Greater Accuracy -- With Customer Buy-In

The other two test vendors are hardware-based -- that is, they design their own equipment that’s installed at the meter or in the home to analyze household electricity. Krish Gomatom, senior engineer at EPRI, separated these technologies into two categories: “waveform-based analysis,” which uses signal frequency as the key measure to differentiate one load from another, and “spectral signature analysis,” which relies on current and voltage readings and a mathematical procedure known as the Fourier transform, to get the job done.

Once again, EPRI didn’t disclose which two hardware vendors it worked with on its test. But some startups taking a hardware-based approach to energy disaggregation include Navetas, Verdigris, Enetics, Energy Aware, and LoadIQ.

This category also includes two deep-pocketed players. The first, consumer electronics giant Belkin, has taken disaggregation technology from Zensi, a startup it acquired in 2010, and applied it to the Echo electricity monitoring devices it started piloting earlier this year. The second, Intel, has a similar technology out of Intel Labs that is being piloted in the Austin, Texas-based Pecan Street smart grid project, though it has yet to announce any commercialization plans.

Of the two hardware-based products tested by EPRI, both were able to deliver data down to the fifteen-minute level, compared to the software products’ daily readouts. They were also about to capture more devices with more accuracy: the first, “Product C,” was able to deliver 83 percent to 87 percent accuracy on HVAC, water heater and pool pump, and 74 percent to 86 percent accuracy on refrigerator and clothes washers and dryers.

But “Product D” really stood out. As the chart below indicates, it was able to pick out ten distinct household loads, including lighting, which is often a stumbling block for technologies like these, as well as to deliver results in up to five-minute increments at a very high level of accuracy.

Product D was also quite a bit less expensive than the other hardware-based technology, and close in cost to the less-accurate software systems as measured over a five-year deployment. It’s important to note, however, that the cost calculations here are complicated, involving comparisons between software licensing fees and hardware installation costs, as well as variable costs based on the number of units deployed. 

Once again, it’s important to note that these systems aren’t accurate enough to stand in for revenue-grade submetering for any set of household loads. But they could help homeowners see their energy use patterns, identify wasteful behaviors, pinpoint which appliances are costing them the most, and other such tasks -- all without forcing them to install sensors at outlets, invest in smart appliances, or other expensive and time-consuming barriers to more active home energy management.

Next Steps in Bringing Disaggregation Into the Smart Energy Toolkit

EPRI’s second round of testing, scheduled for mid-2014, will add three to four new vendors to its roster, along with those from its first round that choose to continue to participate, Gomatom said. In the meantime, some real-world tests of NILM technology are already underway, according to Dave Kresta, emerging technology product manager for the Northwest Energy Efficiency Alliance.

His group and DOE’s Pacific Northwest National Laboratory (PNNL) are testing technology from five vendors -- Belkin, Enetics, Energy Aware, PlotWatt and Bidgely -- over the coming year, both in two PNNL test homes and in 30 real-world homes across the Pacific Northwest equipped with submetering equipment as part of a separate regional building assessment project.

Kresta noted that the 100 homes in this previous project cost about $30,000 apiece to hook up for the project -- some of it for hardware and telemetry, but most of it to pay the electricians who install and maintain it, as well as the PNNL scientists collecting, verifying and analyzing the data.

If that’s how much money it takes to hook up every load in a house with real-time metering, clearly it’s not an affordable route for homeowners or utilities -- which is why energy disaggregation is of such interest to utilities, he said. But that doesn’t mean that the NILM systems being tested are a piece of cake to implement -- several have been difficult to install, and some have overwhelmed the cellular network that’s carrying the data they collect back to the cloud, he said.

That’s part of the reason EPRI and DOE held last week’s workshop. The event brought many of the disaggregation technology vendors mentioned in this story into the same room with utility and regulator representatives, to start working on a set of standard definitions for how the accuracy of these technologies are measured, and how their data can be shared.

“We think this technology has the potential to be market-changing,” according to Cody Taylor of DOE’s Building Technologies Program. But that will require “common metrics that vendors can use, to tell how their product is better than some benchmarks – or better than competitors,” he said.

It may also require “data protocols and standardization that can allow common communication from hardware to software layers, common communication from event detection to customer notification,” he said. That may be harder to agree on, given the highly technical competitive differentiation between companies fighting for credibility in this new field.