There’s a lot of argument in the smart grid industry about just how effective home energy management technologies are in actually getting homeowners to save energy. Now the American Council for an Energy-Efficient Economy has a new survey that shows that real-world results can vary widely, from up to 20 percent savings to nothing at all.

ACEEE surveyed nine pilots in the U.S., U.K. and Ireland, all using different combinations of technology and customer rate structures. The top-scoring project was in Northern Ireland and yielded an average 19.5 percent reduction by combining real-time power use feedback with customer prepayment plans -- an indication that people who’ve already paid for their power are likely to keep a closer eye on how they’re using it.

In contrast, two projects ACEEE surveyed actually yielded no average efficiency benefits at all, even though they also delivered real-time information to households.

One of those projects, carried out by Chicago utility Commonwealth Edison from 2009 to 2010, shows an interesting set of factors adding up to that poor overall figure, by the way. ComEd didn’t ask customers to sign up, or opt in to, the project. Instead, it automatically enrolled customers into a variety of different combinations of in-home technology and variable pricing plans, in a so-called “opt-out” method.

Among those that didn’t back out of the automatic enrollment process, a subset of customers enrolled in critical peak pricing programs did manage to drop energy use by an average of 21.8 percent during peak events, ACEEE found. But the majority of customers enrolled into the program appear not to have done much at all with their new energy data and restructured rate plans, leveling out the average savings to zero.

ACEEE’s findings reinforce the idea that “early adopters” may be the most valuable initial market for in-home energy technology. Across several pilots, the survey found a small percentage of households achieving savings of up to 25 percent. Identifying these “cyber-sensitives,” as ACEEE calls them, in advance of rolling out technology could be useful for utilities hoping to get the biggest bang for their buck in home energy management deployments.

But for utilities that are looking at ways to reach their customers en masse, the survey results were less than encouraging.  Across the nine projects, average savings added up to only about 3.8 percent, the survey found.

That’s lower than the 10 percent to 15 percent savings reported from past home energy management pilots, and much lower than the up to 33 percent drop that Silver Spring Networks, Energate and Oklahoma Gas & Electric found from their smart meter-smart thermostat summer peak price projects, for example.

But of course, the OG&E pilot, now being rolled out to 50,000 customers, used severe peak prices and automated thermostats to drive behavior change in customers who opted to sign up for the challenge. Earlier home energy management pilots engaged a small number of customers, who may well have felt inspired by their selection to interact more closely with their devices.

ACEEE’s survey did inspire commentary from Daniel Yates, the CEO of Opower, a startup that has pushed behavior modification, rather than in-home technology, as the key to getting lots of utility customers to save energy.

“Many companies focused on high-tech hardware and in-home devices,” Yates said in a Friday statement. “Yet those devices have proven to be complex, expensive and have achieved only modest results. At Opower, we instead decided to reach people through a channel to which they are accustomed: the mail.”

Indeed, Opower has yielded 2 percent to 4 percent energy reductions across millions of homes through its mail, email and text message alert systems, which apply behavioral science to drive homeowners to make simple and easy decisions to save energy. We’ve seen other home energy management providers like Tendril and EnergyHub embrace behavior science as a way to better engage customers.

At the same time, Opower is working with Honeywell to bring its behavior modification smarts to Honeywell’s two-way communicating home thermostats. That’s an indication of the fact that getting lots of people to turn down their power all at once -- say, to respond to peak demand events -- could require a more tech-centric approach.