Greentech Media’s Soft Grid 2014 conference is just around the corner -- and that makes it a good time to examine the progress over the past year among utilities making the transition from traditional IT and operations to a “software-defined” future. That includes a hefty amount of silo-breaking in the world of data integration, and some early steps into real-time, cloud-connected IT-OT convergence.  

These are some of the findings that Ben Kellison, GTM Research’s director of grid research, has collected in this handy primer on the key trends in the soft grid over the past year. Some of these transitions are moving along more smoothly -- consider the growing number of utilities bringing data from disparate and often isolated business units into use for enterprise analytics. Others, like the integration of grid operational data for real-time analytics and controls, or the shift to a cloud-based data sharing architecture, are still in their early stages.

Here’s our breakdown, along with some real-world examples to flesh out the broader trends. Let’s start with the fundamental challenge facing utilities: creating a single IT enterprise out of what have traditionally been separate business units with little or no contact or integration.

As Kellison notes above, not all utility silos are meant to be broken -- for example, deregulated utilities cannot share data between their transmission and distribution operations and their energy trading or retail energy units. But there are plenty of areas where crossover between silos is creating value that couldn’t be realized without it.

Examples include using SCADA data to inform asset management systems, or tying advanced metering infrastructure (AMI) data to distribution grid monitoring or automation. Customer data is a huge realm of interest for utilities, judging from GTM Research analysis on the subject -- and the wider world of distributed energy resources is going to make customers even more important in years to come. 

One big question on the data integration front is whether it will come all at once or piece by piece. On a grand scale, companies like C3 Energy are working with utilities like Baltimore Gas & Electric on enterprise-wide data collection and analysis, though these massive projects don’t come cheap. Smaller-scale, “a la carte” data analytics also have their role, particularly to serve smaller utilities that can’t afford their own cutting-edge IT departments.

As for what utilities are doing with the data they’re integrating, this second slide highlights an important new development of the past year -- the move toward distributed computing for managing devices on the edge of the grid.

Kellison notes how concepts like real-time, autonomous federated decision-making amongst disparate devices (solar PV inverters, distributed batteries or plug-in electric vehicles, grid voltage management systems, or demand-side energy management and controls) are starting to become a reality on the grid edge.

We’ve covered multiple efforts underway on this front, including cutting-edge work being done by the likes of San Diego Gas & Electric, Ontario Hydro, Germany’s RWE, and Duke Energy’s “coalition of the willing” projects. Similar work is underway among smart-grid networking providers like Silver Spring Networks, Cisco and Itron.

It’s important to note, however, that these systems have a ways to go before being applied in the field. While smart meters, grid sensors and grid-edge devices may be delivering data in something approximating real time, making use of that data in the moment is a tougher task.

As Kellison notes, the analytics use cases associated with smart meter data today -- theft detection, asset management, load forecasting and customer engagement insight -- are not real-time applications. Utilities will have to move beyond traditional batch data processing and toward more complex event processing if they’re to gain situational awareness into grid operations -- and real-time machine-to-machine data processing and analytics are still in the future.

The advanced distribution management system platforms from grid giants like ABB/Ventyx, Schneider/Telvent, General Electric, Alstom and Oracle can be the basis for this kind of distributed intelligence -- but as GTM Research notes, these are expensive and complex systems limited to the largest utilities. If AMI vendors, distribution automation providers, and other smart grid technology providers can build these kinds of distributed intelligence capabilities into their own platforms, that could provide the foundation for a new evolution of software to make use of them.

As for the much-ballyhooed but slow-moving transition to cloud computing, Kellison notes that accessibility of data is a key driver. That starts with an imperative to share data with a wide variety of users within the utility, but could be expanded to include utility customers, which are being promised free access to all sorts of new data, and to government regulators intent on creating metrics for new regulatory regimes that take disruptive influences on the grid edge into account, like those being contemplated in California and New York.

Let’s end the discussion with a reality check on the business potential of analytics for utilities, as expressed in the amount of venture capital investment going into the sector over the past three years.

GTM Research predicts the global utility data analytics market will grow from $1.1 billion last year to nearly $4 billion by decade’s end, for a cumulative $20 billion between 2013 and 2020. But as this chart makes clear, the slow-moving, quasi-monopolistic utility sector isn’t exactly the hottest target for VCs looking to invest in data analytics.

Indeed, the one outlier in this chart, the $137 million investment in the second quarter of 2013, was driven by a $105 million investment by GE into Pivotal -- a data analytics platform with much more than the electric grid on its list of target industries. There’s plenty of reason to expect that innovations from outside the traditional utility sphere will have an impact on the soft grid’s growth.