It appears that 2014 is the year that the power grid’s physical security is going to come to the forefront of national attention. Now the question for utilities and regulators is whether they’re going to overreact, under-react, or find a balanced response.

It all started last month, when a Wall Street Journal article revealed previously unknown details on how an unknown assailant (or assailants) managed to disable a California substation last year by shooting out a series of high-voltage transformers with a rifle.

The attack on Pacific Gas & Electric’s Metcalf substation didn’t cause any blackouts. But it did lead former Federal Energy Regulatory Commission (FERC) chairman Jon Wellinghoff to label it the “largest incident of domestic terrorism” against the grid in U.S. history -- despite a lack of evidence regarding the identity or intent of the attacker, or attackers, in question.

That helped boost the story to headline status on cable news shows, as well as in Congress, where lawmakers moved quickly to demand that FERC, which has federal jurisdiction over the power grid, do something about it. Earlier this month, FERC issued an order (PDF) directing the North American Electric Reliability Corp. (NERC), the industry-funded nonprofit that manages grid security via its critical infrastructure protection guidelines, to identify the critical assets in the nation’s power grid and “develop, validate and implement” plans to protect them against attack.

Then, last week, Wall Street Journal reporter Rebecca Smith followed up with another article, citing a previously undisclosed federal analysis that indicated that similar attacks on as few as nine substations around the country could cause a nationwide blackout. That remote possibility would require a coordinated assault on unidentified critical targets, but could lead to extended blackouts, depending on the available supply of replacement high-voltage transformers, which are built by only a handful of manufacturers.

This most recent article has led grid authorities to attack the messenger. The Edison Electric Institute industry group has called for a federal investigation into Smith’s sources. FERC Acting Chairwoman Cheryl LaFleur said in a statement that the article “crosses the line from transparency to irresponsibility, and gives those who would do us harm a roadmap to achieve malicious designs.”

Meanwhile, FERC’s demand for new physical vulnerability protection standards has put NERC under the gun to deliver not only a list of the nation’s most critical grid sites, but also a plan to protect them -- all within 90 days. That’s an extremely tight deadline, as Tom Alrich, Honeywell’s energy sector security lead and a grid security blogger, pointed out in a recent blog post. Indeed, given the complexity of the task, it’s unlikely to be met, he wrote.

There’s also a danger of overreaction, as FERC Commissioner John Norris noted in a statement appended to the commission’s March 9 order. The rush to implement physical security rules could lead to “the electricity sector potentially spending billions of dollars erecting physical barriers,” while diverting attention from cybersecurity, natural disaster protection and other threats, he wrote. “We simply cannot erect enough barriers to protect North America’s [more than] 400,000 circuit miles of transmission, and 55,000 transmission substations.”

The Pros, Cons and Unknowns of Assault-Proofing the Grid

So what’s the measured course to take in response to these threats? Cybersecurity expert Andy Bochman said in a recent interview that a good first step is not to jump to conclusions.

“People use the terms ‘military-style’ and ‘terrorist-style’ assault, and these two terms immediately amp it up to the max,” he said. But commentators “use these words while admitting they don’t know who did it, or what their backgrounds were, or what their motivations were.”

Still, the debate does represent “an opportunity for everybody to revisit their current policies for physical security,” said Bochman, who advises organizations including The Chertoff Group, the security and risk management consultancy headed by former Department of Homeland Security head Michael Chertoff.

“Utilities across the country are reviewing their plans,” he said. “In some cases, they’re deciding to double down on their protective measures. That can include more surveillance cameras, more sensors. Sometimes it includes more security personnel at their substations -- you can’t put people at all of them, but you can decide which are most important and staff them appropriately.”

But less costly improvements can also help, he noted. One seemingly simple, yet potentially effective solution is to place an opaque screen in front of transformers and other vulnerable grid gear to shield them from view from outside the fences that surround substations. “It could be wood," he said. "It doesn’t have to be bullet-proof, it just has to be vision-proof,” to prevent snipers from having line of sight on the critical parts of the equipment they’re trying to disable.

It’s also important to remember that “the lion’s share of the solution here has to be preventative, versus reactive with assault rifles,” he noted. Utilities can’t post guards with orders to shoot at trespassers, after all. But they can do things like stockpile replacement transformers, and coordinate how they can share them with each other in response to emergencies, he said.

This is the same kind of preparation that can help utilities prepare for extreme weather events and other more predictable grid challenges, he noted. In that light, exercises like last fall’s GridEx II, which simulated a combined cyber attack and physical attack on grid assets across the country, help illustrate how important it is to maintain communications and coordination in the midst of extreme events.

That’s a work in progress, as the recently released GridEx II After-Action Report (PDF) indicates. That report on last year’s two-day exercise involving 115 utilities highlighted the need to share information early and often, and the need to “identify redundancies or alternatives to ensure viable communications channels in crises.”

Communications and Resiliency on the Grid Edge

Eric Byres, a security expert for industrial and grid communications and IT vendor Belden, noted in an interview last week that redundant communications are critical for defense. In the case of the Metcalf substation incident, the attacker cut fiber-optic cables adjacent to the site in an attempt to sever communications. But PG&E was still able to receive video and motion-sensor alerts from the site in minutes after the shooting started, according to the Wall Street Journal's February article -- an indication that communications weren’t completely disabled.

Even so, “one of the things they did wrong at Metcalf was camera placement,” Byres said -- the cameras were facing inward at the substation, not outward to view attackers. “If you don’t have visibility into what’s going on, you can’t defend yourself. […] As soon as we lose communications, we’ve lost the war -- whatever the war is.”

Belden has its own set of hardened, cyber-secure field communications and IT equipment to sell to utilities that want to beef up their substation perimeter defense, he added. But as for the relatively few sites that FERC and NERC may end up deeming most important, he said, “We have to make sure the ten to fifteen substations in America that are actually critical for the grid to operate are secured like Fort Knox” -- something that may be hard to do without giving away the location of those critical assets to would-be attackers.

Indeed, focusing too much on “guns, guards and gates augmented by cameras, dogs and drones” distracts from the challenge of making the grid itself more resilient to any single point of failure, Erich Gunther, chairman and CTO of grid technology consultancy EnerNex, pointed out in a March article for IEEE’s Smart Grid newsletter.

“By and large, we have designed a very resilient transmission grid with systems that are capable of automatically reacting to equipment damage no matter what its cause, isolating the damage and almost instantaneously routing power to end consumers from other sources,” he wrote. “This type of performance doesn’t happen by accident.”

At the same time, he cited his work with the team that provided electrical security to the Super Bowl this January as a potential guide to how distributed energy resiliency could help maintain broader grid security (as well as preventing a repeat of the 2013 Super Bowl blackout). The team for this year's Super Bowl prepared for “multiple element contingencies by verifying that automated systems are correctly configured, that multiple energy sources capable of supplying the load are available, and that physical assets are properly maintained and monitored,” he wrote.

“All of the actions we took to ensure a successful outcome of that event could be employed on the electric power system nationwide to further improve the reliability, resiliency and event response posture,” he wrote. Distributed energy resources, microgrids and other grid edge technologies could come into play in this conceptualization of a grid that’s both guarded like Fort Knox in the middle and capable of independence at the edges.

Tags: critical infrastructure, cybersecurity, ferc, grid edge, grid security, homeland security, nerc, pg&e, security, smart grid, substation, terrorism, wellinghoff