Utilities Dumbstruck By Big Data From Smarter Grid

First of Two Parts
As the electric grid gets smarter, vast quantities of data are arriving at utility companies that have no idea what to do with them, according to electric industry experts who gathered in Chicago this morning.
Electric utilities already possessed 194 petabytes of data by 2009, according to one estimate (the entire collection of the Library of Congress is believed to amount to about 3 petabytes), and every day more terabytes are showing up at utility company data centers nationwide.
For example, smart grid investment grants have driven the installation of phasor monitoring units (PMUs) that collect voltage, current and digital status measurements as often as 30 times per second at many points along the grid. The data is supposed to help utilities manage the grid better, especially as renewable sources contribute more power.
"The amount of data this is generating is phenomenal. It's generating terabytes of data," said Paul Myrda of the industry-sponsored Electric Power Research Institute, during a panel at the Great Lakes Symposium on Smart Grid and the New Energy Economy, which concluded today at the Illinois Institute of Technology.
"There's a lot of data there; what do you do with it? How do you take advantage of it? How do you manage it? At what point does it become not useful?"
EPRI recently surveyed utilities to find out what they're doing with the data.
"By and large right now, most of the utilities that have access to this information are acquiring it, they're keeping it forever, few of them have any idea of how they're going to archive it or move it offline. Any decimation of the data is unclear at this point," Myrda said.
"The other problem is that they do not have an easy way to link the data to what the system state was at any point in time, so you have voltage and current measurements without the typology associated with it. What good is it after a day, two days, a week, a year?"
Myrda has been an advocate for the installation of PMUs, he said, and he has followed the development of software that utilizes the data. While some applications have been developed, he said, "there's no killer app."
"These are real problems. There's real issues, and what I've just articulated is one dimension of the problem around big data and utilities."
Another dimension involves information technology. Most utilities have equipped themselves with state-of-the-art computers, databases and networks—today's best-practices in information-technology architecture, said Dan Rosanova, a senior technology architect with the consulting firm West Monroe Partners.
Today's IT architecture works well for today's common uses, which often involve accessing processors and data over networks to solve particular problems.
But the kind of big data utilities are collecting now would overwhelm today's networks, Rosanova said. For example, a single calculation might require access to 4 terabytes of data—more information than the Library of Congress holds, across a network.
"This is where the centralized model, which serves us well for most of what we're doing now, falls apart when you get into big data. And that's with 4 terabytes. Wait until you get to these petabyte scales that we were talking about earlier."
So architects like Rosanova have to plan a new kind of IT infrastructure, but they still don't know exactly what demands it will have to meet:
"We don't know how we're going to use all of this data yet," he said. "And a lot of the tools we're going to use probably haven't been invented yet, to be quite frank. The reason big data is such a big challenge, is not only do we not know what tools we're going to use, but the future demand on storage capabilities is pretty unclear at this point. We just know that they're big."
"This is a bigger challenge than most people are prepared for at this time."
Myrda believes the challenge represents opportunity for engineers to find applications for the data and for large utilities to collaborate.
"Maybe some engineers could make a buck or two by selling their app," he said. "This is a place where we can leverage the industry at large and some of the most innovative talent that's out there."