What is big data and why is it important?
by Ethan Goldman, Energy Informatics Architect
Modern technology has generated enormous amounts of data – everything from purchasing patterns online, to crime rates in cities, or consumption habits at grocery stores. Experts from a range of sectors are now beginning to examine all this new information to explain trends and inform decisions. In other words, the phenomenon of "Big Data" gives us a detailed picture of what is and allows us explore the potential of what could be, in ways that were never possible before.
While the newly-installed advanced meters in Vermont will provide as many as 3,000 data points per month, this is dwarfed by the volume of data produced by video uploads, online ad click and purchase records, even flight data recorders. Still, analyzing large energy data sets is already revealing valuable insights, and the flow of data is still increasing.
Giving perspective and providing insight
As I've said before, information is power – and I believe that analyzing energy data can provide a perspective that could revolutionize the way we view and consume energy.
Here at Efficiency Vermont, we are working to provide our customers with the information they want, in a context that makes sense. By enabling Vermonters to fully understand the impact of energy usage and energy efficiency, families and communities are empowered to meet the economic and environmental goals they set themselves. One way we’ve been harnessing and presenting data is through our interactive map, where we’ve transformed residential energy savings into a visualization that enables people to learn more about the benefits of efficiency.
How else are people using Big Data? Here are a few examples of how numbers are already having an impact.
Big Data by block
It’s one thing to help people understand how much money they can save through energy efficiency, but in order make change possible, they must first step back and see how much they are actually consuming. That’s why the Columbia Engineering School conducted a study to show block-by-block energy usage in NYC. Now, urban planners, homeowners and policy makers can understand exactly how energy is used and figure out the most effective ways to seek improvements.
Walking all over Big Data
Ever wonder how walkable your neighborhood is? Folks have found a way to calculate how easy it is to run (or walk) your daily errands through data from geographic locations of restaurants, coffee shops, schools and more. For those who are looking for a new place to live – or try your current address – you can see how long it takes to bike, walk, or take the bus for those daily trips.
Big Data in the public domain
In Gainesville, Florida, folks are being encouraged to focus on energy efficiency through competition and comparison. Due to local regulation, monthly energy use data from utility bills are posted by address so occupants and neighbors can see how their home compares to the houses down the block.
This approach makes it easier for individuals to access their data and could allow perspective buyers and renters to make more informed decisions. I’ve often noted the importance of providing context when presenting data, and here it could motivate some neighborly competition. However, those potential benefits must also be weighed against the cost to individual privacy. Often, data can just as useful when aggregated or anonymized. At Efficiency Vermont, we would never jeopardize your identity when evaluating and presenting data.
Looking into the future
How can we use Big Data to improve our energy efficiency? In Vermont we are already starting to see a volume of energy data from the new, advanced metering infrastructure that is now in place across the state. The velocity and variety will soon increase as smart appliances and sub-metering become pervasive, but more on that in a future post. This will enable businesses and residents to track their own energy usage in detail – understanding exactly how much they are using, and how much it costs. If they want to take this information to the next level, they can check in with Efficiency Vermont figure out what it all means and analyze their options for decreasing their energy usage and costs. It’s a great opportunity to move our state towards a more efficient future, but seizing it will require us to think and act differently than we are used to about our energy system.
What do you think about how might Big Data help us reevaluate our energy consumption? Let us know with your comments and feedback – and I’ll plan to keep you in loop in future posts.