Can Big Data Help Us Predict The Next Big (Snow) Storm? image 274621 l srgb s gl 0488 0275 rl 00 00I have a confession to make. The weather over the past few months has been infuriating me. I am tired of temperatures that are colder than my freezer. Tired of the snow and ice. And even though I love my kids, they really need to go back to school – now! At this rate, they will be in school until July 4th!

Even more frustrating is when I turn on the local news for the weather forecast. Words like “if”, “possibly”, and “perhaps” are peppered throughout their reports – even when the prediction is 48 hours in the future. And don’t even get me started when they say “the models don’t match up”. Ugh.

So I ask our beloved weather forecasters, “What is going on?” With all of the data available today, why aren’t forecasts more accurate? If a business analyst ever gave a report like these latest weather forecasts with mixed results, they would be fired. But here is one thing that separates weather forecasters from business analysts – you can never predict the unpredictable.

Like the climate, the science of the perfect weather forecast is changing

As a kid, I remember weather forecasters who were like rock stars. They gave a three-day report and a good long range was a forecast of five days out. And for the most part, you could rely on them.

Nowadays, the game is totally different. They are educated, certified meteorologists. Armed with five different model algorithms (Global, U.S., European, U.S. Navy, and Canadian) that are more powerful and up to date than ever before, they are pressured to give a six- or seven-day forecast (or even a seasonal projection) that is accurate.

According to a high-school friend of mine, Wyatt Everhart, Chief Meteorologist for WMAR-TV / ABC 2 News, “With all the data available, you can’t just look at one model. To get the big picture, you really need to look at all of them at the same time. The finer details of the forecast for the immediate area are created by combining that data with their experience with the region’s climate and weather patterns. And really, the data is good only 24 to 48 hours ahead of time. Anything more is an educated guess because weather and jet stream shifts can happen at a moment’s notice.”

“If you take a look at what organizations like National Oceanic and Atmospheric Administration (NOAA) are doing from a national level, the accuracy can be stunning and life-saving,” Everhart continues.

According to NOAA, the devastating impacts of extreme events – such as the 2011 tornadoes in Joplin, MO and Birmingham, AL – can be reduced through improved readiness. As a result, the “Weather-Ready Nation” initiative began in 2011. Over the past two years, NOAA has been doing more than issuing warnings – they’ve been learning how to get people to respond to those warnings. With the help of government officials and community groups, citizens are starting to take these warning seriously and heed the agency’s advice.

The initiative also gave NOAA an opportunity to strengthen its credibility to give accurate forecasts. For example, the implementation of supercomputers can process the five models three times faster and more accurately. NOAA researchers are also using new tools and accepting government grants to learn more about the weather with the intent on saving lives. To make that happen, they are working towards understanding the dramatic shift of the global climate and the changing behavior of weather events such as hurricanes and tornadoes.

Big Data can help us predict a lot of different things. However, there will always be something that cannot be accurately predicted in a long-range report. And right now, weather is one of them – at least until researchers can fully understand the changing global climate. Thanks to the latest innovations for in-memory computing and other sensory devices, researchers are close … they just had to wait for the right technology to come along.