Europeans shine in weather forecasting

Although Hurricane Joaquin passed well off the US East Coast in October, the storm had a different kind of impact: bringing into focus the outstanding performance of one of numerous global weather prediction models. The European Centre for Medium-Range Weather Forecasts (ECMWF) was the world’s only major forecast simulation to pinpoint the hurricane’s track several days out. Others, including the US Global Forecast System (GFS), had charted a course for Joaquin that was hundreds of miles to the west and showed that it would make landfall along the mid-Atlantic coast.

The exceptional performance of the ECMWF model had also been seen in 2012 when it was the only one to predict the left hook that brought Hurricane Sandy ashore in New Jersey. Other models had Sandy staying well out to sea.

“The European center is the best, and the UK Met [Office] is second best,” says Cliff Mass, a meteorologist at the University of Washington. “GFS is third right now” in terms of skill, modelers’ term for accuracy. “In the technology, the UK Met and the European center are better models; they have better physics and better data assimilation,” he says.

Alan Thorpe, who stepped down as director general of the ECMWF in December, does not disagree. “It’s certainly true that we are recognized as the world leader in global numerical weather prediction out to a couple weeks ahead,” he says. “But one can oversimplify this, because that statement about who has the best predictions is the average over many forecasts.” Indeed, the ECMWF does make mistakes. Last winter New York City braced for a snowfall of more than two feet based on the European model’s prediction. As it happened, the worst of the storm passed to the east of the metropolis, taking a path that the GFS got right.

Data input varies

A critical part of the European center’s performance is its method for incorporating information from worldwide weather observations. “The ECMWF is recognized as being at the forefront of how to do that data assimilation to get the most from the observations,” Thorpe says. The center uses a mathematical method called four-dimensional variational data assimilation, or 4D-Var, in which the fourth dimension is time. “We take our observations not at just a single time but over a window of several hours,” Thorpe explains. “Information from prior short-range forecasts is merged with the observations.”

The Met Office also uses 4D-Var. But the GFS model, operated by the US National Weather Service’s (NWS) National Centers for Environmental Prediction (NCEP), uses a somewhat less advanced 3D assimilation system. “Implicitly, a 4D-Var is much more accurate than the 3D-Var method, but the problem is that it is 10 times more expensive to run,” says Hendrik Tolman, director of the NCEP’s Environmental Modeling Center. “You have to be able to run your model forward and backwards, and you need a separate version of the model to do that.”

The NCEP has been making what Tolman calls incremental improvements to the GFS, including a recent boost to its horizontal spatial resolution in forecasts out to 10 days ahead from its previous 27-km-scale global grid to a 13-km grid. That is slightly higher than the ECMWF’s 16 km. “We made [the GFS] better by going to the higher resolution, and we also spent quite a bit of effort to go to better ways of starting off the model with all of the observations we have,” he says.

The acquisition last year of two Cray supercomputers tripled the NCEP’s computing capacity and enabled the refined grid size. Congress provided funding for that upgrade in the wake of Sandy. A similar jump in computing performance expected next year will bring the NCEP’s computing assets into rough parity with those of the ECMWF. But both Tolman and Thorpe agree that an apples-to-apples comparison isn’t meaningful because the NCEP’s computers need to work on a lot more problems than just the medium-range global weather prediction that is the ECMWF’s mission.

“We have to be everything to everybody,” says Tolman. “To their credit, ECMWF have a laser focus. They are really good at that, but they can focus, and we haven’t been able to.” The NCEP generates seasonal predictions and produces 18-hour regional forecasts every hour. And with its increasing focus on severe weather prediction, the NWS soon is expected to begin issuing 2-hour forecasts every 15 minutes.

Following the leader

The NCEP is working to garner some of the benefits of 4D-Var operations through the use of ensemble forecasts. Instead of relying on a single high-resolution model run, an ensemble runs multiple iterations of the model at slightly lower resolutions, each starting from a slightly different set of initial conditions.

“From looking at how different the solutions are from these ensembles, we get an estimate of the certainty of the forecast and whether the weather is forecastable or not,” says Tolman. “It’s particularly important because we can use ensembles to do a hybrid ensemble-based estimate of a 4D approach rather than doing a full 4D as is traditionally done by running the model back and forth.”

The UK Met Office also uses ensembles in its data assimilation, explains its director of science, Andy Brown. “If all of the previous ensembles are predicting the same thing, the model is pretty sure about how and where this is going from the previous observations, and you can give quite a bit of weight to the model,” he says. “If, on the other hand, the ensembles were spreading out a lot and all giving a different answer, then in weighting a single model versus observations, you would give more weight to the observation and less to the model. The model is predicting its own level of confidence in itself.”

Thorpe says the European center pioneered the use of ensembles. It runs 50 iterations of its forecasts, compared with the GFS’s 20. Using ensembles, “we can not only say what is most likely but also what is the degree of spread and what degree of confidence we have in the prediction,” he says. “For a hurricane like Joaquin, what we’re after in the time frame of a week ahead is, what is the chance that a storm will either go over the Atlantic or the opposite. We’re looking at using the probabilities from our ensembles of those outcomes.”

Another differentiator for the ECMWF is the numerical method used for solving partial differential equations. “We use a particular method called a spectral method, and we find this to be advantageous and help with the quality of the forecast,” says Thorpe.

The final ingredient to the European center’s success, says Thorpe, is its method for representing the transfers of heat, energy, and momentum on the scales of motion of the atmosphere. Those transfers can’t yet be resolved in detail. “We’ve put a lot of effort into developing that [method] using the latest science of how the atmosphere works to really represent those transfers,” he explains.

Narrowing the gap

Since the GFS ensembles came into use two or three years ago, the gap in skill with the ECMWF has been narrowed by around 30%, Tolman says. But the ECMWF, which is funded by 34 European countries, including the UK, isn’t standing still. “We bring in a new version—what we call a model cycle, where we introduce improvements to the science—roughly once or twice a year,” says Thorpe. The new scientific knowledge comes from both internal and external research.

Ryan Maue, an atmospheric scientist at the US commercial weather service Weatherbell, says the ECMWF has plans to increase the resolution of the current model to the point that in just five years it will be generating four to eight times as much data as it is now.

One reason the NWS isn’t moving to a 4D system is a “legacy problem,” Maue says. Other NWS models used for regional, aviation, marine, and seasonal forecasts have been attached to the GFS over the years. Changing the global model, he says, “is difficult because you have all this overhang, and if you shock the system by changing the model, the entire supercomputing suite of products has to be adjusted.”

Tolman agrees. “Our model suite is considered somewhat ragtag, and way too complicated, and doing way too many things.” With help from non-NWS researchers, the NCEP is in the process of simplifying that suite to create models that cater to a broader set of users.

The Met Office’s global model is operated not as an end in itself but as an integral part of the office’s suite of products. The Met Office is unusual in covering periods ranging from short-term global and regional forecasts to 100-year climate predictions with the same modeling system, says Brown. “We get really big cost efficiencies and science advantages out of using the unified model. The physics of the atmosphere is the same whether you are doing a weather forecast or climate prediction.”

Opting not to develop their own global models, Australia, India, New Zealand, South Africa, and South Korea all contract to use the Met Office model. “The computer codes are 2 million lines. It takes a huge infrastructure to [run] this, and the problems are just getting harder and harder,” Brown says. For example, “10 to 20 years ago, it was ambitious to say there is likely to be a storm on the west coast in three days. Now the challenge is to be much more precise in terms of what’s the risk of a flood in this catchment area.”

Should the US lead?

Given that forecasters in the US and other nations have access to the ECMWF, does it matter who has the best model? The University of Washington’s Mass says yes: “The global model drives the regional models. Our regional models are driven by the US global model, not by the European model. So that inferiority projects down.”

Jason Samenow, a meteorologist at the Washington Post, consults models for the newspaper’s popular Capital Weather Gang blog . “I think the US should have the best model they are able to have,” he says. “Weather forecasts have a huge economic value, so the US would benefit from having a model which would be considered state of the art and best in the world.”

Samenow always looks at the ECMWF forecast first. “More often than not it’s better,” he says.

Mass sees an underlying problem. “The US developed numerical weather prediction. We have the largest research community in the world. We should overwhelmingly be the best. We’re not, and it has to do with the isolation of the Weather Service from the research community, a lack of leadership and vision, a willingness to be second rate,” he says.

Tolman says the insularity charge is unfair. “A lot of people in research don’t understand the business model of running operations. They assume that if they have a good idea, they can throw it over and we can put it in our models in a few months,” he says. “But we can’t just pick that up and put it into our observational models because we have to make sure that it works all the time, everywhere.”

Some say the superiority of the ECMWF is overblown. “It’s better, but not that much better,” says Maue. When averaged across the world, the European model is more accurate to one-quarter to one-half a day further out than the American model. In other words, the ECMWF’s 5.5-day forecast has typically the same skill as a GFS 5-day forecast, he says.

One thing everyone agrees on: All models have improved markedly. “No matter which center you are talking about, there’s been absolutely remarkable progress in the last decades. It’s a huge achievement of science that isn’t always recognized as such,” says Brown.

“We call this a quiet revolution, because … we don’t think the success of weather prediction has been heralded as much as some breakthroughs in other areas of physics and science in general,” says Thorpe. “It’s crept up on us over 30–40 years so that we are so much better at understanding the atmosphere and predicting the weather. It’s been a whole sequence of incremental scientific developments. It’s a different model to the groundbreaking, paradigm-shifting nature of how science evolves.”

There’s also a lot of sharing among the centers. “There’s a mixture of friendly rivalry between the centers and vast amounts of sharing information,” says Brown, who sits on the NCEP’s modeling advisory committee. “It’s a very internationally connected world.”