Validating the Need to Validate Code

The article “Computational Science Demands a New Paradigm” by Douglass Post and Lawrence Votta ( Physics Today, January 2005, page 35 ) makes some good points about the pitfalls of believing computed physics. The authors propose several criteria for verification and validation of large computational models for making public-policy decisions. Among other examples, they mention climate-change calculations.

The article’s figure 5 shows several famous bridges, including as the third example the infamous “Galloping Gertie,” the Tacoma Narrows Bridge in Washington State, which collapsed in 1940. In that case, previous designs were pushed too far, with just one little thing forgotten: resonance! The film of that collapse is still often used in physics classes to dramatize the importance of resonance. Post and Votta “assert that computational science is currently in the midst of” the stage where computing power makes it possible to outrun good engineering judgment.

The Kyoto Protocol to curtail CO₂ emissions was based on a global circulation model from 1994, now a full decade old. It left out the importance of clouds, because they were just too difficult to model. That model was never even successful in accurately describing the past: It violates the first validation criterion given by Post and Votta. Nevertheless, international policymakers have not attended to validation criteria, have never doubted the truth of the model, and have gathered momentum toward implementing the Kyoto Protocol. Consequently, large economic impact is associated with those computational results.

The Galloping Gertie of environmental science is the Kyoto Protocol. Unless computational scientists learn from its shortcomings, it will discredit future attempts to predict climate change.