Validating the Need to Validate Code

I am also concerned with the issues raised by Douglass Post and Lawrence Votta. A great deal of computational physics involves fitting parameters—for example, some turbulence constants and grid design parameters. Fifty physics models put together, each with a couple of free parameters, could yield 100 parameters that can be used to fit the code to whatever verification and validation tests it needs to pass. Yet we know that an interpolation function fitted to a bunch of points can be wildly wrong between them. This concerns me, before we even start extrapolating the code to regions where its performance is completely untested.

The problem the article highlighted is serious. People who know the issues involved in computational physics are essential. Unfortunately, these days universities turn out users who employ codes as black boxes but do not understand what they do or when their results can be trusted. Moreover, analysis codes are often incorporated into multidisciplinary design optimization algorithms—for example, to design a better aircraft—but the optimization process drives the codes beyond any reasonable applicability. Expert guidance is usually needed to stay within implicit constraints of analysis codes.

Rigorous component testing, although necessary, is not sufficient. Software components can be combined, but their combination could be wrong even though the components test well individually. A combination that is insensitive to minor component errors could still give invalid results. Each component has an unstated region of applicability that is often horribly complicated to describe, yet the combination could unexpectedly exceed individual component limits.