Validating the Need to Validate Code

We who work on the system side of high-performance computing development generally think our job is done when the first two challenges that Post and Votta mention, performance and programming, are addressed. I disagree that those two challenges are less urgent than the prediction challenge. I’ve heard too many complaints about the small percentage of peak that is reached and the dismal state of programming tools. However, from an application viewpoint, I can see that prediction is a formidable challenge.

The article reminds me of a paper I reviewed years ago for IEEE Computational Science and Engineering. The author compared several seismic processing packages, and each claimed to find oil in a different spot. Apparently, the results were often wrong. Nevertheless, users of the codes blindly trusted them and spent huge investments drilling for oil.

I wonder if the great importance of verification and validation could explain why some industries have not jumped more quickly into computational engineering. For example, one aircraft manufacturer is reportedly going back to using real-world wind tunnels for part of its development stage. It would be interesting to survey computational engineering companies about their experience with early computational technology.

One issue in successful validation is the availability of data for comparison. For example, to validate that an ocean simulator predicts correctly, one would need to place a huge number of sensors, which is probably impractical. So even if more project time is spent validating, I wonder how far the validation could go.