General Motors

Though inherently statistical, model evaluation lacks a unifying statistical framework. NISS was hired to help find a overlying system to help with model evaluation. The research team used Bayesian techniques to measure the degree to which a model captures the underlying reality; theory and methods that allowed dual use of data in both estimation of model inputs and evaluation of outputs. SFCME also involved selection of evaluation functions by which a model and reality are compared. It also looked at design for determining what field or computer simulation data to collect.

NISS defined and developed a strategy for the evaluation of GM computer models, in cooperation with GM scientists, and implemented the strategy on test bed problems. Central research issues included the association of confidence limits to predictions of computer models; uncertainty estimates for predictions "beyond the data;" either in the sense of predicting over a new range of inputs or predicting with a variant of already studied models; and determination of sensitivities in model components.