Research Project
Though inherently statistical, model evaluation lacks a unifying statistical framework. NISS was hired to help find a overlying system to help with model evaluation. The research team used Bayesian techniques to measure the degree to which a model captures the underlying reality; theory and methods that allowed dual use of data in both estimation of model inputs and evaluation of outputs. SFCME also involved selection of evaluation functions by which a model and reality are compared. It also looked at design for determining what field or computer simulation data to collect. NISS helped formulate test bed examples (initially, subsurface fluid flow models and traffic simulators). These motivated the formulations and were testing grounds for new theory and methodology. A Virtual Laboratory for Model Evaluation was established to disseminate results, broaden involvement of other researchers (and users), experiment with visualization and other methods in evaluations and created a unique educational and training environment.
This research consisted of two complementary projects. Evaluation of the fidelity of computer models to reality is central to assessing their effectiveness in understanding real phenomena and predicting results of innovative policies.
Names of two projects:
- Statistical Framework for Evaluation of Complex Computer Models (SFCME)
- Mathematically/Statistically Based Validation Systems
Technical Report(s):
Technical Report 128: A Framework for Validation of Computer Models, October 2002
Technical Report 162: A Framework for Validation of Computer Models, April 2005
Technical Report 163: Bayesian Validation of a Computer Model for Vehicle Crashworthiness
Technical Report 165: Computer Model Validation with Functional Output
Does the computer model accurately represent reality? NISS was tasked to find a overlying system to help with model evaluation.
Principal Investigator(s): Jerome Sacks, NISS;
Senior Investigator(s): Jim Berger, Duke; M.J. Bayarri, Valencia; J. Cafeo, General Motors; F. Liu, Castilla-La Mancha; C.H. Lin, General Motors; J. Tu, General Motors
Post Doctoral Fellow(s): Jesus Palomo, Rui Paulo, Danny Walsh