Opening the ‘Black Box’ of Simulations: Transparency of Simulation Models and Effective Results Reports Through the Systematic Design of Experiments
Hamburg University of Technology
affiliation not provided to SSRN
Technical University Hamburg-Harburg (TUHH)
Many still view simulation models as a black box. This paper argues that perceptions could change if the systematic design of experiments (DOE) for simulation research was fully realized. DOE can increase (1) the transparency of simulation model behavior and (2) the effectiveness of reporting simulation results. Based on DOE principles, we develop a systematic procedure to guide the analysis of simulation models as well as concrete templates for sharing the results. A simulation model investigating the performance of learning algorithms in an economic mechanism design context illustrates our suggestions. Overall, the proposed systematic procedure for applying DOE principles complements current initiatives for a more standardized simulation research process.