The "White Rat" of Numerical Reprodicibility

Name:    Prof. Michael Mascagni                 
Address: Department of Computer Science and
         Department of Mathematics and
	 Department of Scientific Computing and
         Graduate Program in Molecular Biophysics	
         Florida State University
         Tallahassee, FL  32306-4530  USA
         AND
         Information Technology Laboratory
         Applied and Computational Mathematics Division
         100 Bureau Drive M/S 8910
         National Institute of Standards and Technology (NIST)
         Gaithersburg, MD  20899-8910  USA

Offices: 498 Dirac Science Library/207A Love Building (FSU) Building 225/Room B154 (NIST)
Phone:   +1.850.644.3290 (FSU) +1.301.975.2051 (NIST)
FAX:     +1.850.644.0058
e-mail:  mascagni@fsu.edu (FSU) mascagni@nist.gov (NIST)

Title: The "White Rat" of Numerical Reproducibility

We explore an application from the author's work in neuroscience. A code used to investigate neural development modeled 100 neurons with all-to-all excitatory connectivity. We used a simple ordinary differential equation system to model each neuron, and this model was used to produce a paper published in the Journal of Neurophysiology. Later a colleague used our code to continue this work, and found he could not reproduce our results. This lead us to thoroughly investigate this code and we discovered that it offered many different ways to thwart reproducibility.

Numerical reproducibility is considered a task that directly follows from the determinism in computations. However, reproducibility has become an intense concern and and issue for research. In fact, the author developed an international workshop of numerical reproducibility that is now a regular offering at the annual Supercomputing XX conference. We will show how this particular code provides a lack of reproducibility from the following sources:

The non-associativity of floating-point operations in two ways
Differences in library mathematical functions whose reliability and correctness we take for granted

This code's sensitivity makes it a very powerful tool to explore many different manifestations of numerical reproducibility. However, this code is by no means exceptional, as in neuroscience these types of models are used extensively to gain insights on the functioning of the nervous system. In addition, these types of models are widely used in many other fields of study.

This is joint work with Prof. Wilfredo Blanco in CS at Universidade do Estado do Rio Grande do Norte in Brazil, and Woohyeong Kim, who is currently my graduate student.