pearson.txt


This linear data set from K. Pearson (Philos. Mag., 2, 559, 1901) with weights from D. York (Can. J. Phys., 44, 1079, 1966) was discussed in the paper by M. Lybanon (Am. J. Phys., 52, 22, 1984) which contains the algorithm we use for uncertainties in X and Y.  The data is arranged as:

x, dx, y, dy


Using a weighted fit with uncertainties in Y only (ignoring the uncertainties in X), the resulting fit using Bevington or Numerical Recipes weighted fitting programs is:
y = a0 + a1*x

a0 = 6.100 +/- 0.205
a1 = -.6108 +/- 0.0300
Reduced Chi-Squared = 4.29

Using Lybanon's algorithm for uncertainties in X and Y gives:

y = a0 + a1*x

a0 = 5.471 +/- 0.295
a1 = -0.4805 +/- 0.0581
Reduced S-Factor = 0.742

In Lybanon's paper, he discusses that this is the exact generalized least-squares solution.

Monte Carlo analysis of this data set (10000 trials, varying both X and Y within Gaussian distributions for the error bars) is in very good agreement with the fit parameters and uncertainties listed above:

a0 = 5.486 +/- 0.396
a1 = -0.4837 +/- 0.0776

Programs that multiply the fit uncertainties by square-root of chi-squared (Such as earlier versions of Sigmaplot,
Psiplot, etc.) the resulting weighted fit with uncertainties in Y only is

y = a0 + a1*x

a0 = 6.100 +/- 0.424
a1 = -.6108 +/- 0.0623
Reduced Chi-Squared = 4.29

Note that the uncertainties in the fit parameters are scaled by about nearly exactly the square-root of chi-squared from the uncertainties from Bevington/Numerical Recipes.

More details of the incorrect uncertainty calculation in weighted least squares fitting programs can be found at

http://physics.gac.edu/~huber/fitting/

Thomas M. Huber; 20-July-2001