Problems with constrained curve-fitting
Background
We need to fit to states of the form
\sum_{n=1}^\infty Z_{n,src}Z_{n,snk} \left( e^{-E_n t} + e^{-E_n (T-t)} \right)
In the case of interest, we're forcing the Z_n to be positive, since src=snk.
I do this by having the free parameter be ln(z_n).
Also, we want the energy splittings to be positive.
This is done in a similar manner: the splitting E1-E0 is determined by a parameter ln(e1).
Finally, to help the computer with roundoff error, the functional form is switched to
\sum_{n=1}^{n_states} {\exp{z_n}}^2 2 \exp{-E_n T/2} \cosh{-E_n (T/2 - t)}
I vary n_states from 2 to 8, and look for a plateau.
Constraints
With many states, it's likely that the fit will wander off to some absurd minimum.
To guard against this, I add in some constraints.
I add in constraints following the Bayes method, where each parameter can augment the fit's total chi2 according to how near is to some prior guess.
To ensure that the priors don't adversely affect the fit results, the results are checked by doubling the prior widths and seeing that the quantities of interest are not affected.
Problems
A few problems have been seen:
Spurious states
As I have defined the fit function, it is possible for the coefficient to become very small, and the energy can therefore be any value.
Similarly, a large energy can have an arbitrarily large coefficient.
Consider, for example, a fit to the .01, .05 b1 (h_c) dataset with the 1S-1S smearing function.
Even if I use the same ensemble of configurations, and fix the priors, varying the starting guess is sufficient to cause a spurious state to appear in some cases.
A fit to n_states=2 gives two solutions:
| E0 | e1 | z0 | z1 | chi2
|
| 1.9793491048 | -0.95789844984 | 0.983532363989 | -0.635040000059 | 9.508
|
| 1.80199815995 | -1.63887834986 | -0.775848800302 | 0.98825580896 | 14.653
|
Here the prior choices were:
E0=2.0 +- .6
e1=-9.16291e-01 +- 4.81212e-01
z0=1 +- 1
z1=0 +- 2
It's possible, for this simple case, that simply repeating the fits with different starting values would eliminate the fits with the spurious state.
In more complicated cases, the solution is not so clear, since the solutions with larger chi2 don't correspond to the fits with the spurious state.
When we adjust the ensemble and prior central value with each bootstrap fit (but keep the starting guess fixed) we find a large tail of spurious states:
This tail is large enough that it causes the errorbar to be very asymmetric:
E0=1.979349 +0.004011 -0.013374
z0=0.983532 +0.011242 -0.060675
One possible solution is to tighten our prior widths to eliminate the tail.
I change the priors to:
E0 = 1.98 +- .02
e1 = -8.67501e-01 +- 4.59815e-01
z0 = .98 +- .1
z1 = 0 +- 2
Unfortunately this still gives asymmetric errors:
E0=1.979538 +0.003298 -0.009188
z0=0.984397 +0.009927 -0.037806
There isn't an obvious place to make a cut, as can be seen by sorting the bootstraps by E0:
or by ln(Z0):
Furthermore, it's questionable whether can justify reducing the prior widths so much, since such a choice comes after looking at fits to the data.
Choice of priors
One of the primary advantages of constrained curvefitting was supposed to be that it removes the subjective analysis required for the unconstrained fits.
We hoped to avoid having to tune parameters such as n_states, tmin, tmax, which datasets we include in the fit, etc.
Instead, we find that not only do we need to tune those parameters, but now we've got additional parameters to tune: the prior central values and widths.
Much of the problem comes from the way we calculate the bootstrap errors.
To do it properly, we have to vary the prior central values for each bootstrap sample.
If the prior width is large (as we'd like it to be to avoid influencing the results) then the central values will vary wildly, leading to many parameter choices that make it hard to obtain a good fit.
Data
It's not clear that we're even starting with data that is correct.
The b1 (h_c) data for the unquenched lattices has been particularly troublesome.
Meanwhile, old data for a quenched lattice (where we had true P-wave smearings) was relatively benign.
Similarly, the S-wave states are not causing nearly as much trouble as the P-wave states.