Comparison of results produced by f-SOLVE and s-SOLVE
ABSTRACT
Differences in results produced by f-SOLVE and s-SOLVE are shown.
Possible reasons are discussed. General conclusion is that these
differences are small enough and they stem from the slight
discrepancies in parameterization schemes used in two versions of SOLVE.
Several different runs were made for comparison f-SOLVE and s-SOLVE
in summer 1998. Runs were made on computer aquila.gsfc.nasa.gov. All
available superfiles used in routine global solutions were used. Control files,
spool files, results of comparison and timing maybe found at
aquila.gsfc.nasa.gov in the directory /data4/super4/solve/solve_test (600Mb).
Summary of results of comparisons are discussed below.
The official version of software for parameters adjustment of VLBI
observation SOLVE used in Goddard Space Flight Center in MAY-AUG 1998 is
called s-SOLVE here. The new version of SOLVE maintained by Leonid Petrov in
Bonn is called f-SOLVE in the present report.
The objective of the comparison runs was to investigate the differences in
results obtained by f-SOLVE and s-SOLVE in order to make conclusion about
the presence of errors in f-SOLVE and possibility of usage of f-SOLVE in
routine data analysis.
- CNT_01
Objective of that run was to test f-SOLVE in one of the most heavy modes:
with estimation of positions of many sources for each day, with estimation
of positions of other sources as global parameters, with estimation of
station coordinates, axis offset and station velocities. Group delay and
rate observables were used. Many arcs are overparameterized. Test was
proposed by Chopo Ma.
Origin: glb1098a
Comments to control file:
arc sources from ICRF - group delays and rates
202 arcs with calc after time resort and other dbedit
axis offsets adjusted for fixed stations including VNDNBERG, YLOW7296
HRAS 2mo piecewise linear; CRIMEA, URUMQI correctly arc
phase delay sessions arc lines corrected
corrected equation of equinox patch (negative UT1 fix)
$96OCT30XA/B excluded - anomalous VLBA positions
by site weights with a posteriori sigmas for better convergence
no-net-translation and rotation TRF constraints:
uniform horizontal positions horizontal velocities
212 defining ICRF95 sources in weighted CRF constraint
no reference day
2808 arcs
| | Control files | Spool files |
| s-SOLVE | cnt_01s |
cnt_01s.spl |
| f-SOLVE (own mode) | cnt_01bb |
cnt_01f_bb.spl |
| f-SOLVE (compatibility mode) | cnt_01bc |
cnt_01f_bc.spl |
Results of comparison:
- cnt_01/01s_fbc Comparison between s-SOLVE and f-SOLVE in compatibility
mode. 43183 differences.
Summary (100 largest differences)
cnt_01/01s_fbc.s100
Comment: the utmost right column contains differences divided by
formal uncertainties, the next left column contains formal
uncertainties and the third right column contains the differences
itself.
Comment: the last 4 lines of that file are wrong: program sp_comp has a
bug in reading huge formal uncertainties of baseline clocks:
it cuts away the first leading digits. Actual formal errors were
447213.6, while sp_comp read 213.6 . Therefore normalized
differences were overestimated by 2000 times and we should put
aside the last 4 lines.
We see that the differences between cnt_01s and cnt_01bc does not exceed
0.01 sigmas and give rise primary due to the rounding errors in
printing results to the spool file.
S-SOLVE AND F-SOLVE in compatibility mode give identical results.
- cnt_01/01s_fbb Comparison between s-SOLVE and f-SOLVE in its own mode.
Summary (100 largest differences)
cnt_01/01s_fbb.s100
| 6055 out of 31176 | 19% |
[ 0.00, 0.01] sigma |
| 25759 out of 31176 | 83% |
[ 0.0, 0.1 ] sigma |
| 29634 out of 31176 | 95% |
[ 0.0, 0.2 ] sigma |
| 1442 out of 31176 | 4.6% |
[ 0.2, 0.5 ] sigma |
| 96 out of 31176 | 0.3% |
[ 0.5, 1.0 ] sigma |
| 5 out of 31176 | 0.02% |
[ 1.0, 2.4 ] sigma |
Only 5 differences in parameter adjustments exceed 1 sigma. The session
which produced the largest differences has three stations NRAO
(NRAO140, NRAO85_1, NRAO85_3) separated by about 1 mile but we estimated
atmosphere parameters every 20 minutes plus atmosphere gradients.
In addition positions of 22 out of 36 sources were estimated
as local parameters. Such a solution is not robust.
- CNT_05
Purpose of this test is to check how f-SOLVE handles adjustments of
parameters in special mode of user-partials. Test was proposed by John
Gipson.
*
*
* Test effect of estimating mapping function coefficients.
* 1.) Estimate DRY_A for NMF
*
* 202 arcs with calc after time resort and other dbedit
* axis offsets adjusted for best fixed stations including VNDNBERG
* HRAS 2mo piecewise linear; CRIMEA correctly arc
* phase delay sessions arc lines corrected
* corrected equation of equinox patch (negative UT1 fix)
* $96OCT30XA/B excluded - anomalous VLBA positions
* by site weights with a posteriori sigmas for better convergence
* no-net-translation and rotation TRF constraints:
* uniform horizontal positions horizontal velocities
* 212 defining ICRF95 sources in weighted CRF constraint
| | Control files | Spool files |
| s-SOLVE | cnt_05s |
cnt_05s.spl |
| s-SOLVE | cnt_05s2 |
cnt_05s2.spl |
| f-SOLVE (own mode) | cnt_05f |
cnt_05f.spl |
| f-SOLVE (FAST_MODE = B1B3D) | cnt_05ff |
cnt_05ff.spl |
| f-SOLVE (FAST_MODE = NONE ) | cnt_05fn |
cnt_05fn.spl |
Control files cnt_05s and cnt_05f specify the same parameterization but
s-SOLVE syntax of batopt language is used in the first file and cnt_05f
uses syntax of batopt language for f-SOLVE. The third control file cnt_05s2
is the same as cnt_05s except 5% difference in specification of the time
spans for atmosphere and clock segments: cnt_05s determines the length of
the time spans for clocks as 60 minutes and for atmosphere as 20 minutes
while cnt_05s2 determines them as 63 minutes for clocks and 21 minutes
for atmosphere.
Two runs of f-SOLVE were made: cnt_05ff and cnt_05fn. Fast algorithm B1B3D
was used used for the first run (FAST_MODE = B1B3D) and (slow) convention
algorithm was used in cnt_05fn (FAST_MODE = NONE). Parameterization was
exactly the same for both runs.
Results of comparison f-SOLVE against s-SOLVE :
- cnt_05/05_f_s_combined.sort -- comparison f-SOLVE versus s-SOLVE
of all parameters except user parameters;
- cnt_05/05_f_s.s100 -- 100 largest
differences f-SOLVE versus s-SOLVE except user parameters;
- cnt_05/05_f_s.user -- differences
in estimation of user parameters f-SOLVE minus s-SOLVE;
Maximal difference in adjustments of user parameters made by s-SOLVE and
f-SOLVE does not exceed 0.9 sigma.
Results of comparison f-SOLVE against f-SOLVE :
- cnt_05/05_ff_fn_combined.sort -- comparison of f-SOLVE used B1B3D
algorithm and f-SOLVE used conventional (slow) algorithm;
- cnt_05/05_ff_fn.s100 -- 100
largest differences f-SOLVE versus s-SOLVE except user
parameters;
12 bottom lines should be ignored due to the bug in sp_comp which cuts off
the leading digits in the adjustments of baseline clock offsets when their
values are huge. Thus the maximal difference is 0.16 sigma for clock
adjustments of clock offsets for the session $98FEB02XA. Nothing surprising:
clock offset for NYALES20 was 0.179 sec for that session and rounding
errors might cause even more substantial differences. New feature of
f-SOVLE: capasity to specify a priori clock model allows us to eliminate
errors od this sort. If to set aside this session then the next maximal
difference is clock offset for station NOTO in experiment 95DEC06XA --
0.03 sigma.
| 55127 out of 57461 | 95.9%
| < 0.0001 sigma |
| 1854 out of 57461 | 2.2%
| [ 0.0001, 0.001 ] sigma |
| 431 out of 57461 | 0.7%
| [ 0.001, 0.01 ] sigma |
| 49 out of 57461 | 0.09%
| [ 0.01, 0.16 ] sigma |
Results of comparison of two s-SOLVE runs with slightly (only at 5%)
different lengths of clock and atmosphere intervals:
- cnt_05/05_s_s2_combined.sort -- comparison s-SOLVE with 60/20 clocks
atmosphere intervals against 63/21 clocks/atmosphere intervals;
- cnt_05/05_s_s2.s100 -- 100 largest
differences cnt_05s against cnt_05s2 except user parameters;
- cnt_05/05_s_s2.user -- differences
in estimation of user parameters by s-SOLVE in solutions
cnt_05s and cnt_05s2.
| 14785 out of 70597 | 20.9%
| [ 0.00, 0.01] sigma |
| 54207 out of 70597 | 76.8%
| [ 0.0, 0.1 ] sigma |
| 63997 out of 70597 | 90.7%
| [ 0.0, 0.2 ] sigma |
| 5518 out of 70597 | 7.8%
| [ 0.2, 0.5 ] sigma |
| 907 out of 70597 | 1.3%
| [ 0.5, 1.0 ] sigma |
| 175 out of 70597 | 0.2%
| [ 1.0, 5.2 ] sigma |
Maximal difference in adjustments of user parameters made in two
s-SOLVE runs reaches 2.6 sigma.
- CNT_03
Objective of that run was to test estimation of high frequency EOP in
f-SOLVE. Test was proposed by John Gipson.
Comments in the control file.
****************************************************
*
* ID GLB1099eop -9800529 g, ITRF96all, HR2mo, axoff-, nUf, swghtn, TRF/CRF-NNTR, NMF20
*
* Special run with slightly changed time spans for intervals for
* clock and atmosphere
*
*
*
* This series of runs is to solve for Tidal EOP.
* Uses all geodetic databases in the can as of May 29, 1998.
* 1099EOPE -- solve for HF-EOP.
*
*
*
* 202 arcs with calc after time resort and other dbedit
* axis offsets adjusted for best fixed stations including VNDNBERG
* HRAS 2mo piecewise linear; CRIMEA correctly arc
* phase delay sessions arc lines corrected
* corrected equation of equinox patch (negative UT1 fix)
* $96OCT30XA/B excluded - anomalous VLBA positions
* by site weights with a posteriori sigmas for better convergence
* no-net-translation and rotation TRF constraints:
* uniform horizontal positions horizontal velocities
* 212 defining ICRF95 sources in weighted CRF constraint
* no reference day
* src9033, itrf96all starting points
* NUVEL-1A using 0.9562 scaling
* reference epoch site positions at 1997.0
* er1096b: -9712, HRAS 2mo, TRF/CRF NNTR, NMF20, -equ equinox, 212 source weighted constraint
* gradients .5 .5 at fixed sites including VNDNBERG, YLOW7296
* 3rd_Tide and consensus relativity
* weights by site at 7 deg with gradients, NMF20
* only most necessary site velocities tied
* Niell mapping and wet partial
* X,Y,U,Udot at middle
* 7 deg cutoff
* episodic motion: 871201 Yakataga, Sourdough, Whitehorse
* 891001 FORTORDS, PRESIDIO
* 901201 NRAO 85 3
* 920627 MOJAVE12, DSS15
* 961001 EFLSBERG
* JMG high freq EOP model hf1033cd - even more terms
* pressure loading pmvd97d tuned to 1067/a
* OCEAN LOADING APPLIED
| | Control files | Spool files |
| s-SOLVE | cnt_03s |
cnt_03s.spl |
| s-SOLVE | cnt_03s2 |
cnt_03s.spl |
| f-SOLVE (old version) | cnt_03f |
cnt_03f.spl |
| f-SOLVE (current version) | cnt_03f2 |
cnt_03f2.spl |
4 runs have been done. cnt_03s is a standard s-SOLVE run. Time span for
clocks was 60 minutes and time spans for atmosphere was 20 minutes.
cnt_03s2 had slightly changed time spans: 63 minutes for clocks and 21
minutes for atmosphere. cnt_03f run was made by old (JUN-98) version of
f-SOLVE. A strategy of assigning time epoch for segmented parameters was
slightly changed in f-SOLVE in August 1998 what reduced probability of
appearance large empty blocks. Run cnt_03f2 was made by the newest
(current) version of f-SOLVE.
Results of comparison:
- cnt_03/03_fs2.hfeop --
comparison of adjustments of high frequency EOP; The column marked
(1) there contains the differences in adjustments of hf-EOP between
the current version of f-SOLVE and s-SOLVE with the same time spans
divided by their formal uncertainties. The column marked (2)
contains the same parameters but for the differences f-SOLVE(old)
minus s-SOLVE. The last column marked by (3) contains the
differences between two s-SOLVE solutions .
- cnt_03/03s_s2_glo.s100 -- 100
largest differences in adjustments of global parameters in
two s-SOLVE solutions: cnt_03s and cnt_03s2. Unfortunately I didn't
have time to wait when s-SOLVE complete the process of estimation
of session-dependent parameters (it takes more than a week of
incessant computations!); thus the spool file of cnt_03s2 has
adjustments of only global parameters. But it is instructive to
see that adjustments of velocities produced by s-SOLVE
changed by more than 2 sigmas when we changed the length of time
span only by 5%. How reliable these adjustments and estimates of
their uncertainties are??
Maximal difference in adjustments of hf-EOP between f-SOLVE and s-SOLVE
does not exceed 0.3 sigma while it exceeds 1 sigma level if to compare
two s-SOLVE solutions made with 60 minutes clock time spans and 63
minute clock time spans.
Conclusion
- f-SOLVE in compatibility mode produces identical solutions as
s-SOLVE;
- No indications on hidden bugs in f-SOLVE have been discovered.
- All differences in solutions s-SOLVE minus f-SOLVE in its own mode
gives rise from the differences in forming and solving equations of
conditions by f-SOLVE.
- When equations of conditions were formed in f-SOLVE scheme but were
solved by s-SOLVE algorithms then the differences in adjustments
w.r.t.
solution produced by B3D algorithm where less than 0.1 sigma. It
allows us to conclude that the only source in discrepancies between
s-SOLVE solution and f-SOLVE solution (in its own mode) is the
difference in forming equations of conditions made by f-SOLVE and
s-SOLVE. These differences are well controlled and
documented.
It wouldn't be amiss to enumerate the most important differences
here once more:
- Time epochs for all stations are exactly the same for
f-SOLVE. s-SOLVE may set slightly different time epochs
for the different stations.
- Initial epoch for segmented parameters (clock, atmosphere
and EOP when they are modeled by linear spline) is set to
the moment of the first observation by f-SOLVE while it may
be shifted in 1 minute in s-SOLVE.
- Final time epochs are set within 1 minute the last
observation at that station in s-SOLVE. More sophisticated
algorithm for adjustment of the final epoch is used in
f-SOLVE. Look in
Release of f-SOLVE issued 23-SEP-98. for details.
- Treatment of clock breaks is quite different in f-SOLVE and
s-SOLVE.
f-SOLVE models clocks as a sum of two piece-wise function:
discontinuous global polynomial of the first or the second
order with breaks in clock break points (if they exist) and
continuous linear function (linear spline) with uniform time
interval. f-SOLVE prints the differences in clock offsets,
clock rates occurred due to the clock break under
consideration in the spool file.
s-SOLVE models clocks as a sum of two global polynomial of the
first, second or third order and a continuous linear function
(linear spline). Additional clock epoch is set in 0.01 minute
just before clock break and in 0.01 minute just after clock
break. Thus, linear spline is re-setup if clock breaks occur.
s-SOLVE prints values of clock offsets and clock rates at the
moments within 1 minute of the first observation and clock
break(s) in the spool file.
Test showed that both scheme provide almost equivalent
results.
- f-SOLVE in its own mode and s-SOLVE yield very similar but not
exactly the same results.
The differences do not exceed 1 sigma
and they are below 0.2 sigma level for 95% of
adjustments. The differences in results stem from the subtle
differences in parameterization. Two s-SOLVE solutions in which epoch
intervals for clock and atmosphere differs at only 5% produces
larger differences in the results. It indicates that choice of 60
minute clock intervals and 20 minutes atmosphere intervals which is
currently used in Goddard for global runs results in unstable
solutions for some sessions.
Back to f-SOLVE home page
This memo was prepared by
Leonid Petrov
Last update: 09-NOV-98 12:06:47