An item previously posted on this site described how uncertainty in the IJS calculation method can produce different results in the Men's event at 2007 U.S. Nationals. More on this subject can be found in a separate article about rounding in the scoring system.
This topic had generated a lot of discussion on the internet over who really won the Men's event. On January 31, USFSA announced they have no issues with the calculation of the scores or the application of the rules. Nevertheless, the question still remains, who should have won the Men's event?
In some respects the answer is, who knows!
The basic premise of IJS is that it will produce a point totals that represents the absolute accomplishments of the skaters, from which an order of finish can be determined. But that is a long way from the truth. The values of the elements are all subject to lively debate, as are the values that should be assigned to the GoEs. There are field of play decisions for the element calls that are subject to debate on the Technical Panels, who in once case at these Championships changed their mind long after a skater performed. There is a wide diversity of opinion for GoE values and Program Component values marked by the judges. What do you make of elements where a few judges mark an element at -1 and a few mark the same element +1?
The scores produced by IJS are only a vague approximation of what any performance might be worth in any absolute sense. When it comes to determining the winner in a closely contested pair-up of skaters IJS is woefully inadequate at determining a precise and accurate result for deciding which performance really was best.
So in any absolute sense, who knows who should have won the Men's event when it is this close.
In terms of crunching the numbers, however, the answer is clearer. Evan Lysacek won the Men's title. If we accept the scores as they are -- the honest opinions of a Technical Panel and a Judging Panel -- the marks received by Lysacek are slightly higher in the overall event than the marks received by Weir. If you average all the marks in a mathematically rigorous way by not rounding until the last step, Lysacek has the higher point total. If you do the same thing with a single trimmed mean, Lysacek has the higher point total. When you try different intermediate rounding schemes sometimes Lysacek wins, sometimes Weir wins, but all these rounding schemes are sloppy arithmetic and produce imprecise results. From a mathematical point of view, only the exact calculation can be relied on to give a correct tally of the points, and in an exact calculation Lysacek has the highest score.
There has been a great deal of discussion on the internet about whether certain rules may or may not have been followed correctly. Opinions vary on the clarity of the rules and how they should be interpreted. Lost in this discussion is the fact that USFSA rules as printed in the rulebook were followed in the largest sense. Further, the USFSA rulebook includes provisions for changing the rules and requirements for IJS during a season, to keep USFSA rules up to date with ISU rules. So if the ISU can document approval of the calculation method in its software, then there is no conflict with the printed rules, since the printed rules would automatically be superceded. Problem is, at this point no one at the ISU has been able to produce an approved document describing the calculation method currently coded in its software. Where is it? It is not in the ISU rulebook and it has yet to be located in any ISU Communication, or IJS document posted on the ISU website.
In any event, so far as USFSA rules are concerned, the official software from the ISU was used (as required by USFSA rules) in good faith, correct procedures were followed in operating the scoring system, no protest of the results was made within the permitted time, and the medals were awarded. Calculation errors are not correctable after medals are awarded, as is well known to all competitors by now, or should be. So those who would call for the results to be changed because one rule may not have been followed correctly are asking for other rules to be broken in its place. Currently, the rule that calculation errors are not correctable after the award of medals trumps all others. It is the final answer. It closes the book.
Perhaps that should not be the case, but it is the rule at this time. Those who feel calculation errors should be correctable at any time have the opportunity to take their zeal to the USFSA annual meeting in May, or the ISU Congress in June, but such a change would throw competitions into chaos. For now, however, the rule stands and so should the results.
As we will discuss in more detail elsewhere, in our opinion the real issues to come out of this event are that the rules covering rounding in the calculation method must be changed at the next ISU Congress, the software must be changed and thoroughly checked to insure an improved calculation method is correctly coded, and the archaic concept of tie breakers should be eliminated from the scoring.
But the result of the Men's event is fine just the way it is. The skater with the best marks was crowned champion.
Copyright 2008 by George S. Rossano