Not Rated in Sequence

Running Chess Tournaments
1

I was just wondering what happen to the rating of players when their game are not rated in sequence.

Say… A player played on a tournament in July 23rd, the MSA shows the tourney as being received but for some reason or another it is not rated yet.

Then the player played another tournament on July 31st, it is now rated.

If player played very well on July 23rd but played so lousy on July 31 or vice versa, which increases/decreases his/her rating.

Does the rating system compute the July 23 game without accounting to the July 31? Or does it compute the new rating using the current (after July 31 rating)?

2

In the current ratings programming, events in a batch being rated are sorted into chronological order based on the event ID (which incorporates the event ending date) and the section number.

However, when an event is rated, the post-event rating from the most recently rated event, regardless of the ending date of the event, is the pre-event rating for each player.

Thus, if the earliest event for a player in the current ratings batch ended before the latest event in the previous batch, it will use the post-event rating of that event, even though it was played afterwards.

According to the ratings committee, the effect of rating events out of order are minimal and should be worked out in 2 or 3 events.

That’s little comfort when trying to explain things to a USCF member or that member’s parents.

One of the facilities being built into the new ratings programming is the ability to re-rate events in bulk to place them into true chronological order.

The plan is to re-rate events ahead of each rating supplement to put them into true chronological order.

There are those who feel that this may actually raise more questions than it resolves. It will mean that your CURRENT rating is subject to change if events for you OR ANY OF YOUR RECENT OPPONENTS have to be re-rated because of chronologcal resequencing, even if you haven’t played any additional games.

I think more people may complain about a lowering of their current rating than if it goes up, but we could get complaints about either effect.

Thus, whether we actually use that facility or not will be something more driven by politics and public opinion than by mathematics. :slight_smile:

3

With the upcoming online rating submission option, this effect (rated out of chronological order) should be minimized, as the players will be rated immediately - assuming of course that each TD submits the reports in a timely manner. Perhaps USCF should look into requiring reports to be submitted within 2-3 days instead of the current 7 day period? I think this is also a contributing problem to events not being rated by ending date.

Although certainly, having a rating which correctly reflects the most recent event is desirable, I’m not sure the idea of re-rating an event is a good one for the reasons you mentioned, Mike. Players may have a harder time accepting the idea that their current listed rating may change.

Currently, my opinion would be to let it remain as is (no re-rating). The ratings may still be mathmatically incorrect, but I think the players will have more complaints otherwise.

4

There are good and bad with re-rating tournaments, just going up or down with re-rating with the difference within the same rating class has little difference. If this person on one rating breaks over 1601, would give the player a rating floor of 1400; if on the other hand the person is re-rated and only has a rating of 1599, then this person would have a rating floor of 100. Then again it can work both sides with re-rating a tournament, just for the players that can go up or stay at the same rating floor. Having a player at 1601 or 1599 is not much of a rating problem, as it will take more then a tournament to break over the 1600 mark or a few tournaments to fall below 1400 as a rating.

If Norm is thinking of changing the ratings with re-rating tournaments that have gone past. Would like to point as a example, as the rating department lost some of my tournaments in the past. Did sent them in for the re-submit, with it being rated after a number of months. With myself being so active with 178 games since the August 2004 suppliment, some how re-rating my tournaments and games with over 100 rated games after the problem, then their games also: would make the change of one rating become a all day job.

5

Somehow, I have a gut feeling that this is one change which, like the 5-second delay, may go down smoothly even if it is rammed down people’s throats.

It is probably not too early, however, to begin educating players with the notion that post-tournament ratings on crosstables are only tentative, and may change by a few points by the time they become official, in the next rating supplement.

I suppose ratings could even change after that, if there is an error of several hundred points (due to missing ID and similar name, for example) in a player’s rating which would change all his opponents’ ratings significantly.

Bill Smythe

6

The problem with the change in ratings after they have been posted, it becomes a cascade effect if the point is to change one rating. It would cause grave problems if the rating department change any rating, as how can the rating department know all the rating errors. Looking at my rating, can show rating errors going back 200 and 400 games back, fixing one problem only change the ratings of everyone that has been on the other side of the board. If they get a different rating, then everyone they have been with there ratings will also be changed. The rating department can hold re-rating all the players to stop the cascade effect just to change one persons rating – then the probelm would be rating inflation or rating deflation.

If the rating department takes care of the cascade effect, if they change the rating of one tournament: then any games that one person plays would need to change the ratings in all the tournament(s) that person did play in. Everyone that person did play with, their rating also will have to change with every person they also did play with. If going back to change the rating errors on my rating, as myself being so active, would have to change the ratings of 150 to 250 players. If two players that never had a rating error that did play, only having in the past been in a game with someone that has a rating error. If fixing whites cascade effect, then later fix blacks cascade effect they and anyone after would have their rating change twice.

Changing the errors in ratings would fall into ‘temperal logic’, example: if going in a time machine just to go back two weeks just to stop a person from being killed, then going back to the present would not effect the present with major change. It will change the time line for decades or centuries to come, if that person that was not killed did produce offspring or change on there own other peoples lives. If on the other hand, going back in time to stop the birth of Jesus, then the history of the world would be changed if and when you come back to the present.

The best information to give too the federation, is leave the rating errors alone. As the cascade effect would leave the idea of what a rating is or is not in a total mess.

7

I see re-rating as a way to get closer to the ideal situation, which is a world in which all TD’s submit their events promptly and accurately.

If that was the case, then we wouldn’t have to deal with correcting events due to wrong ID’s or incorrectly reported results, and when we rate an event it would be in proper chronological order.

Unfortunately, we have problems with both timliness and accuracy. We may be able to reduce both problems, but we cannot eliminate either one.

Based on the feedback we get from MSA, I’m inclined to raise my estimate of the number of events with errors in them that affect one or more post-event ratings from 5% to at least 10%.

Bill, you and I go back far enough that we remember when we only learned our updated ratings once every two months. In that era, if there had been changes in our ratings in between reporting intervals due to corrections or time re-synchronoization we would not have known about them.

That’s no longer acceptable, players want faster reporting of results. I think that in time this will cause the USCF to re-think the whole rating supplement process.

I already see evidence that TD’s are inclined to use the current rating over the published rating, it seems to me that there will come a point when the published rating is largely irrelevant except for situations that demand a cutoff of some kind, such as invitational events or certain national events, where advance planning and eligiblity issues are involved.

The only real difference between what might have happened 25 years ago and what would happen now is that the interim ratings would be known and thus the corrections themselves would also be known.

And what a correction does is get us closer to what SHOULD have happened in the first place.

I suspect that if we are able to indicate WHY someone’s rating changed (e.g., as a result of a tardy or incorrect report) that this will place quite a bit of pressure on TD’s to submit their events more promptly and to take more care in submtitting accurate reports.

If you had a choice between playing in an event run by a TD who has a reputation for submitting late or inaccurate reports or one with a reputation for both promptness and accuracy, might that affect your decision as to which event to play in?

8

There will always be some form of a error in the ratings, from the ‘tournament directors’ or from the data input from the ‘rating department’: when players find a error they ask the ‘tournament director’ that place the error on the ‘rating department’ or the ‘rating department’ place the error on the ‘tournament directors’.

Human error will always be a problem. There will never be a perfect system, there is little way to eliminate all the problems with the rating. Change and tweak the ratings around with re-ratings, will only make the rating have little importance.

If the ‘rating department’ makes the claim of 10% of tournaments are in error with some rating. It would be taking the word of one player that the rating is in error. If a class C players lose to a expert, the ‘tournament director’ reports the claim that the class C player did lose to the expert. If the ‘rating department’ starts to re-rate events, the class C player could lie about the lost game. Even if the class C player shows a score sheet, the score sheet could be a fake, making the ‘expert’ and the ‘tournament director’ show evidence that the game was won by the ‘expert’ and not by the ‘class C player’. Even after the game, some scoresheets are not even signed by both players.

After the tournament, there is nothing in the rules that deal with the ‘tournament director’ holding onto the scoresheets of the tournament. If the ‘rating department’ does go along with the re-rating of the tournaments, there has to be some evidence for the ‘tournament director’ to show if the claim of the player is correct. Even the stronger players will have to show evidence, that they did or did not win the game or even got a draw.

Having re-ratings will drive the stronger players away from the tournaments, or even drive them from being part of the federation. If the expert and master lose or trow away the scoresheet of their game. They have zero evidence to show they did win a game to a weaker player, it would just place the evidence onto the ‘tournament director’ to show the scoresheet. Even if the ‘tournament director’ does have the scoresheet, does not mean that the players did sign the scoresheet in the first place. With players having personal scoresheet books, the ‘tournament director’ will have no record of the scoresheet after the game.

What would happen in a case like this, a expert that wins against a class C player in a weekend tournament during the last round. Both players used personal scoresheets books, making the ‘tournament director’ not having a copy of the scoresheets. The expert informs everyone before the tournament that it will be his last tournament, as he is moving to Europe and his USCF membership will expirer in a few weeks. When the tournament is rated, the class C player can say the crosstables are in error and need to be corrected. The class C player shows a fake scoresheet of the game, the expert is no longer a USCF member and the ‘tournament director’ and the ‘rating department’ do not have the current snail mailing or email of the expert. The ‘tournament director’ never had a copy of the ‘scoresheet’ so the only evidence is the ‘fake scoresheet’ designed by the class C player.

Before the ‘over-the-board rating department’ starts to do re-ratings, they better walk over to the ‘correspondence department’ and ask about how they deal with ‘cheating for ratings’. If the rating department does re-ratings, then the stronger players will always have to defend their games for how long. With ‘correspondence chess’ players have to hold onto their games for six months: how long will the players and tournament directors will have to hold onto the scoresheets?

9

I would say that 90% of the error reports I see are wrong ID’s.

The USCF does not rely on the word of the participants. Corrections must come from the TD or the sponsoring affiliate.

When we get reports of errors from players, we tell them to contact the TD.

However, suppose an event from Oregon has the ID a player from Illinois in it.

The player from Illinois notes the change in his rating and looks it up on MSA. He didn’t play in that event, he’s never played in an event in Oregon, he’s never even BEEN to Oregon.

That also means he doesn’t know the TD for that event. How is he supposed to contact the TD to get the TD to submit a correction?

My all-time favorite example, from a real event, is a scholastic tournament that was held in southern California. The average age of the participants was 9, the average rating was under 800.

However, one of the ID"s was that of a Master from New Jersey who supposedly went 1-4 in that event.

The online tournament submission procedure attempts to catch obvious ID errors. It does catch the New Jersey example, along with about a dozen other examples I’ve collected over the last year and use as a test sample set.

It won’t catch less obvious ones, like when the ID is that of the player’s twin brother and both are rated within a few points of each other.

10

uschess.org/msa/XtblMain.php … 7-12665760

If it is to talk about wrong ID’s, then this one is the best example. IM Ben Finegold rated in the top 15 - 17 in the nation with a 2600 plus rating went to a tournament with his son Spencer Finegold. Look at the crosstables with player 1 and 4 in the tournament, will find a little problem with the crosstables. For some reason they made the mistake of having Spencer Finegold at the time a 1059 player beat a master and expert with a number of stronger players. Being a established player and having you’re rating going up 535 points in one tournament, is something that does not happen that often.

It was a number of tournaments before Spencers rating started to go south, if his rating just broke over that 1600 mark his rating floor would have been 1400 for a class C player. Then again he was around 9 at the time: a 1594 rating for a 9 year old is a nice rating.

Norm is that the rating error you were talking about with the wrong ID numbers. Norm have a little question, how could this crosstable be rated? As it looks to be something that the rating department should have picked up without much of a problem.

11

Yes that is strange that Spencer Finegold is listed twice. His ID number must have been mistakenly used for Ben Finegold.

Also I want to point out that 1400 is class C (1200-1399), not class E (1000-1199).

12

As far as I know, the current ratings programming DOES NOT CHECK to see if the results are reasonable, just that they are internally consistent (in terms of who played whom and the results of each game.)

Depending on how an event is submitted, such an error may not be caught under the current system. If it has to be manually entered, that probably increases the liklihood that it would be caught by an alert operator. If it is on diskette and doesn’t need to be reviewed for other reasons (such as missing IDs or inconsistent results), there’s a pretty good chance NOBODY look at the data to see if it makes sense, there isn’t enough staff time available to do it.

Under the new programming, all events will have to pass a series of validation checks which will look for things such as potential ID errors. It will be possible to override most of the warnings that are generated.

13

There was a tournament last year that had 6 sections. The top section with Masters/Experts looks like it was never rated, but one of the lower sections was rated TWICE! See the following:

uschess.org/msa/XtblMain.php?031207692.0

This tournament was held over 9 months ago. Is it possible to rate the top section this late?

Is it possible to “unrate” the duplicately rated section?

Will the software be updated to check for mistakes like this?

Thanks,
Tom

14

The first thing that needs to be done is to contact the tournament director of this event to see if he still has the copies of this event. It would be up to him to submit the corrected report to the USCF office.

With the new online reporting, my thinking is that it will (or should) be possible to re-rate, and make changes to these events. Not sure, though. I’m sure Mike Nolan can give more on this.

Mike: a few of my events have wrong names, dates etc. Will it be possible to correct this when the online submission becomes available? For instance, one of my tournaments was incorrectly named the 2004 ECC Championship. uschess.org/msa/XtblMain.php?040215227 It should have been the 2003 ECC Championship. I’m not suggesting changes to beginning or ending dates, as I’m sure that could throw everything off.

15

There may need to be some controls in place, but I think TD’s and organizers should be able to make changes that don’t affect the ratings process. The two most common changes I see requested are changes in the name of the event or the chief TD. (I did notice an event, I think it was the 2003 Chicago Open, that was incorrectly coded as having been held in New York.)

The extent to which we will be able to accomodate changes that would affect the ratings process (changes in the player ID, results, event dates and rating parameters of an event) will depend on how well the bulk re-rate concept works out. We may establish some cutoff point and permit changes to our historical records (to correct an ID or result) but not re-rate the event if it is too far back in the past.

It should be possible to go back and pick up the missing section on that event even under the current system, though it would obviously be rated out of chronological order. Deleting a duplicated section is a lot of work on the current system.

Un-rating an event is a non-trivial task in general, though it should be possible to have the computer do most of the work.

I really haven’t decided about checking for potentially duplicated sections during the upload process.

I’ve done some testing for duplicated sections to help the office locate and correct the ones that have already been made, it tends to find a lot of false positives, mostly for small events like quads.

I have yet to find an event with 8 or more players in it which had the same player ID’s and results for all participants that wasn’t a duplicate.

The question with any pre-rating checking is, what do you do about it when you find one? If you just warn about it, there’s no guarantee the TD pays any attention to the warning. If you treat it as an error, then you have to allow some kind of override capability.

The same thing is true about ID and membership issues. How strict should we be?

16

If duplicate tournament reports are checked for by seeing if the players and results are the same, I would think this would be an EXTREMELY time-consuming task, even for a fast computer. If there are, say, 7000 rated tournaments per year, then each of the 7000 would have to be checked against each of the other 6999, necessating almost 50 million pairwise file comparisons.

Bill Smythe

17

If you restrict it to just tournaments from the same affiliate or TD with the same ending dates and with the same number of players and rounds, that keeps the number of events to check down to something reasonable.

Also, if you do it by computing and saving a hash value for each event (based upon the membership ID and result for each player, you can check a much larger set of events for possible duplicates.