Rerates & Rating Floors

Running Chess Tournaments
1

I’m wondering if some of the issues of rerating have been thoroughly thought through. This may have already have been discussed but I couldn’t find it. If so, please direct me to the correct thread.

  1. One strange anomaly that occurred to me is the permanance of the official rating supplement rating vs the changing post-tournament ratings caused by rerates. It seems possible and likely that there will be players who have official ratings that don’t correspond to any if their post-tournament ratings. And in fact may be higher than any rating they ever had after a tournament.

  2. That brings to mind the issue of setting rating floors. At what point in the process will these be set? And when a rerate means that a changed floor shouldn’t have been changed, can that be fixed automatically. I’m not sure how we can explain to someone that they have a floor based upon a rating which now appears to have never existed.

  • Tom Martinak
2

On a slightly different topic, I recently asked for a rating floor and received an affirmative answer from Larry King of the ratings department. The floor pertained to a prize I won back in October. Since then, I’ve played about four events during which my rating has been below that floor. I’m curious how the floor is implemented. From today on or retroactively since the October event?

3

The USCF office didn’t keep very good records on ratings floors that were raised because someone won a large cash prize. Not all of those raised floors have been retained, some of that information is just unavailable.

(I spent most of Tuesday going through what records we had looking for old ratings that needed to be restored and floors that needed to be set before we ran the last grand rerate.)

In the new ratings system, we use the historical (ie crosstable) record for a player to track his floor, plus we can raise or lower someone’s floor
as of a particular date with a ratings correction record.

(I’m not sure if we have many records of lowered floors, either.)

This has created a few problems with ratings floors that I’ve been fixing today.

Most of them appear to have come from events where an ID was changed but the incorrect ID had a higher floor than the corrected ID does. If the floor wasn’t changed along with the ID, that creates an inconsistency.

Since the new ratings system scans the historical record for a player to determine the floor, a few players (maybe 20) were treated as if they had a floor that they shouldn’t have had. In one case a player who should be rated around 1070 was given a floor of 1400, affecting all of her events in the last year.

Floors can be dynamically redetermined during a rerate, because for each event we only look at those events with a lower sequence value.

A related issue has to do with master certificates. I’m still thinking about that one.

4

Mike,

A significant correction needs to be made immediately that should impact the April Supplement. Romallis St.Cyr has been given an incorrect rating floor.

12499860: ROMALLIS T ST CYR

This error appears to go back about 76 tournaments. His rating jumped from the 1500 to a 2000 floor suddenly. He is a very active player entering events almost weekly in Kentucky, but also playing Nationally often. It has inflated a large number of players in Kentucky by a large number of rating points.

On the eve of participating in the Huge event in Minnesota, the Chicago Open, and most early summer events, this will cause players to jump suddenly into a rating class much above where it should be. That will cost these major events some entries.

The potential prizes lost due to a clear error would suggest that USCF has lost control of the rating system totally at this point. If a player has already paid an entry fee, and is suddenly not eligible and not corrected quickly, a lawyer friend suggested that a class action suit would be very bad. And that such a suit would have a good chance of success.

Mr. St Cyr’s lifetime activity is clear as a bell on the USCF website. The floor is inaccurate. It created a massive shift of points in Kentucky where he resides, and had residual effects across the country.

If lawyers are reading this post and others concerning ratings, please respond to the following possibility. Is it possible that a court could order USCF to refund entry fees for players who’s class was changed by an error like this? Is it possible that organizers of events could hold USCF responsible for potential loss of revenue because players avoided events due to rating jumps that did not reflect their true strength?

Our national database should not be a test run for a new program. The dues paying membership is really offended by being turned into test rats. Faith in the fairness and accuracy of the rating system is critical and is being breached. First USCF ignored it’s basic responsibility to the membership by going several months without any rating runs. Now the system is rife with error.

TD’s should look carefully at the rating history of their events. This error may not be the only one of significance. Mike will need this data if he has any hope of fixing this broken program and saving USCF big bucks from the potential this error has of costing them.

Sincerely,

Steve Dillard

5

His rating has already been corrected to 1540 for the April 2005 supplement.

Personally, I believe there are far more errors in the ratings database due to mistakes in reporting events than to programming issues. If people want to hold the USCF responsible for its mistakes, shouldn’t TDs be held responsible for theirs?

Perhaps the USCF should be more aggressive about suspending TDs who submit inaccurate or duplicate reports, since that’s where many of the problems in the ratings database came from.

I recently saw a correction in which it seemed like a third of the results were being changed. How on earth did such an inaccurate report get submitted in the first place?

The USCF receives several reports every day of events with errors in them, many or most of them were made by the TD submitting the event. Based on those reports, it seems possible that 5-10% of the events we rate have at least one error in them that affects the post-event ratings.

And then there are duplicate events. Earlier this week we deleted about 35 sections from events held in the last year that had been rated more than once. I saw a few more reports of duplicated events today, so there are almost certainly more of them out there.

Last year I developed a search protocol to look for duplicated events, it identified well over 120 sections that may have been rated more than once. Several TDs had more than one event on the list. Not all of them were actually duplicated events, but quite a few were. Some had been submitted by more than one TD, or under two slightly different names.

We found one TD who had submitted quite a few events more than once, one of them 3 or 4 times. A couple of duplicate submissions could be chalked up as an accident, this seemed to be more of a pattern.

6

Well I am no lawyer, but that statement has a lot of misconceptions. First USCF has nothing to do with entry fees. That is the responsibility of the tournament organizer. As for organizers holding USCF responsible for potential loss of revenue, that would be hard to hold up in court. It is also counter productive. Filing a lawsuit would just take the time that USCF could use to correct the problem to fight said lawsuit.

Generally the person, in this case ROMALLIS T ST CYR, should have notified the USCF about the problem (which I would assume he has). That is a key to communication and fixing problems. Automatically threatening lawsuits doesn’t fix problems it makes them worse.

As a dues paying member I am not in the least offended by the USCF implementing a new (better) program. Sure it has some initial problems, and I know Mr. Nolan is working very hard to fix them. Recently the corporation I work for put in an upgraded Oracle database. It caused some problems the first few months, but now the benefits are far more greater than the previous system. I have no doubt that when the initial database that USCF used previously was implemented it had bugs as well.

The new database Mr. Nolan is working on is to enhance the service us members recieve. I don’t mind short term problems to do that.

As for the neglect of updating the ratings for months I must agree with you. However, I do not understand all that goes into what the USCF does to get the ratings going so I am glad to extend them mercy. Besides I am sure as a TD I have made it hard on them in the reports I have sent in and they have always been gracious. With this new database I am seeing more of what it takes to rate tournaments and what I need to do to make it easier on USCF staff. :smiley:

7

‘Months’ is a bit of an overstatement. The last ratings run for the February supplement was on January 11th, the next rate run was on February 14th.

Two weeks of that time was the usual down time after a supplement, caused by the fact that the ratings staff in NY lets EVERYTHING ELSE slide in a mad scramble to get as many events rated as possible by the cutoff point.

I think the changes we have made to implement online submission and the office reorganization should cut into the backlog and bring it down to a manageable level. Instead of 2 clerks (only one of whom is a full-time employee) assigned to entering rating reports in NY we now have 5 full time employees and one part time employee in TN trained to enter both memberships and rating reports.

8

Dear Mike,

I am one TD that submitted duplicate reports about 6 months ago. It has happened in the past as well. There was a simple reason for the oversite. U S Chess took so long to rate the event that it appeared that they had not received it. U S Chess did not respond to phone calls or emails questioning the events status. At the time, the received events page was not kept up to date so there was no way to be sure it was handled.

I am sure, that I am not the only person who can confirm that USCF does not effectively answer phone calls, or emails. This may be improving, but the problem with duplicate reports falls back to USCF. When a TD submits a report, and the report is idle for weeks at a time, and they cannot confirm anything after diligent efforts to follow up, then sending a duplicate makes sense.

What do I mean by idle? Idle means to me, after 3 weeks, the report does not even appear of the list of received events, the USCF does not return phone calls about the event, no live person speaks to a concerned TD after several attempts to wait out the USCF answering service, emails go unanswered. That is idle.

Once USCF discovered that an event had been rated twice, they then went into panic. Just as your post suggests, they blamed others when simply answering an email or the phone weeks before would have remedied the solution. You suggest holding the TD responsible. Well, basic services such as response to messages, and answering the phone would completely eliminate many potential errors.

Tournaments with similar names happen. One event that runs regularly in Kentucky is the McAlister’s Open. Every Monday it occurs. I run it. The only difference in name would be McAlisters Open 3-22, McAlister’s Open 3-29. I run about 50 tournaments per year, most of them McAlister’s events. There is no attempt to create problems at USCF by naming the event. But someone who submits such a regular list, should be easy to recognize. I also submit on paper. That should be easy to recognize. Apparently, I am rare. But in Kentucky, I kind of enjoy my dinosaur status.

So, before considering any kind of sanction of a TD. I suggest USCF do a few basic things to eliminate the problems.

  1. Answer the phone.
  2. Respond to emails.
  3. Rate events from all sources in a reasonable amount of time.
    a. USCF suggests 1 week to submit a report.
    b. I suggest 1 week to rate it.

Thanks for providing the forum. I enjoy the discussion here. It is valuable.

Steve Dillard

9

Those are pretty much the same goals that I have been working to achieve at the new USCF offices in Crossville.

The biggest challenge in most computer projects is not the systems design or the programming, it’s the social re-engineering of the organization. The USCF was and in many ways still is highly resistant to change. That’s true of its membership as well.

10

You say it as if that were a bad thing.

11

Ever read Charles Darwin, John?

12

Indeed. Most mutations are not favorable.

13

Dear Mike,

I took you at your word when you stated that St Cyr’s rating had been corrected. But I have now reviewed the crosstables of some of the 50 plus events that he played in. I have also reviewed the crosstables of new events just rated in the last week.

Mr. St.Cyr’s rating is still working off of the 2000 floor. Multiple tournaments have been impacted. Quite a few players have had rating inflation based upon defeating a “2000” rated players. His 2000 rating is falling, but I assume an 1800 floor will kick in soon.

Here are a few questions that have generated because of this problems.

  1. If a rating is boosted for example from 1780 to 1801 because of playing St.Cyr. Will a floor of 1600 be generated? How will the computer be told that it is an incorrect floor.
  2. (not just a St.Cyr question) If a rating crosses a 100 point margin before being re-rated, and after being re-rated falls back, will that kick in a higher floor?
  3. Bill Goichberg is running a large event in April at Executive West in Louisville, KY. Several players were considering entering that tournament. The ratings have gone up much higher than they should. How will their ratings be handled without a complaint of sandbagging.
  4. St. Cyr would like to enter events outside of the Louisville area, and often does. What is his rating? What should he tell organizers and TD’s to avoid the issue of sandbagging?
  5. When rating or re-rating events from the Louisville area, will you or the USCF official responsible for running the ratings watch events that St.Cyr has played in to avoid continuance or repetition of this problem?

A really good answer is needed here. There is a real chance that a Kentucky Judge will be asked to “help” Mr. Goichberg place players in appropriate rating categories so that they will be eligible for appropriate prizes at the Louisville Open. The credibility of the rating system has been impacted on a slew of local players. They need a real rating that they feel is fair. There is a real chance that players going to St. Paul/Minneapolis will seek the same fair rating without bias. Players going to Chicago, New York, etc will want a clear rating.

This forum is a clear piece of evidence that a timely request has been made to clear up this problem. The programmer, and USCF have been given notice of the problem. There is adequate time to repair it or to contact players impacted by the error with documentation as to their real rating. On this forum, Mike Nolan stated St.Cyr’s rating had been corrected. But new tournaments, and old tournaments still show a large number of events rated with bad data.

Please correct the problem. Please observe the results of future event that St.Cyr plays in until it seems the problem is truly gone. Please check the impact of the error on rating floors. Please answer the list of questions contained in this post.

14

We have not done a rerate since those corrections were made. His rating in the April 2005 supplement was manually corrected to 1540, and the other changes were made so that they would be picked up in the next rerate.

Once we have done another rerate, I will see if any of your questions still need answering. Based on a test rerate including events rated after the April supplement cutoff, it appears his rating may drop back below 1500 after all his events are rerated.

In the mean time, I remind EVERYONE that unofficial ratings are unofficial.

His current PUBLISHED rating remains 1468, it will go to 1540 in April.

I have a request for you, Steve. Would you please check all of your events for 2004 and 2005 to see if there are any that have been rated more than once, either as a result of your multiple submissions or because of office errors.