They don’t. Such errors, might, however, hold up the processing of a membership (e.g., if $19 were submitted with a birthdate of 1942), which would, in turn, delay rating (since the player would not have a valid membership).
The rating file format, one created back in the early 1990’s, does not
include the name of the player, just the ID.
So, we cannot match the name against the player. If the ID is valid and current, all we can do is check to see if it looks reasonable.
If the TD inputs the wrong ID or has an ID/name mismatch in his local player database, we may not be able to detect that.
Some work has been done towards a new file format, but that format is still not available for the pairing authors to modify their programs, and even once it does become available it will likely take years before most TDs have upgraded to a new release of their pairing program.
(We still see a few reports sent in with files prepared by pairing programs that are over 10 years old.)
USCF Tournament Rule 23C:
For the inclusive dates of the tournament, each player must be a member in good standing of the USCF, unless USCF regulations waive this requirement.
Assuming the IDs are correct, someone’s membership status does not affect the ratings from an event, so there’s nothing to rerate.
Thus it is stated USCF policy that in most cases players MUST BE MEMBERS in order for their games to be rated (even if otherwise ratable), and the tournament validation program enforces that rule, as it was designed to do.
If you don’t like that, work to change the rules. Unless those rules are changed, enforcement of the USCF membership requirements will remain a part of the validation process, because those were the instructions I was given when developing the software.
Actually, age MIGHT affect the ratings from an event, because for players under 26 their age X 50 is used as an initial estimate of their rating.
However, THAT is something that rerating can correct for.
The online membership programming would not accept a $19 membership for someone with a birthdate of 1942, so the TD would know there was a problem to be corrected before the event could be submitted.
I don’t honestly know what the USCF office would do about it if the rating report and membership were received in the mail.
They might let the event go through and then follow up on the membership problem afterwards. It would likely depend on the TD, some TDs are known to be diligent about follow ups, others have a history of sending in bad data and then ignoring the office’s attempts to contact them.
In the long run, all of the policy level checks are a matter of enforcement. Once the event is rated, what are our enforcement options at that point? If we go back to the so-called ‘Traci letter’, and the TD chooses not to respond, what happens to that membership issue?
I think John Hillery has the right idea when he says that the USCF should be trying to help events get rated, not find reasons to NOT rate them.
However, at some point, policy enforcement has to be taken into account.
But as I have said several times, policy enforcement matters such as membership issues are not, in my opinion, the primary cause of rating reports being submitted late.
The office doesn’t need to guess. The algorithm that flagged the inconsistency in the first place also gives a clue on how to correct it. Simple example to illustrate my point:
Name and ID don’t match. Both are in your database but don’t match in the submitted report. Which is more likely to be wrong? Highly likely that the ID is wrong rather than the name. Simply replace bad ID with correct and send email to organizer for confirmation. Go ahead and rate event without waiting for confirmation. If subsequent confirmation is affirmative (which is highly likely) no further action needed. If confirmation is negative make neccessary correction and re-rate affected events.
This is directly related to the “classic” example you gave. It could all have been flagged and corrected programatically without delaying rating of the event.
In general this should be the policy, do as much as you can without involving the organizer and thereby holding up rating of the event. This doesn’t require office staff intervention, it can all be done programatically.
I’m a little unclear on exactly what your question is. First, we should distinguish between two cases, rating reports submitted on line, and all others. Uploaded rating reports are checked for the presence of ID numbers, valid memberships, consistent internal data, current affiliation, and certified TD. If any of these are off, the report will be kicked back for the TD to correct. The process also flags a number of other things as “warnings,” like a player from a different state, or someone performing much higher/lower than his rating. These have to be changed or overridden, but since most of them are unimportant, they can be overridden by a simple checkbox. Once the TD corrects the errors and overrides the warnings, the re-submitted rating report is generally processed at once. Unless someone can cite an example where this was not done, I fail to see the problem. My experience has been that any rating report that passes validation is rated within hours.
Rating reports not submitted on line are a different matter. When the arrive in the office, they must be deciphered, and any missing data entered. This means processing any memberships (which may have come in the same envelope), generating ID numbers, matching them to any new players, and entering the report into the rating program. Then it can be checked for errors, and may have to be returned ot the TD for clarification. (If, for example, “John Smith” is listed as a new player, but there are five expired John Smiths from the same state with about the same rating. Or, as Mike pointed out, if the ID numbers indicate one player from the other side of the country with a rating 1000 points above the field.) Considering the difficulties involved, the office does a pretty good job with these, but it is not realistic to expect them to be rated instantly.
No. They would only hold up processing if you choose to let it by means of a policy as explained by Mike. And my point is that such a policy is neither neccessary nor desirable. In this particular instance since funds were submitted which were apparently wrong we know that an error needs to be corrected. The most likely error is that the incorrect funds were submitted. We can reasonably assume that the player does indeed wish to be a member and that the funds issue will be reconciled later. Simply let the rating of the event proceed. If the membership fee is not reconciled with a reasonable time frame then simply void the membership and re-rate affected events. Very simple.
The problem is that the rating report file ONLY has the ID, not the name. Once we do the lookup, the name and ID do match, so if they’re wrong they’re BOTH wrong.
About a third of the reports submitted to the office are on diskette. Those receive essentially the same treatment as ones entered online, which is to say that if the ID is wrong but otherwise valid it may not be caught.
As I understand it, the office staff does not take the time to compare by hand the printed crosstable with the IDs on the rating report diskette, and has not done so for many years. Should they? That’s not a matter for me to decide, that’s an office tasking function.
For those reports submitted on paper, the staff is supposed to check that the name matches the ID, in part to catch keying errors. In those cases, if the ID is wrong or invalid, the staff can then check against the paper crosstable and hopefully correct the ID.
And very unnecessary. The rating formulas DO NOT CARE if someone is a USCF member or not. That’s a matter of administrative policy, not mathematics.
Rerating an event just because someone’s membership status changed would not affect the ratings from that event.
I have some sympathy for this position, but the EB does not, and it’s not going to happen unless the Delegates order something different.
As a practical matter, memberships and rating reports are processed separately. The tournament cannot be rated until everyone has an ID number. Your suggestion would require either that an ID number be arbitrarily issued for the rating report even if the membership has not been processed, or that the membership department should process the membership even if they have not received proper payment. Both of these have obvious drawbacks.
So what you’re telling me is that the event report submitted by organizers contains no cross reference between player name and ID??
How is this possible?? Wall charts have names. Pairing programs store player names.
Ok, if the USCF has no rational data model and database schema then I’m just wasting my breath talking about this. My presumption going into this discussion was that with all the software changes made by the USCF recently fundamental things like this were well under control. This just blows my mind.
And please, re-read what you just wrote regarding 23c.
Only iff you guarantee that the player is a valid USCF member during the event. Or are you now saying that it’s ok to violate policy and let the ratings of valid members be affected by invalid members?? How would re-rating not affect an event? The event is now rated minus 1 player. Obviously everyone that he played during the event needs to be re-rated. And any subsequent events that those players participated need to be re-rated as well. Practically speaking the impact is minimal because an error of this type will be caught quickly before the downstream effects are widespread. The re-rating could be run as a batch job overnite.
No, that is not the policy, and never has been. All games played in a rated tournament must be rated. You cannot rate only the games played by current members. This has come up before, and there is no ambiguity. 23C puts the burden on the TD to collect and submit memberships. (How strictly this is enforced is another matter.) But the USCF’s only choices if expired members are present is are to rate the tournament or not rate it. The latter has little or no support.
I don’t know how long it was happening, but for some period of time under the old ratings programming if there was no ID and the office could not figure out the right ID, they would assign a new ID as a non-member and go ahead and rate the event.
This is the policy you appear to be advocating a return to. We know that in many cases the USCF office did not follow up on those non-members. I identifed over $25,000 in lost membership revenue (possibly several times that) in 2003 arising from non-members.
We also know that this poor enforcement of membership policies led to many duplicate members, members with more than one USCF ID. (I did a statistical sampling of the database in early 2004 and estimated that we had over 4000 duplicates in the database.)
Since the new membership database went into use in March of 2004, we have identified over 1500 IDs as duplicates. These flagged IDs can no longer be used on rating reports, they are now considered invalid.
As a result of enforcing membership requirements more vigorously and more consistently, the number of non-member IDs issued in 2005 was 918, less than half of the number in each of the three previous years and the fewest since 1992. We have created only 78 for the first three months of 2006, so we’re on a pace to create perhaps only 300 for the full year.
So while it may appear Draconian to some, the enforcement procedures in the new programming appear to be working to the benefit of improved accuracy of both memberships and rating reports, despite the absence of the name in the reporting file format. And in the process I think we have increased USCF membership revenues a bit, too. (I’m fairly sure that checking for current affiliate memberships when validating events prior to rating them led to about a 6% jump in regular affiliate membership during 2005.)
When I started design work on the process for uploading rating reports, one of the decisions that was made, after consulting a number of TDs and the authors of the major pairing programs, was to support the current file format.
That format, which was originally defined in about 1991, does not contain the player’s name. Some work towards a new format, which includes the player’s name, among other things, has been done, but the process got bogged down by some disagreements over what fields to include.
We were also waiting for FIDE to release their new reporting format, which they did late last fall.
I’ve been studying the new FIDE reporting format and in the continued absence of a new MIS committee I have asked the FIDE events ‘team’ at the USCF office to look at it too. It may turn out that we will adopt the FIDE format (or a slight revision of it) rather than define our own new format. That decision is currently on hold because of a combination of higher priority tasks and the delay in appointing a new MIS committee to help review the format.
Based on several comments I don’t think you understand the ratings process or rerating very well. Rerating is a rather complicated process using a model that is based on queuing theory, a process that Ken Sloan and I came up with last August.
When we rerate, we rerate EVERY event starting with the first event that has been changed since the last rerate. Most of the time that takes us all the way back to January of 2004. We just finished a rerate pass on Monay, if I started another pass today I would currently only have to go back to April of 2005, so maybe we’re FINALLY seeing a slowdown in corrections to events from 2004!
We looked into seeing if we could figure out which events needed rerating based on which players had been affected from previous events, that turned out to be far more complicated and time-consuming than just rerating all the events in the month, since the average event only takes 5-6 seconds to rerate and it was taking longer than that just to check to see which players in that event had been affected by rerating their earlier events.
Each block of events (a month’s worth) takes 2-3 hours to rerate, so a full rerate, going back to January of 2004, takes the better part of a week to complete, usually averaging about 5 months each day. (We take about 10 hours off from rerating for overnight systems tasks and to get our daily refresh of the MSA database done each morning.)
The reason it is so difficult to identify the scope of re-rating is because you don’t have a data model that supports it. In fact having taken part in this discussion I’m fairly certain that nobody at the USCF can even articulate what that model should be and how it is implemented. In order to fully support re-rating with a minimum of resource consumption the re-rating capability must be part of the initial design. From what you’ve described in this forum the software just looks like one procedural hack on top of another. With some policy hooks glomped on top!
Queueing theory? Oh please. With a sound data abstraction the process of re-rating is nothing more than a simple iteration. If you have a white paper that has a coherent explanation of how queueing theory pertains to re-rating of events I’d be more than happy to take a look.
I suspect that in most cases where re-rating is needed the scope is extremely narrow. Consider regions of the country where not many tournaments are held or club players that only play in their club’s events. Doing a full re-rate of everything on account of only one or two events that needed it is a ridiculous waste. And it has iherent dangers too. What if the system crashes during a full scale re-rate? Does the system have transactional safety to allow you to resume the re-rate from the point of the crash? An even more serious danger in doing full scale re-rates is the effect of policy induced rating changes which might not be tracked. Bottom line is that for reasons of data integrity and resource consumption re-rates should be strictly limited to those events that actually need them.
I find it totally perplexing that there seems to be a lot of thrashing with regards to the actual fields in the reporting format. Guys, we’re not dealing with the data sets of subatomic physics experiments. There really isn’t that much data to deal with. If in doubt then include it! It is much easier to not use unneccessary data than it is to require code to compensate for the lack of data! Make everything as data centric as possible. In the future you might just discover that prior inclusion of a seemingly unneccessary data item now yields a huge payoff because there is now a need for it that wasn’t previously envisioned. Modern RDBMSs are efficient and storage is cheap. Let them do as much work as they can.
One reason why we used a queuing theory model is because both Ken Sloan and I understood queuing theory, as did the other members of the Ratings Committee who were in Phoenix last summer. 
Ken wanted me to use smaller blocks than I wound up using (a month), in theory the block size could be as small as 1 event.
The rerating process is not that hard to explain, though it gets a bit complicated to implement in places.
- The events are placed into a defined order based on the event ending date, the event beginning date, the USCF Event ID and the section number. This establishes what the most recent event is for each player.
The latter two sort terms are used to ensure that there is a consistent ordering scheme. The Event ID is essentially a random number, though in fact it is based on the tournament ending date followed by a four digit number that wraps around. Currently the last digit is a ‘0’ for events rated under the old rating programming and a ‘1’ for events that were initially rated under the new programming, though that distinction isn’t really necessary.
We do check that an event ID is unique when assigning it. (If we ever get 999 events that all ended on the same day we’re in trouble, though I could change that to 9999 events by eliminating how the 12th digit is currently used to code the ‘source’ of the event.)
- Once we rerate any event, all subsequent events are considered ‘dirty’ and must also be rerated.
Technically, under a queuing theory model, we would only need to declare an event dirty if the event itself has been changed or if the pre-event rating has changed for any of the players in that event. There are at least five reasons why that could happen:
A. An earlier event was rerated for one or more of the players and was also the most recent event for one or more of those players.
B. An event has been added for one or more of the players, thus changing what the most recent post-event rating is for those players.
C. An event has been deleted (or an ID changed in an earlier event) for one or more of the players, thus changing what the most recent post-event rating is for those players.
D. Something about an event that affects the ratings from it has changed. Usually this means the starting or ending dates of an event have been changed, thus affecting where it is in the sequence of events to be rated and affecting what the most recent event is for one or more players in that event.
E. The pre-event rating for one or more of the players has been changed manually. The two situations that are most likely to cause this are a returning player whose old rating had been deleted and was not restored before his first event after returning to rated chess or an unrated player who has a FIDE rating that was not used when the event was initially rated because we didn’t have that player’s FIDE ID on file yet.
In practice we don’t drill down to the player level. That’s because it currently takes more effort (eg, more database accesses, thus more time) to determine whether any of the players in an event are dirty than to just declare the entire event dirty because it is after an earlier dirty event and rerate it.
It may be possible to change how the database is organized to make it easier to determine if an event is dirty. That’s on my list of things to explore in the future, along with some other database techniques that may enable me to speed up the rerating process. That also might make it desirable or necessary to use a smaller block size.
The ordering process is a bit tricker than it might seem, because changes to the section starting or ending date can change where it appears in the sequence of events.
When rerating an event, some of the steps are slightly different than when initially rating an event. Mostly that was needed for events that were rated under the old system but are within the rerate ‘window’ (events that were initially rated on or after 1/1/2004), because in most cases we need to use the same initial estimate as when the event was first rated, because we have no record of how that estimate was generated other than the initial rate of the event.
Yes, the database has transactional integrity. Also, the table structure design is such that if there is a failure during the rerating of a block, we can just restart that block.
In fact, there have been a couple of database failures during a rerate. In one case they had a power failure in Crossville that lasted longer than the UPS could handle. (I think the power was out for about 18 hours over a weekend and the power failure wasn’t detected for several hours.) We also had a motherboard failure that happened during a rerate.
As a software developer in the telecom industry for 15 years I’ve encountered queueing theory once or twice. And how it applies to a simple tree traversal problem is something that you Ken wouldn’t keep secret, right? 
I have no idea what you mean by block. I’m guessing that by block you mean an interval of time during which events can occur. If that is true which I suspect it is then you are using a completely inappropriate algorithm. A simple tree traversal algorithm which can be found in any text on data structures is right on the mark. This would guarantee that each node is processed once and only once without any unneccessary processing. If you have found some way of contorting queueing theory into producing a way to solve this problem I guarantee that it is less efficient than a simple tree traversal.
It’s as I suspected, you have a bad object model. Each player object should have an event history attribute (you already seem to have that since tournament history can be viewed for any member). The tree nodes are the dirty events. You construct the tree by iterating over over the dirty players in the initial dirty event. This is repeated for every generated dirty event until no additional dirty events are generated. It is a very straightforward algorithm. I suspect in practice you will rarely end up with a big tree, at the very least it is guaranteed to be less work than what you’re doing now. With the right DB schema and a caching ORM solution the re-rating should take a fraction of the time it takes now.
If I’m hearing you right you seem to be expressing a concern over the effect of policy on the re-rating execution. Yes, I would agree with that. If the application or lack thereof of policies that can produce rating exceptions isn’t accounted for in the data and code then blindly grinding through a re-rate could produce errors. It seems the only way to handle this would be in a piecemeal manner requiring intervention at various points of a re-rating run. This would be a very bad design, something that should be cleaned up ASAP.
Perhaps the next time I review the crosstable system data structure you can make suggestions as to how to automatically flag ‘dirty’ results within the context of a relational database model.
The alternate models I’ve tested have not been an improvement computationally. Yes, in theory there are linked list models and such, but in practice they tend to be computationally rather expensive.
(As I recall, the original papers by Codd and Date on the relational data model expressed some concerns that they might not be computationally practical, and that was probably true at the time. Computers are so fast these days that the gross inefficiency of a three or four way table join is no longer particularly relevant.)