I’ve covered this several times in the past, by more complete I mean members that aren’t currently included in the bi-monthly supplement files but are on a Gold Master. This includes new members, unrated players and those who have renewed their membership but whose rating has not changed.
It also includes IDs that have been flagged as duplicates, deceased members and IDs that have been inactivated for other reasons.
Currently if you don’t download the latest Gold Master each time one is released, you don’t have a complete database of USCF members who might play in your tournament. And if you do download a Gold Master, it is already out of date because we add or update over 100 USCF IDs every day, including weekends.
With the new dynamically built update process, you will get all memberships that have been added or changed (for any reason) since the cutoff date you select. That will bring you up to date with the USCF’s internal membership records as of the point where you create the download file.
The custom ratings list feature that is currently available through TD/A has the ability to select any of four sets of ratings: Most Recent plus the last 3 supplements. (You can also select quick or regular ratings, and we may add correspondence to that list fairly soon.)
I had been considering having both the published rating (probably selectable from among the last 3 published ratings like in the custom ratings lists) plus the most recently updated rating in the new format, but I"m now leaning towards not offering the most recently updated rating at all in that format and having it only provide published ratings.
The primary reason for that is the most recently updated rating could change several times a day for someone, so two people could download an update and get different ‘most recent’ ratings for some players. For those who think they need that information, it is already available through the custom ratings lists.
Perhaps I didn’t state my question clearly enough. The bottom line is that I do not particularly care how the database is constructed; what I care about is what it does. Now, I’m pretty sure that the vast majority of people who use it do so though a pairing program (or possibly Tournament Administrator for a few dinosaurs). The pairing programs are set up to import the last published rating when you click on a player. My question was whether this would still work properly with your new-format updates. If you are not going to include “unofficials,” the answer is a simple “yes.” If you do include the late updates, there are three possible answers: 1) yes, always, 2) only if you configure the updates correctly, or 3) no, not if the player’s rating has changed. #3 is obviously silly. #2 would cause you a lot of headaches when TDs did not understand what they were doing. #1 is (in my opinion) the right answer, but then why include the updated ratings? It sounds like you have reached the same conclusion by a different route.
Since this is a new format that will require the developers of the pairing programs to update their programming, presumably if it did contain both the current PUBLISHED rating and the most recently updated rating, the program would know which field was which and handle them appropriately.
My concern is that from a pairing standpoint, I don’t know that the most recently updated rating has as much utility when it is possible to be updated more than once a week, which was about as frequently as ratings batches were run on the old programming. Someone who downloads or looks up someone’s ‘current’ rating on Thursday night might find a different number on Friday night and different again on Saturday morning, making its use at a tournament subject to consistency issues.
Since the new-format supplement file isn’t available yet, this is somwhat speculative. However, based on my tests:
It MIGHT work with Swis-Sys, because it uses a table-driven approach to what’s in the supplement file.
It will definitely NOT work with WinTD until that program is revised to accomodate the new format, because that program requires an exact match with the current format in order to process the file. Thus we can’t add add or change the definitions of any fields, not even to increase the number of characters in the name field.
However, both programs will need to be updated anyway to prepare the new format for the dBase upload files.
What are these custom rating lists? Are they currently available, and where? Are you saying that we have the ability to choose different ratings when downloading the rating files?
Also, you said that:
Exactly what won’t work with WinTD? I haven’t followed this thread much, so please indulge me a bit.
Then I’d say my concern about the continued availablity of the old-format files was justified. You can’t just pull support for something with a large installed base, especially when the replacement doesn’t exist yet.
There is an option in the “TD/Affiliate” area for “Custom rating lists.” This seems to be similar to the state/rating sort you could do on the old (defunct) rating site, though the front end is a bit more complicated.
How far back do you want me to start? Most people use the USCF database through a pairing program. (There are other ways to access it, but they’re kinda clumsy.) Type in an ID (or name) and the player will appear in the database window. Double-click on the name, and the program will import name, ID and current rating. (It’s a little more complicated with WinTD, but it works out about the same in the end.) Merge in the rating supplements every two months and your ratings will always be current. Of course, you can use a pairing program without the database, but you are giving up a lot of functionality.
My question was whether the existing programs would still work with the new-format database Mike intends to provide in four or five months. The answer appears to be “no.” This brings us back to what started the thread, my, ah, strong suggestion that the existing-format files continue to be offered as they have been in the past.
Thanks John - I see it is a tab-delimited file. I was thinking it may be something we could download automatically in the database files for the pairing programs.
How many times do I have to say the same thing until you read and believe it: THE CURRENT FORMAT FILES WILL CONTINUE TO BE AVAILABLE THROUGH THE MEMBERS ONLY AREA.
However, the new format files will ONLY be available throuth the TD/Affiliate Support Area. I also think they’ll be available sooner than 4-5 months from now. I’ve got a program that creates the new-style format files already, I just need to finalize the file format.
It is worth noting, though, that I sent out draft copies of the new dBase rating report format to Thad Suits and Tom Doan before I made them available here, and I have yet to hear one word from either of them about it.
Therefore, I would suggest it may well be 5-6 months if not MUCH LONGER before the pairing programs can either create the new format rating report files or read the new format supplement update files.
Are these files (Golden Database and Supplemental) going to be available in a tab-delimited format as they currently are in the members only area? I see mention of the dBase format (in relation to Thad Suits and Tom Doan - not sure who they are) for the ratings reports, but nothing else. The Vega pairing software already manipulates the tab-delimited Golden Database and Supplemental files and produces the three ratings files in the proposed tab-delimited format specification. If the format for the Golden Database and Supplemental files are going to modified, could you publish the proposed format for them, similar to what you did for the ratings files?
Yes, there will both a dBase and a tab-delimited version of the new supplement file format.
Note that there will be just ONE file format for both bi-monthly supplements and gold master files. (We will continue to make the current bi-monthly and gold master files available as well, so that people on older versions of pairing programs aren’t left stranded.)
TDs will also be able to request a custom supplement file which will include all membership and ratings changes after a given cutoff date, which will enable TDs to bring their database up to current status regardless of when the last time they updated it.
The format is not quite ready for public comments, mostly because I’m kinda busy on other tasks.