(I’m posting this as a separate topic instead of just putting it in the existing “combined database” topic because it may be of interest to those who don’t care about FIDE. Those readers may be skipping over the existing topic.)
For several years, I have been creating a monthly rating database that mirrors the official US Chess golden database but that also includes FIDE ratings, titles, and federations. SwissSys users who direct FIDE events are able to use this database as a replacement for the TARATSUP.DBF golden database.
However, there is now a reason the combined database may appeal to those who don’t care in the least about FIDE. My combined database now includes full player names, not the truncated names (limited to 17 characters) that appear in the golden database. (As a bonus, the database also includes all three over the board US Chess rating systems – regular, quick, and blitz.)
If you don’t care about FIDE but want to use this database, just download the latest version every month at http://www.sudburyriverchess.com/ratings/latest.php. (I post a reminder every month when a new version of the database is available.) Copy the uscf-fide.dbf file over your TARATSUP.DBF file and index the database. You won’t need to make any changes whatsoever to your database settings. (You may for convenience want to change the file name in your database setup to uscf-fide.dbf.)
I have updated the current database (April 2018) with the “long names” version.
Fair warning: because the name field is longer, the “index database” operation will take longer than you might be used to.
(I’m sorry, but this database will not work with WinTD. As of version 4.20, WinTD does not accept a rating database whose structure differs from the golden database.)
I downloaded the combined database and played around with it. When you try registering someone in SwissSys by searching for their name, it doesn’t always work. For example, when I type in “Smith, Micah” it doesn’t find anyone with that name. Only by putting in my ID number does it find my record.
Thank you for reporting this. I think the cause of the problem is that the full names I imported had a space after the comma separating the last and first names (e.g., “SMITH, MICAH”). The golden database has no space (e.g., “SMITH,MICAH”). While SwissSys doesn’t care about the length of the name field, it appears that the extra space does cause a problem with the name index.
I’ve uploaded a revised combined database. Please try it again and let me know if the problem persists or if you have any other problems.
I hope I’m not speaking out of turn on this, but my understanding is that FIDE is considered the authoritative source on FIDE ratings, and they release them (and require their use) on a different schedule than US Chess ratings. Thus the FIDE ratings on a combined database can’t be assumed to be correct for FIDE-rated use.
Tom is correct, organizers and TDs who need correct and current/up-to-date FIDE ratings should get them from FIDE, not a third party source, just as organizers and TD who need correct and current/up-to-date US Chess ratings should get them from US Chess, not a third party source.
Because FIDE releases its ratings for a new month some two weeks after US Chess releases ratings for that month, by the time the US Chess supplement file becomes official any FIDE ratings in it would be obsolete.
We have been testing a JSON-formatted golden master file with additional information in it, such as all 5 US Chess ratings, (but not correspondence ratings), a longer name field, a status field (for things like duplicate IDs and deceased players) and some FIDE information (FIDE IDs, coiuntry codes and titles, but not ratings). Tom Doan (developer of WinTD) has been testing this file and Thad Suits (developer of SwisSys) is getting ready to do some testing of it, too. We hope to be able to put this file into production later this year.
FIDE publishes the FRL (FIDE Rating List) on the last day of each month. That FRL is in force for all tournaments beginning during the month (as is the official rating supplement published monthly by US Chess). I only publish the combined database after I have personally read the announcement on the official FIDE web site (http://www.fide.com/) that FIDE has published the FRL for the upcoming month. The ratings in the combined database are exactly those ratings published in that month’s FRL, just as the US Chess ratings in the combined database are exactly those published in the golden databases (both the one including quick ratings and the one including blitz ratings).
So, Mr. Doan, when might we see a version of WinTD that does not crash when it sees any DBF structure other than the exact structure of the US Chess golden database (including having exactly the same fields and no others but lenghtening the MEM_NAME field from 17 to 35 characters)?
I was addressing myself to Micah’s question about US Chess producing this. You’re releasing your version over a week after the GDB is mastered by US Chess.
Never. The DBF database format has needed to be replaced for years, not just tweaked. It looks like that will be happening relatively soon.
Please kindly refer to my reply to Mr. Doan to rebut the inaccuracy in this statement.
Yes, because FIDE publishes its FIDE Rating List (FRL) only on the day before it becomes active, I am compelled to wait to publish the next combined golden database until the last day of the month. (I invite the interested reader to note the dates on which I have published my announcement of the availability of the newest combined database.) Regrettably, this may make matters harder for those who run FIDE rated events (obviously not norm events) that use US Chess ratings for pairing and prize purposes.
I suppose one way around that problem would be to publish an interim combined database as soon as the US Chess golden master files are available (which would then contain “hybrid” ratings, namely US Chess ratings valid for the next month and FIDE ratings valid for the current month), but I do not envision a need to justify this complexity. If the consumers of my work ask for it, I will consider it.
I am also considering publishing a “shadow version” of the US Chess golden master files with all OTB US Chess ratings (regular, quick, and blitz) with an expanded MEM_NAME field (length 35 characters instead of 17, as 35 characters is long enough to contain the longest name I found in the “long names” I retrieved). (I am already doing this privately for the MetroWest Chess Club.) However, if there is a JSON-formatted master file on the way as Mr. Nolan describes, this too may not be worth the effort.
I’ve noticed that some players’ ratings sometimes have changed slightly on the FIDE website between the first of the month and the end of the month, even during a tournament. Unlike MSA, it doesn’t seem to indicate that these are unofficial changes and not effective until the next rating list. I don’t know that this has ever changed the categories for norm purposes for that player’s opponents, but it seems like it could.
Because we use FIDE ratings as part of the means to initialize US Chess ratings, we keep an up-to-date database of FIDE ratings by month. There’s an automated script that checks for new FIDE files (both the monthly ratings list file and the ‘all players’ file) every morning.
Recently it has seemed like there’s a new FIDE monthly ratings file nearly every day. I’m not sure if it gets updated over the weekend, though.
I’ve noticed this as well. However, I’m not convinced (yet) that published ratings actually change. If this is the “all players” file, I’m guessing it may reflect newly issued FIDE IDs (or perhaps mid-month title changes, such as a title that was approved conditional on the application being posted for sixty days taking effect).
I’m probably curious enough to do an experiment. Meanwhile, I’m still using the published FRL as announced by FIDE as the source for my combined database.
I’m not sure if both files come out every day, I’ll start tracking that.
Tom Doan recently noted that FIDE does not offer a very good API for querying their database. The ‘thin client’ interface that US Chess offers actually does include FIDE ratings, but since it is possible to query it for a specific point in time (eg May 2018 vs June 2018) it does not have the anachronistic issue that the rating supplement files do.
If you need someone else to help test this file, I’d be happy to do so. I also build a database of players with USCF ratings and FIDE ratings at the start of each month for my registration site. Currently I have to process several files in order to get the longer names. It’s cumbersome, but it works. It would also be nice to use the status code field to strip out obsolete player IDs from the list.