Free as in Chess! Part I or how to make Chess Databases

Free as in Chess (part I!) or How to make Chess Databases



This post is about creating a strong, dependable, open and free Chess software suite. Chess can get pretty expensive for dedicated players as they need a strong engine to analyze their games and database software to store their games, add their own comments, create opening reports, check the opening repertoire of their opponents, solve complex endgames, analyze their opening novelties and cross check their analyses with recent top level games.

My aim is to demonstrate from the simple user’s perspective that there is no need to spend a single penny on proprietary software. All the software here is free and there exist downloadable binaries, no need for compilation. Also all software is cross platform, which means you can use it no matter if you like working on Linux, Mac or Windows.

Occasional players, who just want to play a game against a computer now and then, are fine using GNU chess and need to read no further :), the rest of the text is about obtaining impressive free chess software and how to use it. In this part, will cover how to create a database of chess games. In the next parts I will cover how to use the databases, create opening reports, use strong free engines that rank ~2900 Elo to annotate your games and follow the opening developments. In the following parts I will also cover how to make an opening repertoire, practice it and play training matches against the computer, play online chess and use free software for correspondence games. All that using only free software.

My real wish is that someone who runs a chess club or a chess federation reads this and uses the material in order to boost the performance of his club/federation. Imagine how good could it be if all youngsters could get a *free* software suite that has nothing to be jealous of from commercial programs like Chessbase.


The database program we are going to use is SCID (Shane’s Chess Information Database), which can be downloaded from here for windows and linux users or if you are a mac (woot!) user, click here !, download it and install it.

SCID comes, at least in the mac version, bundled with the extremely strong Toga II, engine, which is ranked 4th in the 2008 world championship, drawing with the world computer champion Rybka and has an Elo rating of 2868. Toga II is 100% on par with the best proprietary engines out there, Rybka excluded. If 2868 is not enough, then add Rybka, not open-source but freeware and is rated 3111, grab it from here, I strongly recommend in favor of Toga II as it is free as in freedom, not free as in beer. In the following parts I will cover more free engines, how to tweak them and how to incorporate them into SCID.

In order to create a Database using free software, first we need game collections which the software will manipulate. Some very nice databases are available for download, like the Million Database (1.7 million games!), high quality databases of selected games by Norm Pollock, New In Chess Yearbooks and This Week in Chess Issues.

Other databases worth looking into are ChessOK, ICCF archives, RusBase.

To download multiple .pgn files (e.g. from TWIC) fast use Firefox and DownThemAll!.

Before continuing, if you want to customize your chessboard, go to options->Chessboard.



After downloading these databases, open up SCID. We are going to make a SCID database for the Million game DB and the TWIC issues, the other databases can be constructed in a similar manner. At the end we are going to generate an opening report for Fischer’s defense in King’s Gambit accepted to provide a taste of what is to follow in parts II, III, IV and V !

In SCID, go to Windows->Database switcher.



This is probably the most important SCID window and it is likely that you’ll want it open all the time you are using SCID. It allows you to switch databases, copy games from one database to another and copy games to the clipboard where you will be doing all your searches, reports etc.

Now it is time to create a new SCID database for the Million games pgn file. Go to File->New as shown in the picture,



and name this database MillionDB.



Now go to File->Open



and select the Million database (using a pgn selection) as seen in the pic,



and you should get the following loading window



now, in the database switcher, drag (while left clicing) the pgn opened to the Million DB



voila!

Let’s see how to handle the TWIC database. First uncompress the .zip files, they should give .pgn files. As simple shell command does the tric, cat *.pgn > alltwic.pgn. Windows don’t have a unix shell installed by default but it is quite easy to install one, just grab cygwin.
Now similarly to the Million DB, create an new SCID database called TWIC, load alltwic.pgn
and drag and drop the loaded pgn file to TWIC database.

Now to provide a taste of what SCID can do we are going to see it create an opening report on the Fischer defence for King’s Gambit Accepted using the TWIC database. First drag and drop the TWIC database (holding the left click mouse button!) to [clipbase], which is where all the user processing should take place. Then click on the [clipbase] icon (selectes clipbase as the working database) and after that go to Windows->Tree Window

,

The tree window should now appear to the right,



In order to reach the Fischer defence click on the e4 option,



then click on the e5 option and so on until the Fischer defence Tabiya is reached




Now we are ready to create an opening report! Go to Tools->Opening report



and you should see the Generating report bar



voila! the report should now be right there



One SCID feature I am particularly fond of is the ability to generate LaTeX reports, like the following





To see the latex output of the opening report, you need a tex installation, for macs the one I use is TeXShop.


Now we are going to add player pictures & ratings to SCID so that our interface looks nice & has an up-to-date player information. These can be downloaded from here but today (5 Jul ‘09) this link contains an updated version of the ratings file. Please note, as the SCID website says, “You will only need ratings.ssp instead of spelling.ssp if you want to use the Add Elo ratings maintenance command in Scid” and regarding the photos “Scid can show a photo whenever a game by a player with a photograph is loaded. To install the photos, download photos.zip and extract its files to your Scid user options directory (~/.scid in Unix; the same directory as scid.exe in Windows) and run Scid. The Zip file contains two player photo files: gm.spf has photos for nearly 100 grandmasters and other famous players, and historic.spf contains more than 30 historic (pre-1930s) players.”.

To update the Elo ratings in all your databases, select the database you want from the database switcher window, click options->Select spellcheck file and select the file you downloaded



don't forget to do what this little popup says



and then go to the Windows->Maintenance Window



and click update Elo ratings.

.


Now you have updated the Elo ratings of your databse.


Alot more to come in the following parts: how to use the databases you created, create opening reports, use strong free engines to annotate your games, how to make an opening repertoire using SCID, practice it and play training matches against the computer, play online chess and & correspondence games. All that using only free software! no need to get one of those totally overpriced proprietary licenses