Training Chess Engines

This paper has a very impressive result: They got a neural network based program to play chess at grandmaster level with no search whatsoever. While technically speaking there’s plenty of opportunity for tactical analysis to be happening here that’s all going on within the neural network and not using the custom written code for it.

The approach used is to take a corpus of chess positions, evaluate them using Stockfish (the world’s strongest custom chess engine) and then train a neural network to replicate those evaluations. This sounds like a simple idea, but it’s hard to get the details right and it requires a lot of computational power to actually do it.

The next step for this is to make it a ‘zero’ engine, meaning that it doesn’t have any training based off custom written software. First you start with the engine being random. Then you plug that in as the evaluation function into an alpha-beta pruner which has the extra board evaluation function that it detects when the game has been won. That will spit out board evaluations which, while still very poor quality, are more accurate than random numbers. You then train a new neural network on those numbers. Then you switch to the other corpus of chess positions (you need to alternate between two of them) and repeat the process again. With each generation the neural network will become stronger until it hits the same level it got with the non-zero approach.

Using positions from human or Stockfish games for the corpus of positions should work fine but it seems more appropriate to have the engine generate positions for itself. A reasonable approach to that would be each time an engine is generated it plays a set of games against itself and the positions from those are the alternate set of positions to be used in the generation after the next one.