On anti-cheating in chess, science, reproducibility and variability

Abstract

Cheating in chess is a serious issue, mainly due to the use of chess engines during play. Chess engines like Stockfish or AlphaZero-like variant can give to the cheater a decisive advantage, since almost perfect moves can be played. Already in 2006, Topalov accused Kramnik as part of the world championship. Sébastien Feller case in Chess Olympiad in 2010 was a shock. With the rise of online games and $$$, cheating is even more problematic. A few weeks ago, Magnus Carlsen accused Hans Niemann (HN) of cheating over the board and refused to ever play him again. You may have heard headlines with “anal bead” supposed to help HN.

In fact, I’m not specifically aiming to talk about chess (and cheating). I rather want to discuss how science has been (and will be) at the heart of the anti-cheating chess problem. I will first argue that many people (chess hobbyists/experts, data nerds, etc.), most being non-scientists, have actually done science for trying to demonstrate or refute the cheating case. The basic idea is to confront moves played by humans (players) with those of computer engines. With the sharing of data (analysis of chess games like those played by HN), scripts, and methods, numerous results and conclusions have emerged, getting popularized with social media (twitch, Youtube, twitter, etc.) On the one hand, I’ve been quite excited to see all this energy for trying to advance our understanding and propose interesting ideas/analysis. On the other hand, there have been some failures in the quality of some analysis or the choice of closed systems to compute unclear metrics. In-between, there have been a report by chess.com and the analysis of the computer scientist Ken Regan the world renewed specialist.

I still think the problem is open (eg Regan’s method is too conservative and missing many cases; chess.com methodology, though unclear and opaque at some points, is certainly effective for online cheating, but not over the board detection). I will present a variability model of the space of experiments/methods that can be considered to address the problem. This model can be used to pilot the collaborative effort, to reproduce, replicate or reject some experiments, and to gain confidence or robustness in some conclusions.

Another last point I want to discuss is that most probably the anti-cheating chess problem cannot be resolved solely with retrospective computational analysis.

It’s just too uncertain, especially if cheaters are “smart”. (Cyber-)security experts, psychologists, chess players, and of course computer science nerds/professionals can contribute to address this multidisciplinary problem.