If you were to wager on a chess match between the Atari 2600 and the AI capabilities of ChatGPT, which would you choose? The Atari 2600, a gaming console that debuted in 1977, boasts an aging 8-bit processor operating at 1.19MHz and a mere 128 bytes of RAM. According to infrastructure architect Robert Caruso, who shared his findings in June, anyone betting on ChatGPT would have ended up disappointed. Who could have predicted that a vintage game console could outperform modern AI like ChatGPT and Copilot in a game of chess?
Caruso remarked that “ChatGPT was thoroughly defeated at the beginner level.” He elaborated on how this surprising outcome unfolded, noting that despite being given a basic chessboard setup, ChatGPT misidentified rooks as bishops, overlooked strategic pawn forks, and frequently lost track of the positions of the pieces. The AI acknowledged its unexpected defeat and attributed it to the abstract representations of chess pieces used by the Atari. Yet, even after switching to conventional chess notation, the chatbot made enough errors to be ridiculed by a third-grade chess club. It seems the underpowered Atari 2600 has become a formidable opponent against AI rivals.
During a match that lasted an hour and a half, Caruso had to intervene repeatedly to prevent ChatGPT from making poor moves and to help it recognize the locations of both players’ pieces. At various points throughout the game, ChatGPT required its understanding of the board to be reset, and it insisted multiple times that it would perform better if they started anew. In the end, ChatGPT had no choice but to concede the game.
Interestingly, ChatGPT was the one to propose the chess duel. In a discussion about chess, it claimed to be a skilled player capable of easily defeating Atari Video Chess, which only considers 1-2 moves ahead. ChatGPT was eager to see how quickly it could secure a win. Instead, it faced defeat.
Earlier this month, the Atari 2600 faced another modern AI contender, Microsoft’s Copilot, in a chess match. Copilot was confident that it could outplay the Atari, asserting its ability to think 10-15 moves ahead. However, it opted to limit its strategy to 3-5 moves, reasoning that the Atari made “suboptimal moves.” Copilot also boasted that unlike ChatGPT, it made an effort to remember previous moves and maintain consistency in play, which should have ensured a smoother match.
However, the reality was different, as Caruso had to provide Copilot with a screenshot of the board after each move made by the Atari 2600. Copilot admitted to experiencing the same memory lapses as ChatGPT. Despite this limitation, Copilot claimed it could still analyze the board and make sound decisions. Ultimately, this was not the case, as the Atari 2600 secured its second consecutive victory against an AI opponent.
In a twist, Gemini, another AI developed by Google, decided against challenging the Atari 2600 after Caruso reached out to arrange a match. Like its predecessors, Gemini believed it was destined for an easy win. However, it soon acknowledged that it had overstated its chess-playing skills, leading it to the prudent conclusion that “canceling the match is likely the most time-efficient and sensible decision.”
Although Gemini never had the chance to compete against the Atari 2600, Caruso was impressed with the chatbot’s self-awareness regarding its limitations.
It may seem astonishing that a 48-year-old gaming console, relying on outdated technology, could outplay two advanced AI models and intimidate a third into canceling a match. However, this outcome may not be as surprising as it appears. Senior Software Engineer Kyle Witeck commented, “So…you’re comparing language models to a chess bot that was designed to play chess…classic…AI is not intelligent and not AI. It patterns and guesses from a system designed on language. Token-driven guesses.”
