I don't know, take your own conclusions, I tried what I tried with the results I got. And the reason I created a Monte Carlo Engine to play the game was specifically because of this, I expected ChatGPT to be able to make moves but actually not being good with the game. You can try yourself, the code is available.
> Also in this game if I don't move the queen I force a draw, right?
I don't know as there is no time but I assume it is mandatory to move when, what happens in a chess game with no time if someone does not want to move? Same applies here.
The API cost of the game is getting noticeable, but I think you were just being naive about LLM limitations, there is simply no way that it can answer all questions simply by memorization. A simpler way is to just invent a programming language and ask the model to solve problems with it, at least I don't have to write down the position of a game ahah
Also I have trained models to do additions in the past, I removed many possible combinations of digits to show in the dataset, but after training the model was able to solve all of them, meaning it learned the algorithm and not just memorized the answers, I did it because a friend of mine thought like you that LLMs just memorize the answers from the dataset and cannot learn, but that is not how they work.
About the game, I realize that I cannot move the queen like in chess, so in this game I will eventually fall into a zugzwang, trying not to move the queen.
I was not being naive, as I said above I was expecting the model to be able to move but not to be good. That is why I created an engine to play against, to see how good/bad it is.
My exact entire point was that it would be really surprising that chatgt only knowing chess rules could play at 1400 ELO level.
And I did not give a crap about how to play a single game and was not meant to see how smart HN users are but how chatGPT is, but whatever conclusion a human would take without playing a single game (not trying to move the queeen) is what chatgpt should do, right?
BTW, I simulated few games of the engine against the engine to see what happened and it does not seem as obvious as you say.
I don't know, take your own conclusions, I tried what I tried with the results I got. And the reason I created a Monte Carlo Engine to play the game was specifically because of this, I expected ChatGPT to be able to make moves but actually not being good with the game. You can try yourself, the code is available.
> Also in this game if I don't move the queen I force a draw, right?
I don't know as there is no time but I assume it is mandatory to move when, what happens in a chess game with no time if someone does not want to move? Same applies here.