The thing is that LLMs can only ever go forward. It might go into one direction, once that direction is in the context it will realize that it doesn't work, but can't delete. So it has to backtrack (or double down, see the whole seahorse emoji breakdown).