LLMs inherit Dunning–Kruger effect?
It is obvious why LLM can't do it in a single iteration, but why it still can't count words in its previous messages (which can fully reference), yet correctly answers my message "How many words are in this message?"
I.e. it still has troubles correcting itself even when ran in a feedback mode. Maybe there is some bias towards AI considering its ideas infallible, somehow inherited from the training data?