A Stopped Clock Is Always Wrong
This is a post about AI (actually just LLMs).
There is a common phrase in the USA that says "A stopped clock is right twice a day." It is used to mean something like, "Even this idiot with terrible takes will sometimes say something true/right/agreeable." However, I can't stand this phrase, for a reason that many would think pedantic and pointless. But now, in the ChatGPT era, I actually think it's extremely important for people to understand why this idiom is in fact wrong.
The problem with a stopped clock - that is, an analog clock which is unpowered and just shows a single static time - is that it is never correct. There are no times of day at which it is correct, even the exact time it is stopped at. Why? Because it cannot ever tell you what the current time is. A stopped clock transmits exactly 0 bits of information about the current time. The only way to know if it is displaying the actual time is to verify what it says by looking at another clock - which is just twice as much work as looking at a functioning clock in the first place.
Because it is untrustworthy, reading a stopped clock is always a negative value. Arguably, a stopped clock is hostile decor.
Now, there is room to disagree over the use of "correct" to mean "both accurate and useful," and I accept I am stretching the definition of "correct" a bit here. But I think it's acceptable to do so, because my definition relates to both: 1) the diction used in the idiom, and 2) the actual meaning of the idiom. So let's continue with that definition, because we have to accept the conceit for the rest of this post to matter. (If you refuse to roll with my definition in this context, you can stop reading. (Although the rest of the post doesn't really discuss that, the rest is the LLM stuff, so I've already wasted your time - sorry.))
So what does this annoying-to-me idiom have to do with LLMs? You may have guessed already, but LLMs are stopped clocks. And as a reminder: stopped clocks are always incorrect.
I'll explain. When you ask an LLM to describe something to you, it will respond with text that appears to be a description of that thing. In the same way, when you read a stopped clock, it will display a clock face that appears to be a description of the current time. But the LLM's response contains an unknowable number of errors of unknowable magnitude. That is, they're unknowable until you (thoroughly!) fact check each of its claims. Like a stopped clock telling you the current time is 10:20, the LLM's response has not actually given you any usable information. Even though there may be true information in its response, you have no way of knowing which bits are true and which are not, until you externally verify them.
Unlike the stopped clock, the LLM is actively pretending to have given accurate information, though. Many people accept the LLM response as completely true, even when they say otherwise. They may say, "I asked ChatGPT and it gave me this. There might be some inaccuracies but the broad picture must be correct."
These people are fundamentally misunderstanding how ChatGPT transmits information when they say the broad picture is correct. It is ~just as likely to give you "accurate details, but a fabricated summary" as it is the other way around. It can also give you all falsehoods, or even all true information! But you do not have any way of knowing which bits are true without external validation. A response from ChatGPT that is entirely true is exactly as useful as a response that is entirely false (not at all).
This is different from typical answers given by humans. Some human writing is "nonsensical" (basically, presenting things as connected which do not really have such a connection) like LLM writing, but the majority of what we consume from each other (discounting intentional deception) is not, and follows traceable logical inferences. We may draw the wrong conclusions from accurately reported details, but how we did so is generally legible. We may get everything wrong too! Human writing can sometimes be entirely non-credible. However, once again, the actual chain of reasoning is usually legible and we can learn something from inspecting it. In most cases, humans make mistakes in understandable ways -- plus, we can be taught how to not repeat those mistakes. And yes, it is bad for humans to receive inaccurate information from each other, which is why we've built complex, global systems designed to reduce the errors in what we communicate to each other!
This all goes out the window with LLMs. The summary they give you might have no relation at all to the details they identify. You can even see this very obviously with a lot of current chatbots which will put inline citations in their search-based answers. If you follow those citations, they almost always either don't mention the assertions they're supporting or even directly contradict it. (And in those cases where they do "support" it, even then it's a very good chance that it's the wrong class of evidence for the assertion: for example, an LLM will often only be able to find a single anecdote and will conclude that said anecdote represents broad consensus, which yes humans also do when they're unskilled at research, but which LLMs-as-research-assistants should specifically avoid.)
So even though I know it's useless and annoying to rail against a common idiom (even if that idiom is actually misleading), the reason that I take issue with this idiom is a very important lesson for everyone to learn today. Something that gives you fundamentally untrustworthy information is functionally not giving you any information at all. And, you're paying resources to receive no information (time, energy) because you need to verify every single bit it has given you.
This is why you consistently and remorselessly get downvoted when you leave a comment like "Here's what ChatGPT says on (this topic I don't understand)." You are only wasting everyone's time. There is no upside to you posting this comment. It doesn't help anyone or contribute anything valuable to the conversation. It's generally not possible for it to be a valuable addition to the conversation[1].
Does this mean you should never use ChatGPT? Well... I kind of think "yes," but not for this reason. Actually, LLMs are useful for certain kinds of tasks once you understand that this is the way they operate. All the above really only applies to the common use case "I want to know something about the world, and I'll ask a chatbot." I use LLMs occasionally during computer work myself, not because I need to get some true answer from them, but because they can surface concepts or names of things which can get me unstuck when I'm researching something. They can be wrong about what a term means while still helping, by giving me new ideas to search for. And that's what it comes down to: at the end of the day, we still have to do our own research. ChatGPT can't obviate the need to search for information.
It can't answer your questions, but it can help you to answer them yourself if you use it correctly.
... A stopped clock, however, has no such alternate use. Fix it or throw it in the trash. It can never, ever be useful to you.
[1] Actually it is in theory possible to paste a chatbot response in a forum and have it be useful, such as when it surfaces new concepts or terminology to the readers in the way I described above. But the chances of that are low enough that my general advice about uncritically pasting LLM outputs into conversations is "don't".