[GoLUG] LLMs can't reason

[Kyle Terrien]

On Fri, Jun 13, 2025 at 03:19:23AM -0400, Steve Litt wrote:
> I had the King Cobra in there not to introduce innebriation, but to
> introduce stupidity. So let me define stupidity a different way...
> 
> I was a bicycle mechanic in 1972, and rode a Sears three speed, with a
> basket, to work and all over Chicago. A customer came in and said to me
> "hey, you can get an employee discount on a 20 pound Bob Jackson bike,
> which is half the weight of your Sears, and then you'll go twice as
> fast."
> 
> Where do I even start? ChatGPT would NEVER make that mistake.

Even though I agree with everything else in the message (and I have
met my fair share of crazies who clearly missed out on high school
physics), I have to question the one assumption that ChatGPT would
never make *that* mistake.

There is a fundamental rule that you cannot give what you don’t have.
This is echoed in Wikipedia’s policy: “There is a saying that if the
prevailing opinion was that the earth was flat, then Wikipedia would
also say that the earth is flat.”

Most of these AIs were trained on web resources, especially web forums
like Reddit and StackOverflow.  So, their results are going to be
bounded by quality of the sources used for training.

If the users in those forums insist that there exists a O(N) sorting
algorithm, but it has been withheld from everyone because of a vast
conspiratorial plot to keep programmers employed, then ChatGPT is just
going to echo those thoughts, regardless of whether those thoughts are
true.  (They are not.)

Of course, that is an extreme (fictitious) example (or is it? 😉) But
think about how much bad advice is out there with regards to
e.g. shell scripts or GNU administration.

> In Miami in 1973 I knew a hippie who told me he and his girl friend
> used "astrological birth control". 
> 
> I said "you mean rhythm, right?"

Yep, I have run into some of them (although usually it’s “flat earth
conspiracy”; I have never heard the phrase “astrological birth
control” even though I am aware of the method that tracks hormonal
cycles).  I usually tell them that my astrological sign is a duck
because astrology is a quack, and usually try to give a quick physics
or biology lesson which of course goes over their heads.  The reaction
is usually either a stoned stare or anger and aggression.

--Kyle

-- 
[*] Kyle Terrien
    Terrenus => from the Earth, to the Cloud
    https://terren.us/

Dilexisti justitiam, et odisti iniquitatem.  -- Psalmus 44:8