Zompist Bboard Again

zompist wrote: ↑Fri Apr 18, 2025 2:47 am

I can't speak for him either but I'm reasonably familiar with procedural AI, and I'd be wary of claims that it could "reason". Certainly no system could reason in general as a human can. But procedural AI is basically following simple instructions (fast enough that you can do complex things) written by a human programmer. You can program rules of inference, but does the program "know how to reason", or only know how to follow carefully-laid-out instructions? It's reminiscent of Socrates' "demonstration" that Meno's slave knew the truths of geometry.

I'd also be wary of claims that procedural AIs can't reason. One could still maintain, in 2000 or so, that programmers could eventually produce a robust enough procedural AI to match human intelligence. As ever, this was expected 5 to 50 years in the future.

Ah, thank you!

zompist wrote: ↑Fri Apr 18, 2025 1:22 am

bradrn wrote: ↑Thu Apr 17, 2025 11:48 pm

keenir wrote: ↑Thu Apr 17, 2025 11:34 pm go ahead, tell me about an existing AI that can reason.

Many current AIs can do so. Not as well as (some) humans, admittedly, but they can do it.

I think an extraordinary claim like this needs extraordinary backing. Especially after you just said, correctly, "current AI benchmarks are badly misleading"!

Look up ‘chain-of-thought prompting’. The output generated by such methods certainly looks like reasoning, albeit reasoning which is clearly inferior to most human reasoning.

(I would like to say it’s inferior to all human reasoning, but, well, we live in a world where Trump is President…)

Going further than this requires defining what ‘reasoning’ is. I’m not sure I’m up to that intellectual challenge just at the moment (my brain is a little foggy), but I’ll keep on thinking about it.

The article you posted is interesting but not as skeptical as it could be... but it raises a good question: if an LLM can "pass the law exam", does that mean it can practice as a lawyer?

No. Its reasoning capabilities aren’t nearly good enough. And — as mentioned in the article — said ‘law exams‘ mostly test factual recall, not reasoning ability (which is what makes them ‘misleading’ to me).

Raphael wrote: ↑Fri Apr 18, 2025 1:52 am I can't speak for bradrn, but I'm not sure if he even meant LLMs. There might be other, older approaches in AI research that came closer to reasoning ability. Could you clarify, bradrn?

Yes, I was talking about LLMs. (Sorry for the ambiguity.)

Having said that, it’s interesting to compare to previous AIs. For instance, could ELIZA ‘reason’? For that matter, does a Prolog interpreter or a SAT solver ‘reason’? My intuition is that they do, in some weak sense — and the latter two more than the former, even — but as mentioned above I want to keep on thinking about this before making any definite assertions

bradrn wrote: ↑Fri Apr 18, 2025 5:29 am

Raphael wrote: ↑Fri Apr 18, 2025 1:52 am I can't speak for bradrn, but I'm not sure if he even meant LLMs. There might be other, older approaches in AI research that came closer to reasoning ability. Could you clarify, bradrn?

Yes, I was talking about LLMs. (Sorry for the ambiguity.)

Thank you for clearing that up!

bradrn wrote: ↑Thu Apr 17, 2025 11:25 pmI think the word you’re looking for is ‘agentic’: the ability to act as an agent under one’s one volition. I agree that there is a fundamental difference between agentic technologies and non-agentic ones, and that prior to AIs we haven’t really seen any technologies with a significant amount of agency. But the authors of that article are aware of that — indeed, they say that ‘for highly agentic systems, power and loss of control are tautological’ (which I agree with). This is why they, and I, focus on restricting the scope of AI agency — my point (c) — which I think is a very achievable goal.

Fine then. Why do we need agentic technologies at all? We could avoid the problem of keeping artificial agents (hard not to imagine the Matrix here) from going rogue by not creating them in the first place.

malloc wrote: ↑Fri Apr 18, 2025 7:28 am

bradrn wrote: ↑Thu Apr 17, 2025 11:25 pmI think the word you’re looking for is ‘agentic’: the ability to act as an agent under one’s one volition. I agree that there is a fundamental difference between agentic technologies and non-agentic ones, and that prior to AIs we haven’t really seen any technologies with a significant amount of agency. But the authors of that article are aware of that — indeed, they say that ‘for highly agentic systems, power and loss of control are tautological’ (which I agree with). This is why they, and I, focus on restricting the scope of AI agency — my point (c) — which I think is a very achievable goal.

Fine then. Why do we need agentic technologies at all? We could avoid the problem of keeping artificial agents (hard not to imagine the Matrix here) from going rogue by not creating them in the first place.

Firstly: it is very easy to not imagine the Matrix here. I don’t. That’s your problem alone. Besides, ‘going rogue’ is the whole thing we want to prevent here, by limiting what the agent can do in the first place.

But to be honest, I get a bit stuck here too. I tend to agree with zompist that human-level AIs aren’t terribly useful. But there are plenty of uses for what he calls ‘subsmart’ agents — things which could augment our intelligence or physical abilities in directed and controlled ways. Imagine, for instance, an automated lab assistant which can handle dangerous chemicals as well as humans. And that’s just one example, from a field I’m very familiar with.

bradrn wrote: ↑Fri Apr 18, 2025 8:34 am I tend to agree with zompist that human-level AIs aren’t terribly useful.

Repeating myself here, if they should ever become technologically feasible, they might be adopted as a fad, or as status symbols, even if they aren't that useful. How useful is caviar? How useful were Tamagotchis?

Raphael wrote: ↑Fri Apr 18, 2025 8:51 am

bradrn wrote: ↑Fri Apr 18, 2025 8:34 am I tend to agree with zompist that human-level AIs aren’t terribly useful.

Repeating myself here, if they should ever become technologically feasible, they might be adopted as a fad, or as status symbols, even if they aren't that useful. How useful is caviar? How useful were Tamagotchis?

I really think people haven't thought this through. Having an android sexbot might indeed be a fad... but not one that doesn't feel like having sex. Ditto for secretary, bodyguard, housecleaner, chauffeur, masseur, child companion, or whatever their purpose is.

Even if agentivity can be bypassed, one wonders if the techbros really want AI assistants who are far smarter than they are. A little smarter in one domain is one thing, but what if it's clear that the AI can do every aspect of their job better than they can?

zompist wrote: ↑Fri Apr 18, 2025 2:47 pm A little smarter in one domain is one thing, but what if it's clear that the AI can do every aspect of their job better than they can?

That sounds suspiciously like mallocism. (mallokism?)

keenir wrote: ↑Fri Apr 18, 2025 3:13 pm

zompist wrote: ↑Fri Apr 18, 2025 2:47 pm A little smarter in one domain is one thing, but what if it's clear that the AI can do every aspect of their job better than they can?

That sounds suspiciously like mallocism. (mallokism?)

Nah, because it only applies to CEOs. It's already uncanny how much ChatGPT can imitate a CEO. Their job is about the same: produce corporate-sounding bullshit and sound extraordinarily confident about it.

I wonder how long it will take before shareholders decide that AI's can 'increase shareholder value' better than human CEO's...

Raphael wrote: ↑Sat Feb 22, 2025 1:23 am Now, this doesn't even hold true among humans - someone who's very good at Chess can't even necessarily play Go all that well

Have you heard of the Pentamind? https://www.youtube.com/@mindsportsolympiad/videos

rotting bones wrote: ↑Fri May 02, 2025 4:11 am
Have you heard of the Pentamind? https://www.youtube.com/@mindsportsolympiad/videos

I hadn't, thank you!

The coffee-making test bothers me. I fear I would fail it. Even an instant coffee-making test might be hard.

Some of us seem quite good at failing the Turing test.

Horses as transport have niche rôles for policing and for hunting with dogs across fenced land. Of course, humans being reduced to niche rôles is consonant with Malloc's fears.

Weird article about chatGPT leading people to quasi-religious delirium.

malloc wrote: ↑Sun May 04, 2025 9:51 pm Weird article about chatGPT leading people to quasi-religious delirium.

Jorjani's Erosophia is a hyper-intellectual version of basically that IIRC.

malloc wrote: ↑Sun May 04, 2025 9:51 pm Weird article about chatGPT leading people to quasi-religious delirium.

Also, I feel like I'm losing my mind just from the news.

malloc wrote: ↑Sun May 04, 2025 9:51 pm Weird article about chatGPT leading people to quasi-religious delirium.

Now even the common man can speedrun his descent into having schizophrenic hallucinations! Thanks AI!

jcb wrote: ↑Mon May 05, 2025 1:46 am Now even the common man can speedrun his descent into having schizophrenic hallucinations! Thanks AI!

Is it better to find out sooner if it's going to happen anyway?

zompist wrote: ↑Fri Apr 18, 2025 2:47 pm I really think people haven't thought this through. Having an android sexbot might indeed be a fad... but not one that doesn't feel like having sex. Ditto for secretary, bodyguard, housecleaner, chauffeur, masseur, child companion, or whatever their purpose is.

Even if agentivity can be bypassed, one wonders if the techbros really want AI assistants who are far smarter than they are. A little smarter in one domain is one thing, but what if it's clear that the AI can do every aspect of their job better than they can?

Let's suppose building replicants were actually feasible... Then yes, we'd have production lines churning out millions of Nexus-6s right now, because tech companies are not good at thinking things through, plus there'd definitely be a market for slaves (because honestly, slaves is exactly what they'd be), following into, well, every robot story ever playing out.

(I'm more into Blade Runner than the Matrix )

I don't believe it's going to happen, though. A good heuristic is that science-fiction isn't very good at predicting the future.
I don't think SF ever predicted AI would be excellent bullshit generators, and nobody anticipated the issues with gen AI, such as enormous power use, or intellectual property theft.

Ares Land wrote: ↑Mon May 05, 2025 4:00 am A good heuristic is that science-fiction isn't very good at predicting the future.
I don't think SF ever predicted AI would be excellent bullshit generators, and nobody anticipated the issues with gen AI, such as enormous power use, or intellectual property theft.

Interesting observation. For US writers, I think there are a few reasons for this:

1. Writers usually focus on completed, working systems. Few people want to read about a time machine that doesn't work yet. So the AI is always imagined as human-level, the Mars colony as a working city, etc.
2. That damn John Campbell. He was a crank and you can trace his libertarianism and elitism and eugenics hobbyhorses in many of the writers he published.

One of the few '50s writers who creates believable futures, in my opinion, is Alfred Bester. Not in the SF details themselves, but in the picture of a complicated, confusing late-capitalist world. It helped that he didn't buy into Campbell's notions.

I think cyberpunk was pretty good at picturing corporate dystopia... again, not good on the details (no one understood that commerce, news, and entertainment would all move into cyberspace, and we don't have body-embedded computers and guns yet), but they nailed the corporate dominance and the degradation of ordinary life and politics. It could be argued that both Gibson and Bester succeeded by picturing "present-day America only more so".

I've maintained before that predicting 40 years in advance is not that difficult: SF writers normally get it wrong by assuming things will change far quicker than they do, and in the directions they fear or favor.

Right now, though, it feels like we all have Dr. Manhattan's problem in Watchmen: there are too many crisis points that are going to resolve one way or another in the next couple of decades, and it's all going to go horribly wrong or it won't. I don't think 2075 will just be 2025 but more so. There's climate change, for one thing. And as Paul Krugman likes to say, if things can't go on this way forever, then they will stop.