AIs gunning for our precious freelancers

[malloc]

The opposite is also true, as mentioned -- trying to create an AI based on 'logic' doesn't get you far, and modern advanced AI's really only came about when people decided to abandon 'logic' as the basis of AI.

That seems genuinely counterintuitive and quite far from what science fiction has promised or threatened. What if not inherent rationality ensures that the superintelligent AGIs sought by the tech industry will avoid falling into nonsense?

The thing, though, is that mere calculators are not capable of anything resembling "intelligence". Advanced AI's as we know them are based on extensive training and pattern-matching rather than on simple calculation. They actually have to be specially programmed to handle things like numbers because normal modern AI techniques are much better suited to things like natural language and image processing/generation than to calculation.

Surely the ability to find the square root of massive numbers almost instantly qualifies as intelligence. No human however gifted can match that feat. Likewise the ability to beat any human at chess or other games we would generally consider intellectually difficult.

[bradrn]

The opposite is also true, as mentioned -- trying to create an AI based on 'logic' doesn't get you far, and modern advanced AI's really only came about when people decided to abandon 'logic' as the basis of AI.

That seems genuinely counterintuitive and quite far from what science fiction has promised or threatened.

Yes, this is very far from what SF (or at least 50s SF) has threatened. And why shouldn’t it be? We don’t have flying cars either, and everyone promised that too. SF is no guide to the future!

What if not inherent rationality ensures that the superintelligent AGIs sought by the tech industry will avoid falling into nonsense?

Same thing that stops us falling into nonsense: having a grounding in the real world, and having an ability to reflect on your own thought processes. Current LLMs don’t have much of either, which is why they do fall into nonsense quite regularly.

The thing, though, is that mere calculators are not capable of anything resembling "intelligence". Advanced AI's as we know them are based on extensive training and pattern-matching rather than on simple calculation. They actually have to be specially programmed to handle things like numbers because normal modern AI techniques are much better suited to things like natural language and image processing/generation than to calculation.

Surely the ability to find the square root of massive numbers almost instantly qualifies as intelligence. No human however gifted can match that feat.

No, that’s a mere matter of computing power. I can write a program to compute square roots using Newton’s method in, oh, let’s say about 50 lines of code. (Probably fewer.) Doesn’t mean that program is intelligent.

[malloc]

Same thing that stops us falling into nonsense: having a grounding in the real world, and having an ability to reflect on your own thought processes. Current LLMs don’t have much of either, which is why they do fall into nonsense quite regularly.

Except that plenty of people do fall into nonsense all the time. Conspiracy theories, bigotry, and so forth are massive problems that jeopardize the future of humanity. Despite the overwhelming evidence that vaccines are safe and effective for instance, millions of people oppose them.

No, that’s a mere matter of computing power. I can write a program to compute square roots using Newton’s method in, oh, let’s say about 50 lines of code. (Probably fewer.) Doesn’t mean that program is intelligent.

What do you consider intelligence, then?

[bradrn]

Same thing that stops us falling into nonsense: having a grounding in the real world, and having an ability to reflect on your own thought processes. Current LLMs don’t have much of either, which is why they do fall into nonsense quite regularly.

Except that plenty of people do fall into nonsense all the time. Conspiracy theories, bigotry, and so forth are massive problems that jeopardize the future of humanity. Despite the overwhelming evidence that vaccines are safe and effective for instance, millions of people oppose them.

Yes, you’re right. Humanity has its problems. My point is that a sentient AI will almost certainly have its own problems like these.

No, that’s a mere matter of computing power. I can write a program to compute square roots using Newton’s method in, oh, let’s say about 50 lines of code. (Probably fewer.) Doesn’t mean that program is intelligent.

What do you consider intelligence, then?

It’s one of those things which is very difficult to define. I think it includes traits like the ability for creative problem-solving and the ability to communicate, but I’m not even going to attempt to give a comprehensive definition beyond that.

Whatever the case, I know that it’s not this program:

Code: Select all

module Main where
import System.Environment (getArgs)
iter, sqrt' :: Double -> Double -> Double
iter a x = (x + (a/x)) / 2
sqrt' a x =
    iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $
    iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $
    iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a $ iter a x
main = do
    [x'] <- getArgs
    let x = read x' :: Double
    print (sqrt' x x)

(This is a Haskell program I wrote to calculate square roots. Despite how poorly it’s written, on my machine it can correctly calculate the square root of a 13-digit number in 0.012 s.)

[keenir]

Same thing that stops us falling into nonsense: having a grounding in the real world, and having an ability to reflect on your own thought processes. Current LLMs don’t have much of either, which is why they do fall into nonsense quite regularly.

Except that plenty of people do fall into nonsense all the time. Conspiracy theories, bigotry, and so forth are massive problems that jeopardize the future of humanity. Despite the overwhelming evidence that vaccines are safe and effective for instance, millions of people oppose them.

In programming terms, such things are usually (so far as I'm aware) phrased as "garbage in, garbage out."

[Raphael]

malloc seems to be a symptom of how people who know nothing about computers see computers.

That seems genuinely counterintuitive and quite far from what science fiction has promised or threatened.

Science. Fiction. Is. Not. Reality.

What if not inherent rationality ensures that the superintelligent AGIs sought by the tech industry will avoid falling into nonsense?

Nothing, which is why they don't avoid it.

Surely the ability to find the square root of massive numbers almost instantly qualifies as intelligence. No human however gifted can match that feat. Likewise the ability to beat any human at chess or other games we would generally consider intellectually difficult.

Well, that's the naive view of "intelligence" that people had in the early days of computers. They thought "intelligence" was all about doing things that are difficult for very smart people. After a ton of research along those lines, it turned out that a lot of "intelligence" is more about doing things that are easy for most people. Only very recently have computers gotten somewhat good at some of those tasks, often requiring a lot more energy for them than human beings.

[malloc]

Nothing, which is why they don't avoid it.

That would imply that something can claim the mantle of superintelligence and yet still do stupid stuff.

Well, that's the naive view of "intelligence" that people had in the early days of computers. They thought "intelligence" was all about doing things that are difficult for very smart people. After a ton of research along those lines, it turned out that a lot of "intelligence" is more about doing things that are easy for most people. Only very recently have computers gotten somewhat good at some of those tasks, often requiring a lot more energy for them than human beings.

If something comes easily to even the dullest person then by definition it requires little intelligence I would say. For that matter, it seems like computers today have actually mastered most intellectual tasks, everything from mathematics to playing mental games, to drawing pictures and composing texts. Currently it seems difficult to think of many intellectual tasks that only humans can still do.

[Raphael]

Nothing, which is why they don't avoid it.

That would imply that something can claim the mantle of superintelligence and yet still do stupid stuff.

Yes. Contrary to common belief, intelligence and stupidity are by no means mutually exclusive. (I think I'm a reasonably smart person, and I've sure done my share of stupid things in my life.)

If something comes easily to even the dullest person then by definition it requires little intelligence I would say.

And that's exactly where you are wrong. Take, for instance, moving my hands and fingers in the way I have to in order to type this post. Easy, right? Yet until fairly recently, it was incredibly difficult for robot hands. And that was at times, I think, when computers could already beat champion chess players. The nitty-gritty of everyday life requires more brain power than stereotypical "smart" things.

[malloc]

And that's exactly where you are wrong. Take, for instance, moving my hands and fingers in the way I have to in order to type this post. Easy, right? Yet until fairly recently, it was incredibly difficult for robot hands. And that was at times, I think, when computers could already beat champion chess players. The nitty-gritty of everyday life requires more brain power than stereotypical "smart" things.

I would hardly call simple muscular movement, however precise or useful, intelligence. Even the humble housefly can maneuver in the air with incredible precision yet nobody considers them intelligent. Most people at least would characterize intelligence as abstract mental activity, not simply anything the physical brain does. You might as well count sleep or secreting thyroid stimulating hormone as examples of intelligence by your definition. When it comes to abstract mental activities like mathematics and processing language, computers have almost matched humans.

[Raphael]

I would hardly call simple muscular movement, however precise or useful, intelligence. Even the humble housefly can maneuver in the air with incredible precision yet nobody considers them intelligent. Most people at least would characterize intelligence as abstract mental activity, not simply anything the physical brain does. You might as well count sleep or secreting thyroid stimulating hormone as examples of intelligence by your definition. When it comes to abstract mental activities like mathematics and processing language, computers have almost matched humans.

You're basically gerrymandering the definition of "intelligence" to be all about things computers are very good at (and were already very good at long ago), and then you make a big deal of the fact that, under that definition, computers are very intelligent. That's pretty much a form of playing word games.

Oh, and if you're looking specifically at people who are unusually bad at basic arithmetics, they might be beaten at calculating basic stuff by a centuries-old abacus. Does that mean that abacus is smarter than a bad-at-math human being?

[malloc]

You're basically gerrymandering the definition of "intelligence" to be all about things computers are very good at (and were already very good at long ago), and then you make a big deal of the fact that, under that definition, computers are very intelligent. That's pretty much a form of playing word games.

No, I am using the term the way the vast majority of people use it. If you find typing or other basic physical activities intellectually demanding, that is your problem. Meanwhile the vast majority of people associate intelligence with Einstein, not stenographers with their impressive WPM nor insects with their nimble flight.

Putting it another way, the world is not ruled by those can type fast or punch hard but rather by those who can order others to type and punch in clever ways. The ability to control your fingers does not translate into uncovering the secrets of the universe or wielding substantive power over the course of civilization.

Oh, and if you're looking specifically at people who are unusually bad at basic arithmetics, they might be beaten at calculating basic stuff by a centuries-old abacus. Does that mean that abacus is smarter than a bad-at-math human being?

Maybe in some sense, although the abacus requires conscious effort and even skill to use, especially for more complex forms of arithmetic. Finding the cube root of any number is trivial with a computer yet not with an abacus.

[Travis B.]

The big thing is that the kinds of things humans do on an everyday basis are actually far more computationally intensive than merely crunching ordinary mathematical equations. The human brain is a far, far more powerful computer than any man-made computers that exist today. It just happens that it is not optimized for mere number-crunching -- humans and their forebears have largely had no evolutionary need for number-crunching. It also happens that trying to emulate human intelligence similarly produces systems not optimized for number-crunching unless such a capability is specifically programmed into them. Attempting to emulate human intelligence through what is essentially number-crunching turned out to be a dead end; one simply cannot achieve 'intelligence' that way.

[Raphael]

You're basically gerrymandering the definition of "intelligence" to be all about things computers are very good at (and were already very good at long ago), and then you make a big deal of the fact that, under that definition, computers are very intelligent. That's pretty much a form of playing word games.

No, I am using the term the way the vast majority of people use it.

Well, that's not the way serious researchers into intelligence and related fields use the term. At least not any longer today; they might have used your understanding when those fields were new.

The human brain is a far, far more powerful computer than any man-made computers that exist today. It just happens that it is not optimized for mere number-crunching

That is, conscious number-crunching. Unconsciously, catching a ball someone throws at you, or escaping from human hands if you're a fly, requires fairly advanced math.

[malloc]

Well, that's not the way serious researchers into intelligence and related fields use the term. At least not any longer today; they might have used your understanding when those fields were new.

So the situation is much like how physicists use "work" to describe something quite distinct from the average person. Then let me clarify that I am talking about intelligence in the informal sense of skill at abstract mental activity. Regardless of what these researchers are seeking, I don't find it particularly useful to include everything the brain does under the umbrella of intelligence. That includes too much stuff that clearly has no relation to intelligence in the sense most people would understand it and leads to absurd conclusions like superintelligent houseflies or getting an erection as an expression of intelligence.

[bradrn]

Well, that's not the way serious researchers into intelligence and related fields use the term. At least not any longer today; they might have used your understanding when those fields were new.

So the situation is much like how physicists use "work" to describe something quite distinct from the average person. Then let me clarify that I am talking about intelligence in the informal sense of skill at abstract mental activity.

Well, in that case — and like I already showed you — you’re beaten by a 11-line-of-code program. That doesn’t seem like a useful definition of intelligence to me!

More widely speaking, in terms of raw abstract skills, computers have had humans beat for years. Software like Mathematica can manipulate ten-line-long mathematical expressions in milliseconds, with perfect accuracy. SMT solvers like Z3 can solve logical problems which could take months for the best humans. If you define ‘intelligence’ to include everything where computers beat humans and nothing where humans beat computers, how is that a useful definition?

Regardless of what these researchers are seeking, I don't find it particularly useful to include everything the brain does under the umbrella of intelligence. That includes too much stuff that clearly has no relation to intelligence in the sense most people would understand it and leads to absurd conclusions like superintelligent houseflies or getting an erection as an expression of intelligence.

There is a difference between instincts and intelligence. Those things you described are purely instinctual — they might as well be hardcoded. Intelligence is a different story.

Let me perhaps put it another way. Travis recently mentioned the Coffee Test — to go into a random kitchen, unknown to you, and make yourself a coffee. Leaving aside extraneous comments (which we discussed at the time), what skills does this require? You need the ability to locate unknown objects (without necessarily knowing their precise shape or location, which may be hidden), to work out how to use them, and to create a sequence of logical steps to achieve your goal given only what is available. This clearly requires intelligence. Not the same kind of intelligence as that required by abstract thought, but still intelligence of a different kind. And it is this kind of intelligent problem-solving ability which computers are most clearly lacking.

[malloc]

Well, in that case — and like I already showed you — you’re beaten by a 11-line-of-code program. That doesn’t seem like a useful definition of intelligence to me!

More widely speaking, in terms of raw abstract skills, computers have had humans beat for years. Software like Mathematica can manipulate ten-line-long mathematical expressions in milliseconds, with perfect accuracy. SMT solvers like Z3 can solve logical problems which could take months for the best humans. If you define ‘intelligence’ to include everything where computers beat humans and nothing where humans beat computers, how is that a useful definition?

Certainly my definition makes humans look terrible in comparison but that hardly makes it useless or flawed. If flies could talk they could complain your definition of intelligence makes them look stupid after all. The purpose of definitions is not coddling the pride of anyone, whether human or housefly.

My main concern these days is how humans are supposed to maintain access to prestigious and influential positions within civilization when they have to compete with computers now. People make much of the fact that humans can operate coffeemakers while robots cannot. Perhaps all the tasks included in that chore constitute intelligence in their own right. Nonetheless, it is not baristas who run civilization but administrators and scientists and so forth. Once computers take over all those positions, humans will find themselves excluded from effective control over civilization, on the same level as oxen or pets.

[bradrn]

Certainly my definition makes humans look terrible in comparison but that hardly makes it useless or flawed. If flies could talk they could complain your definition of intelligence makes them look stupid after all. The purpose of definitions is not coddling the pride of anyone, whether human or housefly.

I guarantee you that if you asked 100 people ‘are humans currently smarter than computers?’, 100 of them would say ‘yes’. If you want to use a definition which disagrees with that used by practically everyone else, I suppose you’re free to do so, but just make sure to keep the distinction in mind.

(Also, let’s not take our analogies too far. Flies can’t talk — indeed they have no method of communication beyond scent. That’s one of the reasons we consider them less intelligent than humans.)

My main concern these days is how humans are supposed to maintain access to prestigious and influential positions within civilization when they have to compete with computers now. People make much of the fact that humans can operate coffeemakers while robots cannot. Perhaps all the tasks included in that chore constitute intelligence in their own right. Nonetheless, it is not baristas who run civilization but administrators and scientists and so forth. Once computers take over all those positions, humans will find themselves excluded from effective control over civilization, on the same level as oxen or pets.

And yet… running a civilisation probably has as much in common with being a barista as it does with mathematics. Both require skills such as flexibility in changing conditions, and an ability to communicate with people (who may be themselves tired or impatient or whatnot). People who hold fast to complicated abstract theories generally make terrible leaders.

[malloc]

I guarantee you that if you asked 100 people ‘are humans currently smarter than computers?’, 100 of them would say ‘yes’. If you want to use a definition which disagrees with that used by practically everyone else, I suppose you’re free to do so, but just make sure to keep the distinction in mind.

But what criteria would they use to reach this conclusion? What exactly would they cite that gives humans an edge over computers? If one considers people like Einstein the gold standard of intelligence, how do humans outperform computers?

And yet… running a civilisation probably has as much in common with being a barista as it does with mathematics. Both require skills such as flexibility in changing conditions, and an ability to communicate with people (who may be themselves tired or impatient or whatnot). People who hold fast to complicated abstract theories generally make terrible leaders.

Sure but smart enough computers could legitimately claim that they are more qualified to run civilization than us. There will come a point where we must decide whether to let suboptimal entities continue to run the show or hand the reins of power to someone more qualified. My honest opinion is that we really ought to avoid that crossroad since it yields ugly results for us either way. All those tech CEOs really ought to consider their own futures in a world where computers can manage corporations and create software far better than humans.

[bradrn]

I guarantee you that if you asked 100 people ‘are humans currently smarter than computers?’, 100 of them would say ‘yes’. If you want to use a definition which disagrees with that used by practically everyone else, I suppose you’re free to do so, but just make sure to keep the distinction in mind.

But what criteria would they use to reach this conclusion? What exactly would they cite that gives humans an edge over computers?

Probably more or less the same things I’ve cited: problem-solving ability, communication skills, the ability to utilise unknown objects. Plus, of course, there’s the fact that computers have not taken over the world, while humans have.

And yet… running a civilisation probably has as much in common with being a barista as it does with mathematics. Both require skills such as flexibility in changing conditions, and an ability to communicate with people (who may be themselves tired or impatient or whatnot). People who hold fast to complicated abstract theories generally make terrible leaders.

Sure but smart enough computers could legitimately claim that they are more qualified to run civilization than us. There will come a point where we must decide whether to let suboptimal entities continue to run the show or hand the reins of power to someone more qualified.

Perhaps, yes. But I think that will be a long, long time in the future. Besides, we’d have plenty of warning.

[Ares Land]

But what criteria would they use to reach this conclusion? What exactly would they cite that gives humans an edge over computers? If one considers people like Einstein the gold standard of intelligence, how do humans outperform computers?

The idea of humans outperforming computers, or computers outperforming humans sounds is a very capitalistic way of looking at the world. You're more of a socialist, you don't have to adopt this unhealthy train of thought.

Human beings don't need to perform, let alone outperform anything to have value.

There's a take I like about human civilization (attributed to Margaret Mead, but apparently apocryphal): it beings when we see human beings caring for the wounded(1). That's the measure of a human being not, I don't know, chess.


(1) by that measure, we arguably still haven't reached the civilized state, and incidentally I do believe we should start caring for the wounded more than obsessing about the next tech toy.