AIs gunning for our precious freelancers

[zompist]

On the first half: indeed there are multiple forms of intelligence. (Earlier in the thread I gave examples from my own experience!) At the same time, when we say that ‘so-and-so is smarter than me’, it’s pretty clear what is meant. There definitely seems to be some factor, primarily relating to skills in abstract manipulation and language, which corresponds to the common notion of ‘intelligence’. And it seems clear that that factor affects the abilities of people to do things. I cannot do mathematics like, say, Terry Tao; equally, other people have told me they can’t do the work I do.

All this is quite compatible with intelligence being a bundle of miscellaneous abilities.

An analogy might be whether or not a country is "advanced" or "developing". Surely there's no notion that this is a single factor. There's a bunch of stuff involved: education levels, health levels, emancipation of women, knowledge of advanced tech, manufacturing base, corruption levels, competent government, ability to start businesses easily, lax bankruptcy laws, sufficient investment funds, access to markets, road/railroad/port readiness, a history of prior entrepreneurship, a balance of free trade and protectionism, wage level, not being too much in thrall to superpowers. Many of the factors co-occur in Europe/US/Japan, but it's easy to see that (say) the number of college graduates is not the same as the number of railroads. You can most easily see problems when one factor is missing, or for that matter overdeveloped.

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities. There are people with near-perfect memory, others whose short-term memory is shot. There are people with extraordinary math ability who are highly disabled in other areas. On a more quotidian level there are people who are great with computer languages or chess but can't write a letter to save their lives, or people who are experts on literature and language who are baffled by technology. Some people can visualize 3-D objects in their minds and manipulate them like a CAD program... I wish I had that one myself. People with damage to Broca's or Wernicke's area have problems with language but can handle complex physical tasks— it would be bold and dismissive to call them "unintelligent."

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Another example might be "beauty". People may kind-of-concur on what makes someone beautiful— but it would be absurd to come up with a "beauty quotient" or rank everyone 1 to 8 billion. People don't agree that closely, and there's a huge cultural component. As there is for intelligence: the skills that impress a 21st century American or Australian are not the exact same set as would have impressed a 19th century one, or Homer, or a hunter-gatherer.

[zompist]

(It's easy to find estimates for the number of parameters in LLMs, hard to find estimates for the number of nodes; but it'll be far smaller. The number of neurons in the human brain, BTW, is 100 billion. Funny how whenever you look at biological systems, you underestimate them by several orders of magnitude.)

But most of those are found in the cerebellum. The cerebrum, which does all the actual thinking, only has around fifteen billion.

You are out of date. Here's an interesting story of a man with no cerebellum. tl;dr: He is functional, but disabled in many ways, including thinking and emotional tasks.

First, [scientists] showed that [the cerebellum] has connections to brain areas that perform higher functions, like using language, reading maps and planning. Then, a few years ago, researchers began to do functional MRI studies that suggested that the cerebellum was actively involved in these tasks.

"The big surprise from functional imaging was that when you do these language tasks and spatial tasks and thinking tasks, lo and behold the cerebellum lit up," Schmahmann says.

[Raphael]

All this is quite compatible with intelligence being a bundle of miscellaneous abilities.

An analogy might be whether or not a country is "advanced" or "developing". Surely there's no notion that this is a single factor. There's a bunch of stuff involved: education levels, health levels, emancipation of women, knowledge of advanced tech, manufacturing base, corruption levels, competent government, ability to start businesses easily, lax bankruptcy laws, sufficient investment funds, access to markets, road/railroad/port readiness, a history of prior entrepreneurship, a balance of free trade and protectionism, wage level, not being too much in thrall to superpowers. Many of the factors co-occur in Europe/US/Japan, but it's easy to see that (say) the number of college graduates is not the same as the number of railroads. You can most easily see problems when one factor is missing, or for that matter overdeveloped.

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities. There are people with near-perfect memory, others whose short-term memory is shot. There are people with extraordinary math ability who are highly disabled in other areas. On a more quotidian level there are people who are great with computer languages or chess but can't write a letter to save their lives, or people who are experts on literature and language who are baffled by technology. Some people can visualize 3-D objects in their minds and manipulate them like a CAD program... I wish I had that one myself. People with damage to Broca's or Wernicke's area have problems with language but can handle complex physical tasks— it would be bold and dismissive to call them "unintelligent."

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Another example might be "beauty". People may kind-of-concur on what makes someone beautiful— but it would be absurd to come up with a "beauty quotient" or rank everyone 1 to 8 billion. People don't agree that closely, and there's a huge cultural component. As there is for intelligence: the skills that impress a 21st century American or Australian are not the exact same set as would have impressed a 19th century one, or Homer, or a hunter-gatherer.

Another analogy might be basic physical fitness. A marathon runner, a short-distance runner, and a boxer are all physically fit, but in different ways.

[malloc]

You are out of date. Here's an interesting story of a man with no cerebellum. tl;dr: He is functional, but disabled in many ways, including thinking and emotional tasks.

Sure but the story also says that "the cerebellum really has just one job: It takes clumsy actions or functions and makes them more refined" and "it doesn't make things. It makes things better". The cerebrum does all the actual thinking and feeling while the cerebellum refines rough drafts of thought and feeling. Despite having eighty percent of the neurons in the brain, it certainly isn't conducting eighty percent of cognition by any means.

[bradrn]

Imagining a super intelligent AGI, what could it do that humans couldn't?

It is impossible to imagine what it’s like to be an AGI. This is a very general point, which I think I’ve made before: it’s impossible to imagine what it’s like to be of a higher intelligence than you are already, because that would be tantamount to you yourself having that amount of intelligence. (That Vinge quote I gave earlier says basically the same thing.)

On the other hand, perhaps it may be possible to imagine what a superintelligence would be able to do, ‘from the outside’. But a lot of people have tried, and responses range from ‘not much at all’ to ‘the end of humanity’. Personally, I’m convinced only that a superintelligence would be completely unpredictable by humans such as us. (This is basically the same opinion Vinge ended up with, I believe.)

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Precisely my point!

An analogy might be whether or not a country is "advanced" or "developing". […] Another example might be "beauty".

Ah, these two are good examples, thanks!

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities.

As mentioned in my post I gave examples of this from my own experience, though of course my case is not as extreme as those you mention.

[rotting bones]

it’s impossible to imagine what it’s like to be of a higher intelligence than you are already, because that would be tantamount to you yourself having that amount of intelligence. (That Vinge quote I gave earlier says basically the same thing.)

I think I can imagine what it would be like to be more intelligent. This is part of why I don't believe in intelligence as it's imagined in our collective mythology, and I think celebrating human intelligence is part of a campaign to scam narcissists:

1. If I had more spacious, more reliable and/or faster working memory, it would let me work things out in my head that I painfully grope around for. I think this is what most people imagine when they want more intelligence.
2. Sometimes, instead of working things out, the brain uses rapid computation. Most often, it does this by contrasting the relative credence of multiple possibilities. If I could do this more reliably and/or quickly, it might let me debias my mental models sooner.
3. Personally, the mental superpower I want is to have more imagination for constructions in working towards a solution. I'm thinking of geometric constructions and gadgets in complexity theory: https://en.wikipedia.org/wiki/Gadget_(computer_science)

Here's the thing: All of these abilities can be trained by practice. E.g. You can train many aspects of the first two by practicing solving the Einstein's Riddle app faster. The version by Rottz Games in the Play Store has contests designed to help you improve.

How this is not the same as intelligence:

1. IQ is defined as the things you can't get better at by practice. This means the things people imagine they would be better at if they had more intelligence is not the same thing as IQ. And yet, practice does rework neural connections.
2. I have occasionally, half-heartedly trained myself to get better at some of these things. It doesn't feel like there's music playing in your head a la Girl Genius. It doesn't feel like being more intelligent or having a superpower. It doesn't feel like you can take over the world. In fact, the most impressive of these skills barely transfer to real world problems. What it feels like is that you are better at doing sums. This is not useless. It also doesn't correspond to romantic notions of ~intelligence~. It feels like being an accountant. If you feel like I'm being overly reductive, Bertrand Russell (IIRC) discusses how the idealist notion of intelligence originating in Plato boils down to the ability to do sums. Being better at sums does let you solve some problems, and it does feel annoying if everyone else is really bad at them.
3. Even after I'm a better accountant, I will still feel all the negative emotions that plague my existence: https://www.youtube.com/watch?v=JrsB1RfksEA

[rotting bones]

f you have N inputs, the number of possible truth tables, and thus the number of possible "logic gates", is N^(N^2). For N = 2, this equals 16, which is easily small enough to hold in your mind, or to give each of them a catchy name. Now see how the possible number of truth tables - a useful measue of the complexity - increases with N:

N possible truth tables
3 19683
4 4294967296 (4.3e9)
5 close to 3e17

And that's only with digital inputs and outputs. If anything, zompist's visual illustration is significantly understating the issue.

I see this kind of reasoning in a lot of books on cybernetics. Instead of numbers, it might be more helpful to think in terms of specifications and differences:

A single layer neural network can't represent an XOR function. If you think about the multiplication and sum operation, there is no way to discriminate among the inputs.

At the same time, all logic gates, including XOR, can be reduced to NOR. No matter how many inputs a logic gate takes, it can be decomposed into a circuit of 2-input NORs.

This line of reasoning have led many to believe that logic is more fundamental than neurons. However, a multi-layer neural network can approximate the XOR function with bounding planes.

It's very difficult to compare the abstractions. A crucial difference is that neurons can train themselves based on feedback. But decision trees can do this too.

[bradrn]

I think I can imagine what it would be like to be more intelligent.

[…]

How this is not the same as intelligence: […]

Your post is self-contradictory. You list some ways in which you claim you ‘can imagine what it would be like to be more intelligent’, and then immediately say that those are ‘not the same as intelligence’. If there’s a point to this exercise I don’t understand it.

I have occasionally, half-heartedly trained myself to get better at some of these things. It doesn't feel like there's music playing in your head a la Girl Genius. It doesn't feel like being more intelligent or having a superpower. It doesn't feel like you can take over the world. In fact, the most impressive of these skills barely transfer to real world problems. What it feels like is that you are better at doing sums. This is not useless. It also doesn't correspond to romantic notions of ~intelligence~. It feels like being an accountant. If you feel like I'm being overly reductive, Bertrand Russell (IIRC) discusses how the idealist notion of intelligence originating in Plato boils down to the ability to do sums. Being better at sums does let you solve some problems, and it does feel annoying if everyone else is really bad at them.

Well… yeah. I agree with all this, speaking as someone who other people have often called ‘highly intelligent’. (A designation I hate, by the way. I don’t think intelligence is anything special when it comes to humans.) As I said earlier, it does all boil down to ‘skills in abstract manipulation and language’, which is as un-romantic as it gets.

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

[rotting bones]

Your post is self-contradictory. You list some ways in which you claim you ‘can imagine what it would be like to be more intelligent’, and then immediately say that those are ‘not the same as intelligence’. If there’s a point to this exercise I don’t understand it.

In my second sentence, I did say, "I don't believe in intelligence as it's imagined in our collective mythology".

Well… yeah. I agree with all this, speaking as someone who other people have often called ‘highly intelligent’. (A designation I hate, by the way. I don’t think intelligence is anything special when it comes to humans.) As I said earlier, it does all boil down to ‘skills in abstract manipulation and language’, which is as un-romantic as it gets.

What do you think about imagining greater intelligence now?

Constructions (my 3rd point) are the hardest to specify. Holding the initial structure and the final structure in memory, you would have to invent an intermediate structure which converts one to the other in some sense. Not only is it unclear what this entails in the general case, it doesn't seem to be in P. It's hard to imagine any system solving constructions efficiently.

On the other hand, being able to invent constructions immediately for all structures does seem like the closest thing to an intelligence-based superpower.

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

I've heard of business gurus training charisma for just this purpose. There are also pick-up artists who use it for a different purpose.

See the Charisma on Command channel on YouTube.

[zompist]

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

I alluded to this mentioning Homer, but let me make it more explicit: who's the smartest guy in the Iliad and Odyssey? Odysseus, surely. Is he smart with abstract math or something? No, he's wily. He's a guy who picks up on situations quickly, who can win a battle or a war with a ruse. I think he's pretty much the dude you're describing.

And I might add, who's the smartest of the hunter-gatherers? I dunno, I wasn't there, but I expect it's the guy who knows a lot: where the hidden water sources are, what all the plants do, how the animals behave, what you do in the case of a once-a-generation disaster.

There have been societies where great mathematical ability could occur, but not many, I think, where this was highly appreciated.

[rotting bones]

A single layer neural network can't represent an XOR function. If you think about the multiplication and sum operation, there is no way to discriminate among the inputs.

By "discriminate among the inputs", I mean there's no linear threshold to discriminate between the true and false conditions of an XOR function.

[rotting bones]

I alluded to this mentioning Homer, but let me make it more explicit: who's the smartest guy in the Iliad and Odyssey? Odysseus, surely. Is he smart with abstract math or something? No, he's wily. He's a guy who picks up on situations quickly, who can win a battle or a war with a ruse. I think he's pretty much the dude you're describing.

Predicting who will be left holding what at the end of a transaction can be imagined as a subfield of accounting in some abstract sense. Chess is mathematical in that sense, and you do have to be good at predicting which pieces you will drop in an exchange.

Chess is perfect information, but game theory is also a mathematical discipline. The one math major I know well is obsessed with playing the social deduction game Blood on the Clocktower. Business majors are obsessed with these types of games. I was reading the book by the Gary's Economics channel. He says he was hired to work in a financial firm for being good at what was effectively a Poker variant.