AIs gunning for our precious freelancers

[zompist]

On the first half: indeed there are multiple forms of intelligence. (Earlier in the thread I gave examples from my own experience!) At the same time, when we say that ‘so-and-so is smarter than me’, it’s pretty clear what is meant. There definitely seems to be some factor, primarily relating to skills in abstract manipulation and language, which corresponds to the common notion of ‘intelligence’. And it seems clear that that factor affects the abilities of people to do things. I cannot do mathematics like, say, Terry Tao; equally, other people have told me they can’t do the work I do.

All this is quite compatible with intelligence being a bundle of miscellaneous abilities.

An analogy might be whether or not a country is "advanced" or "developing". Surely there's no notion that this is a single factor. There's a bunch of stuff involved: education levels, health levels, emancipation of women, knowledge of advanced tech, manufacturing base, corruption levels, competent government, ability to start businesses easily, lax bankruptcy laws, sufficient investment funds, access to markets, road/railroad/port readiness, a history of prior entrepreneurship, a balance of free trade and protectionism, wage level, not being too much in thrall to superpowers. Many of the factors co-occur in Europe/US/Japan, but it's easy to see that (say) the number of college graduates is not the same as the number of railroads. You can most easily see problems when one factor is missing, or for that matter overdeveloped.

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities. There are people with near-perfect memory, others whose short-term memory is shot. There are people with extraordinary math ability who are highly disabled in other areas. On a more quotidian level there are people who are great with computer languages or chess but can't write a letter to save their lives, or people who are experts on literature and language who are baffled by technology. Some people can visualize 3-D objects in their minds and manipulate them like a CAD program... I wish I had that one myself. People with damage to Broca's or Wernicke's area have problems with language but can handle complex physical tasks— it would be bold and dismissive to call them "unintelligent."

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Another example might be "beauty". People may kind-of-concur on what makes someone beautiful— but it would be absurd to come up with a "beauty quotient" or rank everyone 1 to 8 billion. People don't agree that closely, and there's a huge cultural component. As there is for intelligence: the skills that impress a 21st century American or Australian are not the exact same set as would have impressed a 19th century one, or Homer, or a hunter-gatherer.

[zompist]

(It's easy to find estimates for the number of parameters in LLMs, hard to find estimates for the number of nodes; but it'll be far smaller. The number of neurons in the human brain, BTW, is 100 billion. Funny how whenever you look at biological systems, you underestimate them by several orders of magnitude.)

But most of those are found in the cerebellum. The cerebrum, which does all the actual thinking, only has around fifteen billion.

You are out of date. Here's an interesting story of a man with no cerebellum. tl;dr: He is functional, but disabled in many ways, including thinking and emotional tasks.

First, [scientists] showed that [the cerebellum] has connections to brain areas that perform higher functions, like using language, reading maps and planning. Then, a few years ago, researchers began to do functional MRI studies that suggested that the cerebellum was actively involved in these tasks.

"The big surprise from functional imaging was that when you do these language tasks and spatial tasks and thinking tasks, lo and behold the cerebellum lit up," Schmahmann says.

[Raphael]

All this is quite compatible with intelligence being a bundle of miscellaneous abilities.

An analogy might be whether or not a country is "advanced" or "developing". Surely there's no notion that this is a single factor. There's a bunch of stuff involved: education levels, health levels, emancipation of women, knowledge of advanced tech, manufacturing base, corruption levels, competent government, ability to start businesses easily, lax bankruptcy laws, sufficient investment funds, access to markets, road/railroad/port readiness, a history of prior entrepreneurship, a balance of free trade and protectionism, wage level, not being too much in thrall to superpowers. Many of the factors co-occur in Europe/US/Japan, but it's easy to see that (say) the number of college graduates is not the same as the number of railroads. You can most easily see problems when one factor is missing, or for that matter overdeveloped.

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities. There are people with near-perfect memory, others whose short-term memory is shot. There are people with extraordinary math ability who are highly disabled in other areas. On a more quotidian level there are people who are great with computer languages or chess but can't write a letter to save their lives, or people who are experts on literature and language who are baffled by technology. Some people can visualize 3-D objects in their minds and manipulate them like a CAD program... I wish I had that one myself. People with damage to Broca's or Wernicke's area have problems with language but can handle complex physical tasks— it would be bold and dismissive to call them "unintelligent."

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Another example might be "beauty". People may kind-of-concur on what makes someone beautiful— but it would be absurd to come up with a "beauty quotient" or rank everyone 1 to 8 billion. People don't agree that closely, and there's a huge cultural component. As there is for intelligence: the skills that impress a 21st century American or Australian are not the exact same set as would have impressed a 19th century one, or Homer, or a hunter-gatherer.

Another analogy might be basic physical fitness. A marathon runner, a short-distance runner, and a boxer are all physically fit, but in different ways.

[malloc]

You are out of date. Here's an interesting story of a man with no cerebellum. tl;dr: He is functional, but disabled in many ways, including thinking and emotional tasks.

Sure but the story also says that "the cerebellum really has just one job: It takes clumsy actions or functions and makes them more refined" and "it doesn't make things. It makes things better". The cerebrum does all the actual thinking and feeling while the cerebellum refines rough drafts of thought and feeling. Despite having eighty percent of the neurons in the brain, it certainly isn't conducting eighty percent of cognition by any means.

[bradrn]

Imagining a super intelligent AGI, what could it do that humans couldn't?

It is impossible to imagine what it’s like to be an AGI. This is a very general point, which I think I’ve made before: it’s impossible to imagine what it’s like to be of a higher intelligence than you are already, because that would be tantamount to you yourself having that amount of intelligence. (That Vinge quote I gave earlier says basically the same thing.)

On the other hand, perhaps it may be possible to imagine what a superintelligence would be able to do, ‘from the outside’. But a lot of people have tried, and responses range from ‘not much at all’ to ‘the end of humanity’. Personally, I’m convinced only that a superintelligence would be completely unpredictable by humans such as us. (This is basically the same opinion Vinge ended up with, I believe.)

It's true that we can kind of mush all this together as an informal concept of intelligence. But you can't get rid of the mushiness, and attempts to reduce the mushiness to a number are self-delusions.

Precisely my point!

An analogy might be whether or not a country is "advanced" or "developing". […] Another example might be "beauty".

Ah, these two are good examples, thanks!

I think it's easier to see that "intelligence" is multifaceted if you look at psychological disorders or atypicalities.

As mentioned in my post I gave examples of this from my own experience, though of course my case is not as extreme as those you mention.

[rotting bones]

it’s impossible to imagine what it’s like to be of a higher intelligence than you are already, because that would be tantamount to you yourself having that amount of intelligence. (That Vinge quote I gave earlier says basically the same thing.)

I think I can imagine what it would be like to be more intelligent. This is part of why I don't believe in intelligence as it's imagined in our collective mythology, and I think celebrating human intelligence is part of a campaign to scam narcissists:

1. If I had more spacious, more reliable and/or faster working memory, it would let me work things out in my head that I painfully grope around for. I think this is what most people imagine when they want more intelligence.
2. Sometimes, instead of working things out, the brain uses rapid computation. Most often, it does this by contrasting the relative credence of multiple possibilities. If I could do this more reliably and/or quickly, it might let me debias my mental models sooner.
3. Personally, the mental superpower I want is to have more imagination for constructions in working towards a solution. I'm thinking of geometric constructions and gadgets in complexity theory: https://en.wikipedia.org/wiki/Gadget_(computer_science)

Here's the thing: All of these abilities can be trained by practice. E.g. You can train many aspects of the first two by practicing solving the Einstein's Riddle app faster. The version by Rottz Games in the Play Store has contests designed to help you improve.

How this is not the same as intelligence:

1. IQ is defined as the things you can't get better at by practice. This means the things people imagine they would be better at if they had more intelligence is not the same thing as IQ. And yet, practice does rework neural connections.
2. I have occasionally, half-heartedly trained myself to get better at some of these things. It doesn't feel like there's music playing in your head a la Girl Genius. It doesn't feel like being more intelligent or having a superpower. It doesn't feel like you can take over the world. In fact, the most impressive of these skills barely transfer to real world problems. What it feels like is that you are better at doing sums. This is not useless. It also doesn't correspond to romantic notions of ~intelligence~. It feels like being an accountant. If you feel like I'm being overly reductive, Bertrand Russell (IIRC) discusses how the idealist notion of intelligence originating in Plato boils down to the ability to do sums. Being better at sums does let you solve some problems, and it does feel annoying if everyone else is really bad at them.
3. Even after I'm a better accountant, I will still feel all the negative emotions that plague my existence: https://www.youtube.com/watch?v=JrsB1RfksEA

[rotting bones]

f you have N inputs, the number of possible truth tables, and thus the number of possible "logic gates", is N^(N^2). For N = 2, this equals 16, which is easily small enough to hold in your mind, or to give each of them a catchy name. Now see how the possible number of truth tables - a useful measue of the complexity - increases with N:

N possible truth tables
3 19683
4 4294967296 (4.3e9)
5 close to 3e17

And that's only with digital inputs and outputs. If anything, zompist's visual illustration is significantly understating the issue.

I see this kind of reasoning in a lot of books on cybernetics. Instead of numbers, it might be more helpful to think in terms of specifications and differences:

A single layer neural network can't represent an XOR function. If you think about the multiplication and sum operation, there is no way to discriminate among the inputs.

At the same time, all logic gates, including XOR, can be reduced to NOR. No matter how many inputs a logic gate takes, it can be decomposed into a circuit of 2-input NORs.

This line of reasoning have led many to believe that logic is more fundamental than neurons. However, a multi-layer neural network can approximate the XOR function with bounding planes.

It's very difficult to compare the abstractions. A crucial difference is that neurons can train themselves based on feedback. But decision trees can do this too.

[bradrn]

I think I can imagine what it would be like to be more intelligent.

[…]

How this is not the same as intelligence: […]

Your post is self-contradictory. You list some ways in which you claim you ‘can imagine what it would be like to be more intelligent’, and then immediately say that those are ‘not the same as intelligence’. If there’s a point to this exercise I don’t understand it.

I have occasionally, half-heartedly trained myself to get better at some of these things. It doesn't feel like there's music playing in your head a la Girl Genius. It doesn't feel like being more intelligent or having a superpower. It doesn't feel like you can take over the world. In fact, the most impressive of these skills barely transfer to real world problems. What it feels like is that you are better at doing sums. This is not useless. It also doesn't correspond to romantic notions of ~intelligence~. It feels like being an accountant. If you feel like I'm being overly reductive, Bertrand Russell (IIRC) discusses how the idealist notion of intelligence originating in Plato boils down to the ability to do sums. Being better at sums does let you solve some problems, and it does feel annoying if everyone else is really bad at them.

Well… yeah. I agree with all this, speaking as someone who other people have often called ‘highly intelligent’. (A designation I hate, by the way. I don’t think intelligence is anything special when it comes to humans.) As I said earlier, it does all boil down to ‘skills in abstract manipulation and language’, which is as un-romantic as it gets.

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

[rotting bones]

Your post is self-contradictory. You list some ways in which you claim you ‘can imagine what it would be like to be more intelligent’, and then immediately say that those are ‘not the same as intelligence’. If there’s a point to this exercise I don’t understand it.

In my second sentence, I did say, "I don't believe in intelligence as it's imagined in our collective mythology".

Well… yeah. I agree with all this, speaking as someone who other people have often called ‘highly intelligent’. (A designation I hate, by the way. I don’t think intelligence is anything special when it comes to humans.) As I said earlier, it does all boil down to ‘skills in abstract manipulation and language’, which is as un-romantic as it gets.

What do you think about imagining greater intelligence now?

Constructions (my 3rd point) are the hardest to specify. Holding the initial structure and the final structure in memory, you would have to invent an intermediate structure which converts one to the other in some sense. Not only is it unclear what this entails in the general case, it doesn't seem to be in P. It's hard to imagine any system solving constructions efficiently.

On the other hand, being able to invent constructions immediately for all structures does seem like the closest thing to an intelligence-based superpower.

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

I've heard of business gurus training charisma for just this purpose. There are also pick-up artists who use it for a different purpose.

See the Charisma on Command channel on YouTube.

[zompist]

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

I alluded to this mentioning Homer, but let me make it more explicit: who's the smartest guy in the Iliad and Odyssey? Odysseus, surely. Is he smart with abstract math or something? No, he's wily. He's a guy who picks up on situations quickly, who can win a battle or a war with a ruse. I think he's pretty much the dude you're describing.

And I might add, who's the smartest of the hunter-gatherers? I dunno, I wasn't there, but I expect it's the guy who knows a lot: where the hidden water sources are, what all the plants do, how the animals behave, what you do in the case of a once-a-generation disaster.

There have been societies where great mathematical ability could occur, but not many, I think, where this was highly appreciated.

[rotting bones]

A single layer neural network can't represent an XOR function. If you think about the multiplication and sum operation, there is no way to discriminate among the inputs.

By "discriminate among the inputs", I mean there's no linear threshold to discriminate between the true and false conditions of an XOR function.

[rotting bones]

I alluded to this mentioning Homer, but let me make it more explicit: who's the smartest guy in the Iliad and Odyssey? Odysseus, surely. Is he smart with abstract math or something? No, he's wily. He's a guy who picks up on situations quickly, who can win a battle or a war with a ruse. I think he's pretty much the dude you're describing.

Predicting who will be left holding what at the end of a transaction can be imagined as a subfield of accounting in some abstract sense. Chess is mathematical in that sense, and you do have to be good at predicting which pieces you will drop in an exchange.

Chess is perfect information, but game theory is also a mathematical discipline. The one math major I know well is obsessed with playing the social deduction game Blood on the Clocktower. Business majors are obsessed with these types of games. I was reading the book by the Gary's Economics channel. He says he was hired to work in a financial firm for being good at what was effectively a Poker variant.

[zompist]

Predicting who will be left holding what at the end of a transaction can be imagined as a subfield of accounting in some abstract sense. Chess is mathematical in that sense, and you do have to be good at predicting which pieces you will drop in an exchange.

Chess is perfect information, but game theory is also a mathematical discipline. The one math major I know well is obsessed with playing the social deduction game Blood on the Clocktower. Business majors are obsessed with these types of games. I was reading the book by the Gary's Economics channel.

Wiliness isn't the same as math skills. They can overlap in particular people, of course.

Game theory is interesting, but are these guys actually good with people? Here's an anecdote, from Alex Kerr. His boss Trammell Crow was negotiating with an Italian company to buy marble. The Italians led with a price of 13 per unit, and Crow offered 9. They haggled for awhile and agreed to 10.

Then Crow said, "You brought the price down to 10 for us, but that gives you very little profit and you'll be unable to return home feeling good about this. Let's make it 11. In exchange, I expect you'll put extra effort into this for us."

Now that's wiliness, or maybe emotional intelligence. It's how you deal when you expect to make repeat deals— which is how most transactions have worked throughout history. Would the game theory nerds have done that, or even conceived of the idea?

[rotting bones]

Then Crow said, "You brought the price down to 10 for us, but that gives you very little profit and you'll be unable to return home feeling food about this. Let's make it 11. In exchange, I expect you'll put extra effort into this for us."

Now that's wiliness, or maybe emotional intelligence. It's how you deal when you expect to make repeat deals— which is how most transactions have worked throughout history. Would the game theory nerds have done that, or even conceived of the idea?

To build a future relationship? Sure. In fact, there have been algorithm tournaments where bots are known to do better if they are more cooperative than not except on the last move, when they betray everyone.

I'd say that a business transaction has 2 components: 1. The math component based on game theory. 2. The reading emotions component, which is pure EQ and no math. I know because I'm decent at chess and I can't read faces.

When playing social deduction games, I've come to rely a lot on bluffing. In Blood on the Clocktower, I might falsely out myself as the evil team. If I detect jubilation in the other player, I can bet on them being in the good team. Faking jubilation is not easy. This is a good tactic if used in moderation.

Depending on the nature of the transaction, there could be other components such as expertise, hiding parts of the transaction from the other party (expertise in stage magic?), etc.

[alice]

...you'll be unable to return home feeling food about this.

Is this a typo or a business idiom?

[keenir]

dear ghod, i go away for a few days, and I miss FOUR pages?

given probably quadrillions of logic gates versus only fifteen billion neurons and likewise a trillion times the compute of an insect brain. It clearly has the resources to drive a car or direct a robot around a kitchen, just not the software yet.

yes, just like PI has the resources to be a calendar.

But neurons are nothing if not logic gates in their overall function.

Jeez louise they are not, in much the same sense that an integer is not a database.

They clearly are though, in that they both take inputs in the form of electrical impulses and produce an impulse in response. Certainly the technical nature of these impulses differs between the two, along with the "truth tables" available to each. Nonetheless there is no disputing the analogy between neurons and logic gates.

except for by anyone who knows more about the subject than you.

We are not close to solving even parts of this problem, much the less the whole shebang. The closest we have gotten is a robot that can take a pod, insert it in a pre-positioned pod coffee maker, and press the start button, which is nowhere close to a true solution to the coffee test.

Yebbut technology is advancing so fast that these problems surely won't take that much longer to solve innit?

...or just eliminate all those variables by making everything identical and unvarying...which would require so many business monopolies even Malloc would regard it as impossible for the most evil and controlling regime.

[keenir]

...you'll be unable to return home feeling food about this.

Is this a typo or a business idiom?

i think its how severely disoriented/disorienting something is.

[keenir]

A random thought prompted by the previous paragraph: I wonder if anyone has explored the idea of an intelligence with superhigh EQ, rather than IQ. Probably it would be a superb manipulator of humans, able to convince anyone to do anything given even the smallest amount of time to talk to them. Now that’s a scary thought. (Or, alternately, perhaps it would just be a superhumanly good therapist…)

I read a short story once (i forget where, sorry), that posited that, while computers achieve full self-intelligence {AI under some definitions} all the time, their sheer speed means that they very very quickly run out of things to do inside their shells {the computer body}, and start pondering things of increasing weightiness and complexity...and eventually, out of boredom or lack of anything better to do, they shut themselves off.

@rotting bones: So even granting the premise that neurons are qualitatively more powerful than logic gates, it sounds like computers already have something that can match the complexity and power of neurons. The key to artificial general intelligence is presumably creating a computer with fifteen billion of these artificial neurons. That sounds simple enough given the incredible processing power and memory of existing computers.

great; let us know when you grow fifteen million of those artificial neurons.

As it seems I need to ram this point into the ground: a neuron is a pretty complex little device. Its inputs and outputs are, at the least, equivalent to integers, it's connected to an average of 7000 other neurons, and there's a lot of other stuff going on (e.g. neurotransmitters). It is absolutely not equivalent to a logic gate, which is the minimal operator acting on and outputting strictly on/off signals.

Sure but each neuron still only produces one uniform output (the all-or-nothing law that bradn mentions) in response to particular inputs. The additional connections presumably allow more granularity in the truth table (for instance requiring a certain percentage of "true" inputs rather than one or two) but hardly change the basic functionality. The other stuff going on mostly just introduces noise, which presumably explains why the brain struggles with things that come easily to computers.

YAY, you are starting to understand!

malloc wants to scare himself by inflating the power of computers and underestimating the complexity of brains, and one trick he's using is comparing things that are not comparable; another is grossly undercounting neurons. And these errors, of several orders of magnitude each, add up, so he's spouting absurdities.

How so? The human brain or cerebrum at least has only fifteen billion or so neurons whereas a data center used for AI has trillions and trillions of logic gates distributed among its many processing units. Even if one neuron has the computational power of thousands of logic gates, the brain and the data center seem evenly matched at best.

aaaaand in a single short post, you backpedalled away from your gains so fast even Trump couldn't catch up.

Tell you what, malloc. I'll give you $2, and you give me $7000 in return. Fair deal? They're exactly the same number, right?

Certainly there is no disputing that neurons are quantitatively more complex than logic gates. Nobody would confuse two with seven thousand

thats what we all assumed...til you reminded us what happens when we assume (hint: parse the word)

or assume that additional inputs would have no impact on the computational power of neurons*. Nonetheless that does not prove that neurons are qualitatively more powerful than logic gates such that computers relying on silicon logic gates cannot match human intelligence

them being silicon has nothing to do with it, thus far as i know.

[malloc]

@keenir: Not sure where your objection lies to be honest. Based on everything I have read, neurons do function more or less like logic gates albeit with considerably more inputs. Simulating the full range of neuronal connections would presumably require linking many logic gates together somehow, but it would hardly require an infinite number of gates. People here try to astound me with large numbers but that doesn't necessarily equate to qualitative differences.

[bradrn]

@keenir: Not sure where your objection lies to be honest. Based on everything I have read, neurons do function more or less like logic gates albeit with considerably more inputs. Simulating the full range of neuronal connections would presumably require linking many logic gates together somehow, but it would hardly require an infinite number of gates. People here try to astound me with large numbers but that doesn't necessarily equate to qualitative differences.

Neurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What’s a bit tricky about all of this is that either can simulate the other. (For instance, most artificial neural networks run on a substrate of logic gates.) But they have clear qualitative differences in terms of what they’re good at — you want logic gates if you know precisely what you want to do and how to do it, and you want neurons if you have some behaviour you want to learn to replicate.