AIs gunning for our precious freelancers

[zompist]

It turns out ordinary people love cyberspace— but they're happy with web pages, they don't want to walk around cartoon spaces as cartoon avatars.

It's annoying to have to walk around when you just want to be told an answer. On the other hand, physical spaces aid memory like in a memory palace. Are there games that let you walk around a custom 3D palace as an aid to memorization? That might be a good use for a Metaverse.

In my experience in video games, walking around gets old fast. Thus the plethora of fast travel systems.

Most games try to limit how much you have to memorize. But a few seem close to your idea— rather than checking inventory, changing outfits, or choosing perks in dialog boxes, they transport you to a virtual space like a pocket dimension. Or you have to go talk to specific NPCs: the armorer here, the potion seller there, etc.

[rotting bones]

In my experience in video games, walking around gets old fast. Thus the plethora of fast travel systems.

Yes, but walking around a charming 3D palace like in Mario 64 sounds more fun than memorizing random facts. If you can populate the palace with the organization of facts, it might help players remember them. I haven't played Mario 64 in decades, and I can still visualize the palace layout. I don't remember what I was working on when playing that game.

[malloc]

Tell you what, malloc. I'll give you $2, and you give me $7000 in return. Fair deal? They're exactly the same number, right?

Certainly there is no disputing that neurons are quantitatively more complex than logic gates. Nobody would confuse two with seven thousand or assume that additional inputs would have no impact on the computational power of neurons*. Nonetheless that does not prove that neurons are qualitatively more powerful than logic gates such that computers relying on silicon logic gates cannot match human intelligence.

*Though there are presumably diminishing returns with additional synapses since each neuron is still limited to producing one Boolean value regardless of input complexity.

[rotting bones]

Certainly there is no disputing that neurons are quantitatively more complex than logic gates. Nobody would confuse two with seven thousand or assume that additional inputs would have no impact on the computational power of neurons*. Nonetheless that does not prove that neurons are qualitatively more powerful than logic gates such that computers relying on silicon logic gates cannot match human intelligence.

*Though there are presumably diminishing returns with additional synapses since each neuron is still limited to producing one Boolean value regardless of input complexity.

I don't know if this language of "complexity" is helpful. It might be better to list differences:

An artificial neuron using ReLU remembers how much "credence" to place on each input. This credence value is called the weight. It takes a numerical value from each input, multiplies them with their respective weights and adds them up. If the result is negative (because weights were negative), it sets the answer to zero and passes the result to the output.

A logic late accepts a true/false value from each input. Its output is decided by the row of a finite truth table corresponding to the true/false configuration from its input. It's amazing that this is sufficient to store information in memory: https://www.allaboutcircuits.com/upload ... -latch.jpg

I also vaguely remember that biological neurons use exponentiation instead of multiplication, but don't quote me on that. This could be related to their binary system, to prevent the signal from degrading. I suspect biological neurons reason best using very wide networks with a lot of inputs. Long chains of biological neurons are better at transmission than reasoning.

[rotting bones]

And of course, artificial neurons change the credence value based on feedback.

PS. How biological neurons work: https://en.wikipedia.org/wiki/Biological_neuron_model

The memory palace method I was referring to: https://en.wikipedia.org/wiki/Method_of_loci

[bradrn]

Orson Scott Card had his interstellar internet become sentient, but he was pretty clear about the whole thing being (in effect) a con-religion.

It’s more interesting than this: in my opinion, Ender’s Game is a very unusual example of a series passing gradually from science-fiction into fantasy. If you think about it, by the end of Xenocide he’s pretty much introduced a complete magic system (abilities: telepathy, teleportation, bilocation, matter creation ex nihilo), while the scientific aspects have decreased to almost nothing. Children of the Mind is even more lopsided. It’s a wonder he was able to pass it off as science fiction in the first place. (Then again, if people can accept Star Wars…)

[rotting bones]

Star Wars might be soft SF, but telepathy is surprisingly common in hard SF. It might be a metaphor for how nerds like me take pride in what we do. Or maybe it has to do with the magazine editor's involvement with Dianetics.

[bradrn]

Star Wars might be soft SF, but telepathy is surprisingly common in hard SF. It might be a metaphor for how nerds like me take pride in what we do. Or maybe it has to do with the magazine editor's involvement with Dianetics.

As I understand, it’s traceable back to John W Campbell, editor of Astounding Science Fiction, who was fascinated by ‘psi’ and did his very best to encourage stories about it in his magazine.

EDIT: oh right, that was who you were talking about when you mentioned ‘involvement with Dianetics’.

But still: I’d say that telepathy can be done in a more science-like way or a more magic-like way, and Ender’s Game is definitely on the more magic-like side, in that it’s reliant on spirituality and emotional connections as opposed to technology.

[malloc]

@rotting bones: So even granting the premise that neurons are qualitatively more powerful than logic gates, it sounds like computers already have something that can match the complexity and power of neurons. The key to artificial general intelligence is presumably creating a computer with fifteen billion of these artificial neurons. That sounds simple enough given the incredible processing power and memory of existing computers.

[bradrn]

@rotting bones: So even granting the premise that neurons are qualitatively more powerful than logic gates, it sounds like computers already have something that can match the complexity and power of neurons. The key to artificial general intelligence is presumably creating a computer with fifteen billion of these artificial neurons. That sounds simple enough given the incredible processing power and memory of existing computers.

Let me quote Ares Land back to you again:

This all sounds like an old SF tropes.
In 50's and 60's SF computers became sentient when you added enough logic gates to them. In the 90s they even upgraded this, as computer networks or even the Internet became sentient as soon as people connected enough computers to them.

Nothing remotely like that ever happened. I'm pretty sure SF writers were aware the idea was nonsense -- but why let that get in the way of a good story?

How is it that you still keep on confusing 50s science fiction with reality? Even the 50s science fiction writers didn’t do that!

[zompist]

Tell you what, malloc. I'll give you $2, and you give me $7000 in return. Fair deal? They're exactly the same number, right?

Certainly there is no disputing that neurons are quantitatively more complex than logic gates. Nobody would confuse two with seven thousand or assume that additional inputs would have no impact on the computational power of neurons*.

Nobody except you, who used this innumeracy to compare "quadrillions of logic gates versus only fifteen billion neurons".

Let's see what happens if we compare comparable things and also correct your distorted numbers.

gpt-4 has about 1.7 trillion parameters. These are comparable to connections between neurons in the human brain... of which there are 80 to 100 trillion.

(It's easy to find estimates for the number of parameters in LLMs, hard to find estimates for the number of nodes; but it'll be far smaller. The number of neurons in the human brain, BTW, is 100 billion. Funny how whenever you look at biological systems, you underestimate them by several orders of magnitude.)

[bradrn]

gpt-4 has about 1.7 trillion parameters. These are comparable to connections between neurons in the human brain... of which there are 80 to 100 trillion.

And let me add that even if we made a system with 100 trillion connections it would still not necessarily be intelligent. That requires, not merely a certain number of connections, but for those connections to be made in a particular way and for them to be able to update themselves.

[Ares Land]

Orson Scott Card had his interstellar internet become sentient, but he was pretty clear about the whole thing being (in effect) a con-religion.

It’s more interesting than this: in my opinion, Ender’s Game is a very unusual example of a series passing gradually from science-fiction into fantasy. If you think about it, by the end of Xenocide he’s pretty much introduced a complete magic system (abilities: telepathy, teleportation, bilocation, matter creation ex nihilo), while the scientific aspects have decreased to almost nothing. Children of the Mind is even more lopsided. It’s a wonder he was able to pass it off as science fiction in the first place. (Then again, if people can accept Star Wars…)

I think OSC was at a point in his career where he could publish about anything as long it had the name 'Ender' in it. (Xenocide was interesting... Children of the Mind, though, urgh.)

Star Wars might be soft SF, but telepathy is surprisingly common in hard SF. It might be a metaphor for how nerds like me take pride in what we do. Or maybe it has to do with the magazine editor's involvement with Dianetics.

A bit of both, I guess and also this: SF used to love the idea of the superman, the next step in human evolution.
The thing is, it's not that easy to figure out what the superman would do with his superbrain. Especially if you have to come up with something reasonably spectacular that can fit in a pulp magazine or a short paperback. Especially if the superman has to be persecuted by the ignorant mass and spectacularly turn the table.
About the one good solution that fits that set of constraints is that the superman moves matter with his superbrain, or has telepathic powers, or predicts the future.

Depicting superhuman intelligence reasonably is a difficult problem. When the Singularity was in fashion, the supersmart computers were hardly better -- there's one book, I don't remember which, where the AI turns the solar system into, I kid you not, 'computronium', but what it does with all that brain matter remains unscrutable to mortals.

Personally, I think there are good reasons to believe that 'superhuman intelligence' doesn't make that much sense.

[zompist]

Depicting superhuman intelligence reasonably is a difficult problem. When the Singularity was in fashion, the supersmart computers were hardly better -- there's one book, I don't remember which, where the AI turns the solar system into, I kid you not, 'computronium', but what it does with all that brain matter remains unscrutable to mortals.

I think you either read, or read my review of, Stross's Accelerando.

Personally, I think there are good reasons to believe that 'superhuman intelligence' doesn't make that much sense.

I think it makes sense, it just isn't readily imaginable.

The best we can do is kind of skirt around it. Clarke's Rendezvous with Rama works pretty well, I think, precisely because it doesn't answer most of the questions it raises.

[bradrn]

(Xenocide was interesting... Children of the Mind, though, urgh.)

Agreed, yes.

A bit of both, I guess and also this: SF used to love the idea of the superman, the next step in human evolution.
The thing is, it's not that easy to figure out what the superman would do with his superbrain. Especially if you have to come up with something reasonably spectacular that can fit in a pulp magazine or a short paperback. Especially if the superman has to be persecuted by the ignorant mass and spectacularly turn the table.
About the one good solution that fits that set of constraints is that the superman moves matter with his superbrain, or has telepathic powers, or predicts the future.

Now this is an interesting perspective. I never quite linked the two, but I think you’re right. I suppose we can trace a line from this straight to Yudkowsky et al.’s theory that a superintelligence can achieve Godlike powers via nanotechnology. It may be less physically implausible than the old SF psi powers, but it uses all the same narrative tropes.

(Semi-relatedly: Vernor Vinge reports having once tried to write a story from the perspective of a superman, and getting a rejection letter from John Campbell: ‘Sorry—you can’t write this story. Neither can anyone else.’ He eventually managed to solve the problem by writing about superintelligences instead, though even there I find the aliens much more compelling than the AIs.)

Depicting superhuman intelligence reasonably is a difficult problem. When the Singularity was in fashion, the supersmart computers were hardly better -- there's one book, I don't remember which, where the AI turns the solar system into, I kid you not, 'computronium', but what it does with all that brain matter remains unscrutable to mortals.

Stross’s Accelerando, perchance? [EDIT: hah, zompist got there before I did!]

As for ‘computronium’, I think the idea is less silly than the name. Given the nature of materials science, I can conceive the possibility of a state of matter able to perform computation to the maximum theoretically possible extent. What I can’t conceive is that any human or other intelligence will be able to create such a state of matter in, let’s say, the next 1000 years.

Personally, I think there are good reasons to believe that 'superhuman intelligence' doesn't make that much sense.

Oh, really? Why? What makes human brains the upper limit of possible intelligence?

[Ares Land]

Oh, really? Why? What makes human brains the upper limit of possible intelligence?

I have reservations about the question itself. One first question is, is 'intelligence' even a unitary thing?
We group a number of things human do is 'intelligence' -- and even what we group together is highly dependent on culture. We don't really count being good at sports as intelligence (we do depict athletes and high school jocks as a little dumbs), but the ancient Greeks did which frankly, makes more sense (the brain works just as much during a football as during a math exam)
We do distinguish culture, wisdom, erudition, book smarts, street smarts, IQ and EQ and so on... All of this is processed by the brain, of course, but so does everything.
We're all aware of being very smart when it comes to certain areas, and very stupid when it comes to others. That definitely fits my own experience anyway

OK, let's assume human intelligence is really a single quality (oversimplifying the case a bit.)

Is it a measurable quality? Can we measure intelligence?
We can measure computer's capabilities in megaflops or terabytes, or by number of transistors. For starters, the brain doesn't work like a computer, it doesn't store data in the same way (to the extent we can talk about the brain storing data!)
The first idea is brain size. This works in paleontology, to some extent (Neanderthal had a bigger brain than we do.) Yet smart people turn out to have the same hardware than stupid people. (They never did find anything unusual with Einstein's brain.)

How about IQ? IQ is a poor tool that doesn't measure much, there are many known issues (many of which discusses in the Mismeasure of Man). It's widely known as unreliable. I've never taken an IQ test, so maybe I'm wrong, but as far as I know, it's heavy on verbal skills and memorization; this runs into the problem of multiple intelligences I mentioned earlier. What about an excellent football player? Doesn't his brain work just as hard during a match as a physicist teaching quantum mechanics?
How about low IQ people? IQs below 80 or 70s? It's not like their brains are just slower or their memories shorter... Low IQ correlates with severe health disorders.
How about empathy? It is a form of intelligence, but despite the emotional IQ nonsense, you can't measure empathy -- you can see if it's lacking, which is another thing.

Anyway... My point is, intelligence might not be a single thing, and several of the various phenomena we lump under 'intelligence' are not measurable.

[bradrn]

Anyway... My point is, intelligence might not be a single thing, and several of the various phenomena we lump under 'intelligence' are not measurable.

I agree that these are both true. At the same time, this feels like avoiding the question.

On the first half: indeed there are multiple forms of intelligence. (Earlier in the thread I gave examples from my own experience!) At the same time, when we say that ‘so-and-so is smarter than me’, it’s pretty clear what is meant. There definitely seems to be some factor, primarily relating to skills in abstract manipulation and language, which corresponds to the common notion of ‘intelligence’. And it seems clear that that factor affects the abilities of people to do things. I cannot do mathematics like, say, Terry Tao; equally, other people have told me they can’t do the work I do.

The second half is dismissed by a more or less similar argument. There are plenty of things in the world which are difficult or impossible to measure! Doesn’t mean we can’t talk about them, or compare them. I could say, for instance, that some medical intervention yields ‘greater quality of life’ than another: even though measuring ‘quality of life’ is difficult or impossible, and different people might disagree to some extent, overall the idea makes sense. (While searching for examples I found that this is known as the McNamara fallacy.)

[malloc]

(It's easy to find estimates for the number of parameters in LLMs, hard to find estimates for the number of nodes; but it'll be far smaller. The number of neurons in the human brain, BTW, is 100 billion. Funny how whenever you look at biological systems, you underestimate them by several orders of magnitude.)

But most of those are found in the cerebellum. The cerebrum, which does all the actual thinking, only has around fifteen billion.

[Ares Land]

I agree that these are both true. At the same time, this feels like avoiding the question.

Intelligence is a concept that is easy to grasp intuitively, but hard to define precisely. That's why SF and AI doomers even more so have trouble portraying it convincingly.
Imagining a super intelligent AGI, what could it do that humans couldn't?