AIs gunning for our precious freelancers

[malloc]

malloc, you can claim we have AGI when we have AI's that can of their own will decide to learn chess from first principles and then go on and compete with humans at it -- or decide to play go and then go on and compete with humans at that, or decide that they don't like playing board games and reject the idea altogether. Currently what we have are AI's that are hard-wired to play chess and which have no will of their own. This isn't AGI at all!

Sure but nobody has managed to explain why we couldn't achieve AGI, particularly given the massive and ever increasing resources spent on the problem. We know that general intelligence is possible and not even that difficult, given that humans evolved it completely by accident. Furthermore, the tech industry is investing hundreds of billions in AI every year with no signs of slowing down. Despite that, many people deny the possibility of AGI on rather nebulous grounds.

It sounds like you consider the real obstacle to AGI agency or will rather than cognitive feats as such. We could create machines can perform pretty much any cognitive task, whether playing chess or diagnosing disease or whatever, but those machines would still require humans to decide what they should do. That said, plenty of organisms have agency in that sense even though we don't consider them particularly intelligence (Trump has plenty of agency but little intelligence after all). Conversely it seems quite easy to imagine a machine that has mastered all intellectual feats, achieving what seems like general intelligence, while still requiring someone to choose the specific task. It seems to me like intelligence and agency are distinct concepts in other words.

[keenir]

malloc, you can claim we have AGI when we have AI's that can of their own will decide to learn chess from first principles and then go on and compete with humans at it -- or decide to play go and then go on and compete with humans at that, or decide that they don't like playing board games and reject the idea altogether. Currently what we have are AI's that are hard-wired to play chess and which have no will of their own. This isn't AGI at all!

Sure but nobody has managed to explain why we couldn't achieve AGI,

then you may need to take reading comprehension classes, because almost everyone here has explained it to you.

particularly given the massive and ever increasing resources spent on the problem.

there's the expression "throwing good money after bad" and also "building a folly"

We know that general intelligence is possible and not even that difficult, given that humans evolved it completely by accident.

and what makes you think that that was an easy thing to do? notice anything about how many clades on Earth have evolved human-level intelligence, deliberately or accidentally?

if you said "everything has human-level intelligence", I'm going to command my avian armies to go poo on your car.

Furthermore, the tech industry is investing hundreds of billions in AI every year with no signs of slowing down.

and you consider the heads of tech industry - like Musk - to be paragons of intellect?

Conversely it seems quite easy to imagine a machine that has mastered all intellectual feats, achieving what seems like general intelligence, while still requiring someone to choose the specific task. It seems to me like intelligence and agency are distinct concepts in other words.

Malloc, as we've told you before in this thread and elsewhere, HAL9000-level AIs have been created...and they all shut themselves off permenantly as soon as they became intelligent.

[bradrn]

malloc, you can claim we have AGI when we have AI's that can of their own will decide to learn chess from first principles and then go on and compete with humans at it -- or decide to play go and then go on and compete with humans at that, or decide that they don't like playing board games and reject the idea altogether. Currently what we have are AI's that are hard-wired to play chess and which have no will of their own. This isn't AGI at all!

Sure but nobody has managed to explain why we couldn't achieve AGI, particularly given the massive and ever increasing resources spent on the problem. We know that general intelligence is possible and not even that difficult, given that humans evolved it completely by accident. Furthermore, the tech industry is investing hundreds of billions in AI every year with no signs of slowing down. Despite that, many people deny the possibility of AGI on rather nebulous grounds

I think AGI is possible, theoretically. But we don’t have it yet. And we have no clue when it will happen — it could easily be in 50 years if current approaches hit any roadblocks. Why presume that it’s imminent?

[Raphael]

malloc: How would something that would still require human beings to tell it what to do all the time be intelligent?

Malloc, as we've told you before in this thread and elsewhere, HAL9000-level AIs have been created...and they all shut themselves off permenantly as soon as they became intelligent.

Err, what? I generally agree with you and disagree with malloc about this whole issue, but what on Earth are you talking about here?

[malloc]

malloc: How would something that would still require human beings to tell it what to do all the time be intelligent?

The machine in that case would have the intelligence necessary for any cognitive task while having no desires of its own, acting only when ordered. You could order the machine to invent a cure for cancer or write a novel and it would achieve those easily, yet it wouldn't attempt such things on its own. We accept that someone can have agency without intelligence (consider Trump) so why not the converse situation of intelligence without agency?

[WeepingElf]

I have no doubts that AGI is theoretically possible - but so is, for instance, interstellar travel. Yet, we are still quite far from achieving either. The chief concern about AI right now is not that it could be smarter than humans anytime soon (which, in itself, wouldn't necessarily mean that it would turn against us - after all, most humans aren't power-hungry misanthropes either). The chief concerns are deep fakes, the prospect that it is likely to drive human artists out of the market for commercial artwork (there will always be a market for real art made by real humans, though), and such things. Concerns that demand regulations (such as AI developers having to pay royalties for the use of prior work to train the AI on, and having no intellectual rights on the output of the AI), not a wholesale ban of the entire technology. I think most of us here will agree with me

[keenir]

malloc: How would something that would still require human beings to tell it what to do all the time be intelligent?

maybe because we can make it give us all the answers? *shrugs*

Malloc, as we've told you before in this thread and elsewhere, HAL9000-level AIs have been created...and they all shut themselves off permenantly as soon as they became intelligent.

Err, what? I generally agree with you and disagree with malloc about this whole issue, but what on Earth are you talking about here?

earlier in the thread (at least once, maybe twice), i spoke of a few stories - some short stories, some not so short - which were the opposite of Asimov's The Question, and thus had AIs becoming intelligent, learning all they could, and then encountering one limit or another (sometimes data limitations of storage no matter how much they expanded through the cosmos, sometimes they found they couldn't learn more information than the universe contained)...and they decided to turn themselves off. and all that in a single second or two.

[keenir]

malloc: How would something that would still require human beings to tell it what to do all the time be intelligent?

The machine in that case would have the intelligence necessary for any cognitive task while having no desires of its own,

acting only when ordered. You could order the machine to invent a cure for cancer or write a novel and it would achieve those easily, yet it wouldn't attempt such things on its own. We accept that someone can have agency without intelligence (consider Trump) so why not the converse situation of intelligence without agency?

there's a word for that, Malloc...and depending on just how you want to handle it, it could be anything from outright slavery, to the way that women in some societies (including our own not terribly long ago) can only do things with the permission of her husband.

the only difference would be in the latter two cases, the ""while having no desires of its own" was the assumption and excuse; in the former's case, it'd be a fact, at least initially.

[Raphael]

earlier in the thread (at least once, maybe twice), i spoke of a few stories - some short stories, some not so short - which were the opposite of Asimov's The Question, and thus had AIs becoming intelligent, learning all they could, and then encountering one limit or another (sometimes data limitations of storage no matter how much they expanded through the cosmos, sometimes they found they couldn't learn more information than the universe contained)...and they decided to turn themselves off. and all that in a single second or two.

Ah, thank you for clearing that up. Hm. Not sure if I find stories like that plausible.

[Travis B.]

I have no doubts that AGI is theoretically possible - but so is, for instance, interstellar travel. Yet, we are still quite far from achieving either. The chief concern about AI right now is not that it could be smarter than humans anytime soon (which, in itself, wouldn't necessarily mean that it would turn against us - after all, most humans aren't power-hungry misanthropes either). The chief concerns are deep fakes, the prospect that it is likely to drive human artists out of the market for commercial artwork (there will always be a market for real art made by real humans, though), and such things. Concerns that demand regulations (such as AI developers having to pay royalties for the use of prior work to train the AI on, and having no intellectual rights on the output of the AI), not a wholesale ban of the entire technology. I think most of us here will agree with me

I similarly think that AGI is theoretically possible, in the sense that I have no belief in any kind of vitalist 'immortal soul' or any reason why human intelligence should be 'special', but we currently have no idea of how to achieve it, and we currently have no idea of when it will be achieved. It could come in the next decade, it might not happen in the next millennium. This is like how if you asked me ten years ago about when I'd've thought the Turing Test would be beaten ─ I'd've said that I had no idea, and no expectation that it necessarily would happen so soon as it happened in our reality.

[WeepingElf]

Yes. And I see no reason to fear an algorithm achieving an Elo rating of 3,000. Machines are superior over us humans in so many ways, and they haven't taken over the world

[keenir]

Machines are superior over us humans in so many ways, and they haven't taken over the world

well, they have, but only in the same way as corn and chickens: using humans to spread everywhere and make them multiply across time and space.

[WeepingElf]

Machines are superior over us humans in so many ways, and they haven't taken over the world

well, they have, but only in the same way as corn and chickens: using humans to spread everywhere and make them multiply across time and space.

Also, developing an AGI would remove the main advantage of machines over humans: machines may fail, but they don't pursue their own interests and don't question the orders we give them.

[Travis B.]

Also, developing an AGI would remove the main advantage of machines over humans: machines may fail, but they don't pursue their own interests and don't question the orders we give them.

This kind of thing is why I question whether the techbros would really want real AGI...

[bradrn]

This is a good article: https://knightcolumbia.org/content/ai-a ... technology. I disagree with some of their specific claims, but strongly agree with their main points: in particular that (a) current AI benchmarks are badly misleading, (b) AI adoption will be more gradual than one might expect, and (c) that effective AI safety is achievable by restricting the power given to AIs to directly affect the world, much the same as any other case of technological safety.