AI technology thread

[malloc]

Long ago, I raised the problem of automation as a threat to humanity, destroying jobs that people needed to survive. Everyone told me not to worry and insisted that automation mainly affected crappy jobs without much enjoyment or prestige. Now the good jobs like artist and academic are evaporating like steam. I was right all along to worry about automation (in spirit if not necessarily in details) and everyone else was wrong.

[bradrn]

I was right all along to worry about automation (in spirit if not necessarily in details) and everyone else was wrong.

Yes, well done indeed. Now that your sense of self-superiority is assuaged, could you please resist the temptation to make any more such remarks, and try to confine yourself to constructive contributions to the discussion, as others have been doing?

[Torco]

I really do not see any slowdown occurring. (Quite the reverse, in fact — the capability growth seems to be exponential.)

at first it was, yes. i don't have any concrete metrics, but at least some AI researchers are saying this (though there is discussion about whether this is down to trainers running out of data rather than inherent limitations). anecdotally, i do use chatbots constantly in my office monkey job and, for my money, between 2020 and 2022 they got much better, between 22 and 24 they got better, and between 24 and 26 they've gotten somewhat better. and bear in mind that what they did do in 22 they accomplished using fewer chips and fewer watts than they're using today. models have grown in params more than they've grown in usefulness.

[jal]

I still see some people claiming that ‘LLMs are just a marketing scam’ and suchlike. Over the past few months this view has been becoming increasingly untenable, and this is the last nail in the coffin. LLMs can make really significant advantages, and it’s high time we all understood that so that we can act accordingly.

Very few people are claiming a "marketing scam". It's a scam alright, but not a marketing one. Though no doubt some people may be claiming that, it feels a bit like a strawman argument. Also, the fact that we may have a single break-through in mathematics doesn't mean "LLMs can make really significant advantages" in general.

What more than a few people are claiming however, is that the few useful things LLMs may have done do not justify the immense amount of computing power needed and all the detriment coming with it, like environmental polution and shortages of power and certain consumer goods, not to mention the potential abuse by dictatorial governments.


JAL

[bradrn]

I still see some people claiming that ‘LLMs are just a marketing scam’ and suchlike. Over the past few months this view has been becoming increasingly untenable, and this is the last nail in the coffin. LLMs can make really significant advantages, and it’s high time we all understood that so that we can act accordingly.

Very few people are claiming a "marketing scam". It's a scam alright, but not a marketing one. Though no doubt some people may be claiming that, it feels a bit like a strawman argument.

I just saw someone make that exact claim, which is why I put it in quotation marks. Even recently on this forum I’ve seen similar sentiments.

Also, the fact that we may have a single break-through in mathematics doesn't mean "LLMs can make really significant advantages" in general.

Yes, hence the several bullet points I devoted to their limitations, insofar as I can discern them.

[alice]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

[bradrn]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

Generally nanotechnology is considered the most worrying possibility. (See this for a more detailed discussion, including of plausibility.) However, there’s no shortage of possibilities for less-than-extinction-level catastrophes, thanks mostly to good ol’ human stupidity (and in particular that strain of it which makes people think that giving an LLM complete, unsupervised control over their computers is somehow a good idea).

[KamareKitsune]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

By whatever real-world fanfiction scenario they can come up with, naturally.

[malloc]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

The most likely outcome is humans losing their utility to civilization once AI and robots have mastered anything they can do. This leads to mass unemployment but also the deterioration of infrastructure that humans need. Farms and apartment blocks give way to data centers, hospitals and schools are shuttered, and so forth. The last humans find themselves scrounging for scraps amidst a civilization no longer built with them in mind. It hardly requires direct malice from AI to drive us to extinction, only the loss of our economic necessity and practical power.

[bradrn]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

The most likely outcome is humans losing their utility to civilization once AI and robots have mastered anything they can do. This leads to mass unemployment but also the deterioration of infrastructure that humans need. Farms and apartment blocks give way to data centers, hospitals and schools are shuttered, and so forth. The last humans find themselves scrounging for scraps amidst a civilization no longer built with them in mind. It hardly requires direct malice from AI to drive us to extinction, only the loss of our economic necessity and practical power.

I don’t get the logic there… why would ‘AI mastering everything’ necessarily lead to ‘schools and hospitals shutting’? As opposed to, say, AI teachers and doctors being universally available?

(A pretty depressing thought in itself, I must admit. But hardly a case of ‘kill all humans’, even so.)

[malloc]

I don’t get the logic there… why would ‘AI mastering everything’ necessarily lead to ‘schools and hospitals shutting’? As opposed to, say, AI teachers and doctors being universally available?

Apart from anything else, if AI has mastered all cognitive labor, there is no longer any economic reason to educate humans. They don't need any intellectual skills to find employment because any jobs requiring those skills have already been given to AI.

[zompist]

malloc, which part of "Stop it" did you fail to understand?

Everyone else: if you want an AI thread, don't feed malloc, or it becomes a malloc thread.

[Ares Land]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"? By teaching itself to build tiny but lethal insect-like robots? By deliberately collapsing buildings when they're full of people? By cloning Trump and making him at least appear respectable? By convincing us in some sort of incredibly high-tech way that the Sun has stopped shining? Your suggestions here!

Our eyes melt after seeing one ugly Nano Banana flyer too many.
We overdose on LinkedIn AI slope.
We die of boredom after yet another lecture on how Claude is awesome by that techbro coworker at the coffee machine.
And finally, the one closest, I think, to what Anthropic has an mind: we all die in a vague but painful and humiliating way unless governments regulate the market in a way that directly profits Anthropic.

at first it was, yes. i don't have any concrete metrics, but at least some AI researchers are saying this (though there is discussion about whether this is down to trainers running out of data rather than inherent limitations). anecdotally, i do use chatbots constantly in my office monkey job and, for my money, between 2020 and 2022 they got much better, between 22 and 24 they got better, and between 24 and 26 they've gotten somewhat better. and bear in mind that what they did do in 22 they accomplished using fewer chips and fewer watts than they're using today. models have grown in params more than they've grown in usefulness.

I really don't use AI much (as in, maybe once in a few months). But I noted that since about 2024 they've really stopped hallucinating. It's also interesting that we get more choices in models (so we can pick one from a less evil company), and we can run them on local hardware (which opens the perspective of using them without selling your soul to Sam Altman.)

[jal]

I really don't use AI much (as in, maybe once in a few months). But I noted that since about 2024 they've really stopped hallucinating. It's also interesting that we get more choices in models (so we can pick one from a less evil company), and we can run them on local hardware (which opens the perspective of using them without selling your soul to Sam Altman.)

Stopped hallucinating? Really? Perhaps by some definition of "hallucinating", but AI still produces wrong information. About every Google search I do, the AI answer is only partially correct.


JAL

[Richard W]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"?

Perhaps by reading Malloc's posts and concluding that the best solution is to pre-emptively nuke the USA now.

[jal]

Anthropic researcher believes more than 10% chance AI 'could kill all humans'. Not the first time someone has said something has said something like this; the question is: "how"?

What I'm wondering more about is why would a researcher of an AI company claim this? They are totally dependent on AI becoming a financial success. Why claim "it will kill you all"?


JAL

[jal]

Long ago, I raised the problem of automation as a threat to humanity, destroying jobs that people needed to survive. Everyone told me not to worry and insisted that automation mainly affected crappy jobs without much enjoyment or prestige. Now the good jobs like artist and academic are evaporating like steam. I was right all along to worry about automation (in spirit if not necessarily in details) and everyone else was wrong.

The thing to worry about is rampant capitalism. That's the root cause, not "automation".


JAL

[Ares Land]

I really don't use AI much (as in, maybe once in a few months). But I noted that since about 2024 they've really stopped hallucinating. It's also interesting that we get more choices in models (so we can pick one from a less evil company), and we can run them on local hardware (which opens the perspective of using them without selling your soul to Sam Altman.)

Stopped hallucinating? Really? Perhaps by some definition of "hallucinating", but AI still produces wrong information. About every Google search I do, the AI answer is only partially correct.


JAL

The results I got always were correct -- but I really don't use it intensively. Perhaps relevant is that I use a Mistral model set up to be as unimaginative as possible. (There really are 'creativity' settings, which are correlated to hallucinations.)

What I'm wondering more about is why would a researcher of an AI company claim this? They are totally dependent on AI becoming a financial success. Why claim "it will kill you all"?

It's speculated (and I think it makes sense) that the idea is to get governments to implement strict regulations on AI, in such a way that it would benefit the current actors and punish newcomers.

[malloc]

The thing to worry about is rampant capitalism. That's the root cause, not "automation".

Economic and social context matters obviously, but technology also has inherent qualities that influence its impact on the world. Fossil fuels contribute to smog and global warming for unavoidable reasons of chemistry and physics. That remains the case whether capitalists or socialists are the ones burning them. Ecologically minded socialists hardly imagine that socialism will alchemically transmute carbon dioxide into something benign but rather that it will avoid fossil fuels themselves as much as practical by prioritizing health and sustainability over profits.

Certainly the overarching economic system greatly influences the impact of many technologies including those involved in automation. Quite often the threat of automation is that it destroys the livelihood of someone already struggling to survive with few options beside their current job. Meanwhile that job sucks with dangerous working conditions and no room for creativity or personal fulfillment. It hardly makes sense to romanticize the job itself even if someone needs it to survive under capitalism. The problem here is that the tech industry is automating desirable jobs like art and academia, jobs that offer the most room for personal growth and creativity and draw on the best aspects of humanity. Do we really want to live in a world where humans no longer make groundbreaking discoveries or create beautiful works of art?

[WeepingElf]

Your analysis is not wrong but your conclusions are. Economic and political systems can be changed; capitalism is not a natural law, nor is an economy based on fossil fuels or nuclear power. And there are movements which try to change the world to the better; they grow stronger year after year, even in the United States, even if it currently seems as if the reactionaries were stronger. But that's politics; this thread is for discussing AI technology - and understanding the technology is fundamental to assessing the risks.