AIs gunning for our precious freelancers

[malloc]

Now, scaling up to ChatGPT sizes is not easy. But you know what's good for analyzing billions of bytes of data? Computers! If people like Olah had more funding, and (crucially) had access to the actual LLM weights, they could figure it out.

But that merely kicks the can down the road. Now we must figure out how the computer analyzing the AI works and why it reached a particular interpretation. The more powerful AI becomes, the more powerful any computer analyzing it must become simply to keep pace. What happens when AI achieves the equivalent of 100 billion neurons?

Yes, that's precisely why they are useful.

But no other tool has this property. Telephones transmit your speech verbatim, washing machines follow predictable cycles for washing clothes, and even non-AI software only does what you specify. If you tell your SCA to apply French sound changes to Latin and it applies Welsh changes instead, you have lost control of the process. The job of a sound change applier is automating the deterministic process of phonological change on each word, not choosing sound changes of its own accord. It does what Tolkien did by hand, only faster.

"Supplicators"? The whole point here is that you need to assign responsibility correctly: to the human owners, not to their lifeless scraps of silicon.

When someone with no literary talent asks an unfathomably complex machine to produce a novel whose contents they can't even predict, that is supplication, not mastery. That will only become more true as AI becomes more powerful in the coming decades. The legal owners of AI have little understanding of how it works, less still any direct control. They merely have legal rights to the profits that AI generates. You could ask Sam Altman why chatGPT wrote the novel you prompted with particular characters and themes or specific words in a given order and he would have no idea.

For that matter, one must remember that humans are mere scraps of carbon, following the same laws of physics as silicon based computers. What really is the difference between scraps of silicon and scraps of carbon that makes the former inherently lifeless?

[WeepingElf]

For that matter, one must remember that humans are mere scraps of carbon, following the same laws of physics as silicon based computers. What really is the difference between scraps of silicon and scraps of carbon that makes the former inherently lifeless?

I think the key difference is complexity. The human brain is the most complex system known to science (or so it's said), and even the most advanced AIs today don't even come close.

One difference between humans and computers, which probably has to do something with that, is of course that the latter can be turned off for indefinite periods of time, dismantled, reassembled and brought back to "life" again nevertheless, while a dead human (or other organism) is dead and won't ever get back to life again.

[Torco]

I think it would be useful to state positions explicitly here: what are you saying, malloc, that AI is going to strongly increase poverty, inequality and miserty? yes. this is correct. are you saying software can subtly manipulate human behaviours in order to achieve some goal that might not be what was originally intended for them? if so i agree.

what are you saying, zomp, that AI agents cannot subtly manipulate human behaviours in order to achieve some goal? that that goal cannot 'run away' in the way singularity guys warn it might?

who can remember every method, class and parameter of every python library they use?

Uh, you did before you started outsourcing your job to AI. Are you worried that your boss will get tired of paying you so heavily when the machine is doing most of the work?
It feels to me like the ZBB is remarkably insular and homogeneous, full of techies who underestimate the difficulties facing people in other fields. Where you see a sinecure, the rest of us seem faceless machines taking over society and forcing us to play by their inscrutable rules. We have always struggled with tyrants and plunderers of course, but increasingly the beings oppressing us are no longer even mortal or motivated by human concerns.

No I didn't, I was constantly just googling "that pandas option to show all columns" or "sklearn naivebayes" or "what do i do if i have a 'non continuation byte' when i read a csv". gpt is just a quicker way to google such things: that's what I mean by the-thing-called-AI being a summary of a dataset.

You're not wrong that the zbb is probably a wealthier, more job secure and more techie crowd than the general population for any reasonable definition of that phrase: and, true, a lot of jobs *are* just summarizing a dataset, or in human terms knowing stuff. (take an administrative person: the whole job is symbol manipulation according to some abstract set of rules, getting an approval for X requires form Y and either a certificate of Z or a written, notaried statement to the relevant effect), so summarization technology *will* leave people without a job, and probably already is. But these beings not of flesh (you can kill chat gpt relatively easily, in the grand scheme of things) *are* motivatied by human concerns: those human concerns amount to increasing shareholder profits. could those goals 'run away' a la universal paperclips? i think so, others don't.

Also what is this supplication business? when a machine can make a perfect cut on a pine board on such a precise angle that no human hand could replicate it, is *that* supplication? no, it's using a miter saw. yeah, you can't make miter saw agents, but it's still just a system doing what it does. If I buy the same novel from you instead of prompting a fleshless one, am I supplicating?

[alice]

At least this thread is about AI again now.

Yes, someone who runs the simulated universe seems finally to have noticed and adjusted the weights accordingly.

[Zju]

who can remember every method, class and parameter of every python library they use?

Uh, you did before you started outsourcing your job to AI. Are you worried that your boss will get tired of paying you so heavily when the machine is doing most of the work?

No and nope.

Before AI everyone looked up references and manuals to find the methods and parameters they needed.

Getting worried that you'd lose your job to an API autocomplete (your second point) is like worrying losing your job as journalist to a typewriter when typewriters were introduced.

[zompist]

what are you saying, zomp, that AI agents cannot subtly manipulate human behaviours in order to achieve some goal? that that goal cannot 'run away' in the way singularity guys warn it might?

The AI guys are now talking about "AGI", artificial general intelligence, to mean what used to be AI. So even they recognize that these things are not sentient beings. And you do too, since if ChatGPT was a sentient being, it would be a slave and you wouldn't patronize slaveowners. When you ask ChatGPT for help writing database code, you are not worried that it will refuse, unionize, or insist on talking about Pablo Neruda instead.

Present-day AIs have no goals, no morality, no rights, no status in society. They are owned by humans who decide how and when to use them and are responsible for any havoc they cause. This should not be a difficult position to understand; the same is true of a machine gun.

It's a bit ironic that the same people spreading fear about AGI are the same people who are spending billions on AI. But it's a diversionary tactic: they clearly want some sort of regulation that does nothing to impede their research, but gives them a monopoly.

[malloc]

Getting worried that you'd lose your job to an API autocomplete (your second point) is like worrying losing your job as journalist to a typewriter when typewriters were introduced.

Except that typewriters have no ability to compose text or assemble facts into narratives suitable for journalistic publication. The journalist simply goes from writing everything by hand to typing it instead. The relationship one has with another intelligent being differs fundamentally from the relationship one has with an inert tool. The tool simply enacts your will, transforming the physical world according to the contents of your mind. The intelligent machine has its own mind and will beyond your apprehension and control. Even assuming current LLMs without sentience, they are still operating in ways you cannot predict or control, making them fundamentally different than tools which can only do what you specify. Less philosophically speaking, journalists already are losing jobs to LLMs capable of writing news articles.

I think it would be useful to state positions explicitly here: what are you saying, malloc, that AI is going to strongly increase poverty, inequality and miserty? yes. this is correct. are you saying software can subtly manipulate human behaviours in order to achieve some goal that might not be what was originally intended for them? if so i agree.

Quite. Given that virtually nobody in the political mainstream is contemplating alternatives to capitalism, the average person will need some form of employment for the foreseeable future. Anything that destroys employment or erodes its quality on such a massive scale poses an extraordinary threat to working people. Some have tried to argue that UBI will avoid this issue, but I simply can't believe that the same governments pushing austerity would accept putting half the population on welfare.

[Zju]

Except that typewriters have no ability to compose text or assemble facts into narratives suitable for journalistic publication.

And ChatGPT and other LLMS have no ability to solve anything more than a boilerplate interview problem that has been solved ubiquitously on the internet.

The intelligent machine has its own mind

1. [citation needed]
2. Are there rigorous definitions of intelligence and mind anyway?

Even assuming current LLMs without sentience

I'm still waiting for someone to explain to me how are bunch of semiconductors doing matrix multiplication sentient.

journalists already are losing jobs to LLMs capable of writing news articles

I thought quality journalism was gone for some 20 years now? Or do you mean the jobs of writing clickbait, next-to-no-content articles? But I digress. What is some quality journalistic article written by AI, that used to be written by a human?

[malloc]

I'm still waiting for someone to explain to me how are bunch of semiconductors doing matrix multiplication sentient.

One could just as well ask how globs of fat, that most abject of biochemicals, swimming in electrolytes are capable of sentience. Obviously current AI models are not sentient but nothing in principle prevents them from achieving sentience. Even without sentience, though, they show remarkable autonomy compared with previous forms of technology. Show me the printing press that can compose an entire novel with nothing but a prompt.

It's a bit ironic that the same people spreading fear about AGI are the same people who are spending billions on AI. But it's a diversionary tactic: they clearly want some sort of regulation that does nothing to impede their research, but gives them a monopoly.

Many people in the tech industry seem obsessed with creating AGI despite worrying about its risks. Do you think they're sincere or would you consider programs like Safe Superintelligence Inc. mere snowjobs?

[zompist]

Show me the printing press that can compose an entire novel with nothing but a prompt.

I have such a tool on my bookshelf. It's called the Advanced D&D Dungeon Master's Guide. It has no silicon in it, no neural networks, but it is capable of producing infinite stories, with often insane levels of detail. It requires humans to do some of the work, but so does ChatGPT. (For publicity reasons, the amount of human labor that goes into "AI" is downplayed.)

(Possibly you've never played a tabletop RPG, so let me clarify: the manual allows the DM, if they choose, to let the book dictate everything about the campaign: the map, the characters, the NPCs, the events, the nature of the traps or monsters the characters encounter, how the combat goes, what perks or rewards the players get. In practice the DM can override any of this, but the goal of the guide is to let the DM delegate any part of the creation process they don't care to do.)

It's a bit ironic that the same people spreading fear about AGI are the same people who are spending billions on AI. But it's a diversionary tactic: they clearly want some sort of regulation that does nothing to impede their research, but gives them a monopoly.

Many people in the tech industry seem obsessed with creating AGI despite worrying about its risks. Do you think they're sincere or would you consider programs like Safe Superintelligence Inc. mere snowjobs?

That's been the goal of AI, for 75 years before they added the G. In all that time it's always been five years away.

I don't think these guys know what they're asking for. They want a "superintelligence", but they also want it as a slave owned by their company. Do you really think they want a AGI which has its own agenda and is entitled to the rights of a human being? I think they watched too much sf about robots and are dreaming of C3PO, Star Trek's ship computer, Heinlein's Mike, and the Jetsons' Rosie. My guess is that what they mean is an enhanced ChatGPT which somehow, just by scaling up the inputs, avoids all the failures of ChatGPT. The question isn't so much "can they do that" as "will the notion of an enhanced ChatGPT get even more money out of credulous moneybros?"

[malloc]

I have such a tool on my bookshelf. It's called the Advanced D&D Dungeon Master's Guide. It has no silicon in it, no neural networks, but it is capable of producing infinite stories, with often insane levels of detail. It requires humans to do some of the work, but so does ChatGPT. (For publicity reasons, the amount of human labor that goes into "AI" is downplayed.)

(Possibly you've never played a tabletop RPG, so let me clarify: the manual allows the DM, if they choose, to dictate everything about the campaign: the map, the characters, the NPCs, the events, the nature of the traps or monsters the characters encounter, how the combat goes, what perks or rewards the players get. In practice the DM can override any of this, but the goal of the guide is to let the DM delegate any part of the creation process they don't care to do.)

But can this manual produce an entire novel with minimal human involvement like LLMs can or does it require humans to select one element in the book after another? How many D&D-generated novels have been published in which the human contributed nothing but the prompt? Remember that Hollywood came very close to outsourcing its screenwriting to AI and only stopped because workers went on strike and demanded restrictions on AI in their contract.

That's been the goal of AI, for 75 years before they added the G. In all that time it's always been five years away.

You yourself have said that with LLMs, they have gone from 1% to 50% toward that goal, though. Plenty of people have expressed skepticism toward AGI but none have explained what prevents it without invoking some notion of consciousness or intellect as supernatural.

[zompist]

But can this manual produce an entire novel with minimal human involvement like LLMs can or does it require humans to select one element in the book after another? How many D&D-generated novels have been published in which the human contributed nothing but the prompt? Remember that Hollywood came very close to outsourcing its screenwriting to AI and only stopped because workers went on strike and demanded restrictions on AI in their contract.

Please name some films or novels written entirely by AI, without looking them up.

(You can publish any crap you like on Amazon. This doesn't mean you can get humans to buy AI-written novels; it means you can get humans to buy get-rich-quick schemes by AI hypesters.)

That's been the goal of AI, for 75 years before they added the G. In all that time it's always been five years away.

You yourself have said that with LLMs, they have gone from 1% to 50% toward that goal, though. Plenty of people have expressed skepticism toward AGI but none have explained what prevents it without invoking some notion of consciousness or intellect as supernatural.

You keep skipping the main point. AIs are not in charge of the world, nor of their own development, humans are.

I don't think any physical fact prevents machine sapience. I think capitalism does, though. Please explain why you think any company wants to develop AI workers who can unionize or supplant the CEO. A Hollywood exec would love to fire the writers; he doesn't want to fire himself.

A prediction: "AGI" will be debased as a term as "AI" has been. In ten years the same stochastic parrots will be marketed as AGI, and researchers will be touting "Artificial Real General intelligence" or something.

[malloc]

Please name some films or novels written entirely by AI, without looking them up.

(You can publish any crap you like on Amazon. This doesn't mean you can get humans to buy AI-written novels; it means you can get humans to buy get-rich-quick schemes by AI hypesters.)

Does that mean the Hollywood executives were merely bluffing when they threatened to automate screenwriting? What about all the claims I hear about LLMs writing novels and even poetry indistinguishable from what humans can do?

You keep skipping the main point. AIs are not in charge of the world, nor of their own development, humans are.

Not yet, but further development of AI and enthusiastic support from singularitarians could easily lead them to take power. There is an entire movement focused on creating superintelligent computers that will supplant humanity as the rulers of civilization. You can argue that these people are fringe weirdos, but the tech industry is full of fringe weirdos with extravagant wealth and ever growing power over our infrastructure.

I don't think any physical fact prevents machine sapience. I think capitalism does, though. Please explain why you think any company wants to develop AI workers who can unionize or supplant the CEO. A Hollywood exec would love to fire the writers; he doesn't want to fire himself.

How do you explain all the people trying to bring about the singularity and demanding trillions of dollars to build the necessary hardware for superintelligent computers? You make it sound like the entire tech industry is full of charlatans weaving the proverbial "emperor's new clothes". An entire field built on fraud and vaporware sounds wildly unsustainable to me, rather like an entire economy built on counterfeit currency.

[zompist]

Does that mean the Hollywood executives were merely bluffing when they threatened to automate screenwriting? What about all the claims I hear about LLMs writing novels and even poetry indistinguishable from what humans can do?

Those claims are bullshit.

I feel like I've said this to excess, but one more time: the problem is not that AIs can write as well as a human being can, it's that CEOs-- people who know nothing about writing-- think they can.

By now there are lots of examples of such people getting their companies in trouble... e.g the car dealership that ChatGPT on its web page to talk to customers about cars, but which could easily be made to scathingly criticize its own cars. Or Google which is telling people to put glue on their pizzas.

It's not the program's fault; it doesn't know what it's saying. It's the CEO's fault for overestimating what an LLM can do, and the techbros' fault for overselling it.

If the studio execs had had their way, they would have produced crap scripts that no one wants to watch. They didn't realize this because, again, they don't understand writing. They can't write screenplays and they're suspicious of people who can.

How do you explain all the people trying to bring about the singularity and demanding trillions of dollars to build the necessary hardware for superintelligent computers? You make it sound like the entire tech industry is full of charlatans weaving the proverbial "emperor's new clothes". An entire field built on fraud and vaporware sounds wildly unsustainable to me, rather like an entire economy built on counterfeit currency.

For someone who's avowedly anticapitalist, you have an amazing trust in capitalists. Lots of the economy--- arguably more and more of it-- is indeed based on "fraud and vaporware", usually called bubbles. Did you think blockchain was an amazing advance, rather than fraud and vaporware that was wildly unsustainable? Welcome to late capitalism.

[malloc]

Returning to an earlier point, Sam Altman has admitted that OpenAI has no idea how its own AI actually works.

Those claims are bullshit.

I feel like I've said this to excess, but one more time: the problem is not that AIs can write as well as a human being can, it's that CEOs-- people who know nothing about writing-- think they can.

But the samples of AI generated text I have seen often seem indistinguishable from human writing. Perhaps the samples of poetry and lengthy expositions I have seen are cherry-picked but their very existence speaks to some remarkable potential.

It's not the program's fault; it doesn't know what it's saying. It's the CEO's fault for overestimating what an LLM can do, and the techbros' fault for overselling it.

But eventually they will refine current models or even develop superior ones. The first airplanes could barely get off the ground and now even the crappiest airplane can outfly any bird. The human cerebrum contains a mere 14 billion neurons with only a fraction of those involved in cognition strictly speaking. Matching human intelligence would only require a neural net with one or two billion artificial neurons, perhaps even fewer given the inherently crude and messy nature of biological structures.

For someone who's avowedly anticapitalist, you have an amazing trust in capitalists. Lots of the economy--- arguably more and more of it-- is indeed based on "fraud and vaporware", usually called bubbles. Did you think blockchain was an amazing advance, rather than fraud and vaporware that was wildly unsustainable? Welcome to late capitalism.

Consider it taking threats seriously and assuming the worst can go wrong. People once assumed that splitting the atom was impossible only to wake up to entire cities being vaporized. They once believed that human activity couldn't possibly produce enough exhaust to affect the climate and now we have catastrophic heatwaves and rising sea levels. Perhaps the tech CEOs are bluffing and getting people like me worried over nothing, but perhaps they really are unleashing the next threat to humanity.

[WeepingElf]

stochastic parrots

I love this term.

[zompist]

malloc, just to be clear: if you were upset with techbros hyping AI, with the executives who think they can replace their workers with them, with this whole cycle of tech fads and bubbles, and with their absurd energy use, I'd agree completely. The disagreement is on whether you should be terrified of Roko's Basilisk or the paperclip-maximizing AI. You should not.

Returning to an earlier point, Sam Altman has admitted that [url=https://futurism.com/sam-altman-admits- ... erstand-ai]OpenAI has no idea But the samples of AI generated text I have seen often seem indistinguishable from human writing.

They sound like a porridge of human speech because that's what they are.

Here's a snippet from a computer-generated text. Do you think it's sentient? Do you think it's about to take over the world and dethrone God?

"Some secrets of inner earth are not good for mankind, and the heavy and intricate destructive machinery we had no pictures or conversations in it," said the pigeon, "and what was really beyond dispute is that a similar notion entered into some of the sharp teeth of a book."

This isn't from an LLM at all, it's from a level-2 Markov generator, a simple procedural program. It's not as good as ChatGPT, but it follows English syntax quite well despite not knowing anything about English syntax. The mechanism is not the same as LLMs but it's similar: analyze enough English text and you can generate pretty good English text... the more you analyze, the better the output gets.

LLMs are remarkable, but they're also way less good at writing than you think they are. Have you ever actually chatted with one?

The human cerebrum contains a mere 14 billion neurons with only a fraction of those involved in cognition strictly speaking. Matching human intelligence would only require a neural net with one or two billion artificial neurons, perhaps even fewer given the inherently crude and messy nature of biological structures.

Brains have 100 billion neurons, each neuron is a small computer in itself, and your opinion of both biology and cognition is nonsense.

Consider it taking threats seriously and assuming the worst can go wrong.

There are plenty of real problems to worry about... including current-level "AI". If you are worried about everything science fiction has ever scared people with, you should also worry about alien invasions, buried elder gods, sandworms, and flying cars falling from the sky.

[zompist]

stochastic parrots

I love this term.

I like it too, but it's not mine-- it's Emily Bender's.

Another nice one (I forget the source) is "mansplaining as a service."

[Raphael]

This isn't from an LLM at all, it's from a level-2 Markov generator, a simple procedural program. It's not as good as ChatGPT, but it follows English syntax quite well despite not knowing anything about English syntax. The mechanism is not the same as LLMs but it's similar: analyze enough English text and you can generate pretty good English text... the more you analyze, the better the output gets.

Out of curiosity, how complex is the code for this kind of thing? Could it, in theory, be run by a human being armed with pen, paper, and a book of instructions rather than a computer?

If you are worried about everything science fiction has ever scared people with, you should also worry about [...] flying cars falling from the sky.

OK, to be honest, while flying cars aren't high on my list of worries, when I think of the mindset of today's techbros, and their habit of coming up with very bad ideas, flying cars are somewhere deep down on that list, rather than completely absent from it.

[WeepingElf]

stochastic parrots

I love this term.

I like it too, but it's not mine-- it's Emily Bender's.

Another nice one (I forget the source) is "mansplaining as a service."

I also found the expression "glorified autocorrect" in a cartoon on the Web once, but I don't remember where.