Zompist Bboard Again

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

malloc wrote: ↑Sun Mar 02, 2025 7:29 pm @keenir: Not sure where your objection lies to be honest.

given how many objections I've had...

Based on everything I have read, neurons do function more or less like logic gates

we've asked you to cite your references, so we can see the proof you claim is out there waiting to be used/read.

albeit with considerably more inputs. Simulating the full range of neuronal connections would presumably require linking many logic gates together somehow, but it would hardly require an infinite number of gates. People here try to astound me with large numbers

ironic claim to make, given that you're the one arguing that, given your repeated claims of billions or more logic gates and whatever else you think in that moment is needed for AI dominating the universe.

malloc wrote: ↑Sun Mar 02, 2025 7:45 pm

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

the same thing that prevents us from building an Olympic swimming pool collection full of 15,000,000,000 pennies & thereby matching the Look Here Is Big Number that you think creates a superpowerful AI.

keenir wrote: ↑Sun Mar 02, 2025 9:13 pmthe same thing that prevents us from building an Olympic swimming pool collection full of 15,000,000,000 pennies & thereby matching the Look Here Is Big Number that you think creates a superpowerful AI.

So artificial neurons have considerable overhead and such that makes generating billions of them currently unfeasible, you would say?

we've asked you to cite your references, so we can see the proof you claim is out there waiting to be used/read.

Wikipedia maybe, and general knowledge of both neurons and logic gates. Sorry for not having academic papers on this subject, but the similarities between neurons and logic gates seem pretty self-evident.

malloc wrote: ↑Sun Mar 02, 2025 7:45 pm

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

The question is: what would we do with those neurons? To be sure, we already know a bit about how to make artificial neural networks do what we want, but we don’t know how to do it nearly as well as the human brain. This extends to even really basic issues like the overall architecture: unlike the brain, most current ANNs (and in particular transformer models such as GPT) are purely feed-forward networks, organised in discrete layers with each one connected only to the next one. This has many consequences, most notably that the neural network is completely incapable of storing a memory outside the words that you see. (And even then, on every single new word generated the entirety of the text must be fed back to the network in their entirety, because otherwise it couldn’t even retain that much between runs.)

malloc wrote: ↑Sun Mar 02, 2025 9:52 pm

we've asked you to cite your references, so we can see the proof you claim is out there waiting to be used/read.

Wikipedia maybe, and general knowledge of both neurons and logic gates. Sorry for not having academic papers on this subject, but the similarities between neurons and logic gates seem pretty self-evident.

Well, clearly the ‘general knowledge’ is a little too general (so to speak)… because all of us see very clear and obvious differences between them. The similarities are limited to, ‘if you put lots of them together in a specific way they can do calculations of some sort’.

Building a data center with a fifteen billion artificial neuron network and expecting it to be as smart as a human brain would be like growing neural tissue containing fifteen billion real neurons in a vat and expecting it to behave like a human brain. It's not just sheer size; it's also internal structure.

Raphael wrote: ↑Mon Mar 03, 2025 5:08 am Building a data center with a fifteen billion artificial neuron network and expecting it to be as smart as a human brain would be like growing neural tissue containing fifteen billion real neurons in a vat and expecting it to behave like a human brain. It's not just sheer size; it's also internal structure.

Good analogy.

malloc wrote: ↑Sun Mar 02, 2025 9:52 pm

keenir wrote: ↑Sun Mar 02, 2025 9:13 pmthe same thing that prevents us from building an Olympic swimming pool collection full of 15,000,000,000 pennies & thereby matching the Look Here Is Big Number that you think creates a superpowerful AI.

So artificial neurons have considerable overhead and such that makes generating billions of them currently unfeasible, you would say?

that is not what i say.

we've asked you to cite your references, so we can see the proof you claim is out there waiting to be used/read.

Wikipedia maybe, and general knowledge of both neurons and logic gates. Sorry for not having academic papers on this subject, but the similarities between neurons and logic gates seem pretty self-evident.

"pretty self evident" is why Victorians thought there was a class of mammalia they called Pachyderm - they thought elephants, rhinos, and hippos were all closely related animals.

malloc wrote: ↑Sun Mar 02, 2025 9:52 pm

keenir wrote: ↑Sun Mar 02, 2025 9:13 pmwe've asked you to cite your references, so we can see the proof you claim is out there waiting to be used/read.

Wikipedia maybe, and general knowledge of both neurons and logic gates. Sorry for not having academic papers on this subject, but the similarities between neurons and logic gates seem pretty self-evident.

Malloc, I can remember not a lot of things, but one of those i can is that Wikipedia's entry for The Holocaust used to be "It never happened." Full stop, end of page.

However, this time, I'll be charitable (and presume you actually read this far into my post)...and suggest that the wikipedia page quoted a movie - not an actual scientist, saying Judgement Day is inevitable!



also, i'm not asking for academic papers...just a title of a book you read that you get some of these claims from.

keenir wrote: ↑Mon Mar 03, 2025 7:28 am
Malloc, I can remember not a lot of things, but one of those i can is that Wikipedia's entry for The Holocaust used to be "It never happened." Full stop, end of page.

Oh, I'd say that Wikipedia has gotten a lot more useful than it was back then, thanks to improved policies. There are still some caveats, like, for instance, if a section of a Wikipedia article reads like it was written by someone with a particular axe to grind, or a particular obsession, you should dismiss it. But IMO for people who have developed the right kind of Wikipedia-related "instincts", Wikipedia is pretty useful these days.

malloc wrote: ↑Sun Mar 02, 2025 7:45 pm

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

in a word, structure.

on one hand, real neurons have functionality much more complicated than the simple input-activation_function_-output pseudoneurons of the thing we call neural networks. there's also not a single optimizer enforcing gradient descent in brains, nor does it run on matrix multiplication etcetera.

on the other, we know how many neurons a brain has but not what they're doing or how they're arranged or how that arrangement lets the brain do what it does. convolutions and other tweaks people do to neural networks in order to do various tasks better shows beyond a shadow of a doubt that it's not just "more parameters more better".

malloc wrote: ↑Sun Mar 02, 2025 7:45 pm

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

Here are fifteen billion neurons. See if you can connect them all up correctly, without any mistakes.

alice wrote: ↑Mon Mar 03, 2025 2:23 pm

malloc wrote: ↑Sun Mar 02, 2025 7:45 pm

bradrn wrote: ↑Sun Mar 02, 2025 7:42 pmNeurons are interesting, because it’s been proven that a two-layer artificial neural network is enough to simulate any other non-linear function whatsoever. This is a qualitative difference from logic gates, which require considerably more complexity to do universal computation. The other big difference is that neurons (natural or artificial) support learning in a way that logic gates do not.

What exactly prevents us from building a data center with a fifteen billion artificial neuron network and thereby matching the computational power of the human brain?

Here are fifteen billion neurons. See if you can connect them all up correctly, without any mistakes.

I bet you that Malloc (or someone else?) answers "thats one of the things what our AI with fifteen billion logic gates can assemble for us. then the two of them will rule us."

So you would say the main obstacle to AGI doesn't stem from the number of logic gates or artificial neurons but rather how those elements connect with each other. Currently we don't know enough about the human brain and how its connections result in general intelligence to replicate it with computers.

malloc wrote: ↑Mon Mar 03, 2025 7:54 pm So you would say the main obstacle to AGI doesn't stem from the number of logic gates or artificial neurons but rather how those elements connect with each other. Currently we don't know enough about the human brain and how its connections result in general intelligence to replicate it with computers.

That’s one of the obstacles, yes. Another is that our learning methods are basically various forms of gradient descent, which is not how the brain works (as already mentioned by Torco), and this suggests that there are different learning methods to explore. I’m sure that more informed people can suggest many more obstacles in the way of AGI.

malloc wrote: ↑Mon Mar 03, 2025 7:54 pm So you would say the main obstacle to AGI doesn't stem from the number of logic gates or artificial neurons but rather how those elements connect with each other. Currently we don't know enough about the human brain and how its connections result in general intelligence to replicate it with computers.

Another factor is power efficiency. Our brains are so vastly superior in this regard.
Brains use about 20 W to both think and be idle.
A high-end graphics card needs hundreds of watts to run, and has a few dozen billion transistors. Now again, how many transistors were needed to create one artificial neuron?

There are some efforts to bake NN into analogue chips, which would improve power efficiency, but it still won't be close.

Zju wrote: ↑Wed Mar 05, 2025 3:40 pmAnother factor is power efficiency. Our brains are so vastly superior in this regard.
Brains use about 20 W to both think and be idle.
A high-end graphics card needs hundreds of watts to run, and has a few dozen billion transistors. Now again, how many transistors were needed to create one artificial neuron?

There are some efforts to bake NN into analogue chips, which would improve power efficiency, but it still won't be close.

Sure but the human also requires an entire body to support it (putting aside the brain-in-a-vat concept of philosophy anyway) which itself consumes considerable energy. Also the brain spends one third of the day inoperable while computers can work constantly without rest. That doesn't necessarily refute your basic point, but AI advocates could easily point to those as glaring inefficiencies of the human brain vis-à-vis artificial intelligence.

malloc wrote: ↑Fri Mar 07, 2025 3:34 pm

Zju wrote: ↑Wed Mar 05, 2025 3:40 pmAnother factor is power efficiency. Our brains are so vastly superior in this regard.
Brains use about 20 W to both think and be idle.
A high-end graphics card needs hundreds of watts to run, and has a few dozen billion transistors. Now again, how many transistors were needed to create one artificial neuron?

There are some efforts to bake NN into analogue chips, which would improve power efficiency, but it still won't be close.

Sure but the human also requires an entire body to support it (putting aside the brain-in-a-vat concept of philosophy anyway) which itself consumes considerable energy.

This was addressed in the very first paragraph of the article Zju links to. The brain takes up about 20% of the body's energy, which means the body + brain operates on 100 W. A desktop computer takes 80 W.

malloc wrote: ↑Fri Mar 07, 2025 3:34 pm

Zju wrote: ↑Wed Mar 05, 2025 3:40 pmAnother factor is power efficiency. Our brains are so vastly superior in this regard.
Brains use about 20 W to both think and be idle.
A high-end graphics card needs hundreds of watts to run, and has a few dozen billion transistors. Now again, how many transistors were needed to create one artificial neuron?

There are some efforts to bake NN into analogue chips, which would improve power efficiency, but it still won't be close.

Sure but the human also requires an entire body to support it (putting aside the brain-in-a-vat concept of philosophy anyway) which itself consumes considerable energy.

breaking news: nerves are cells, and cells need to be fed!

Also the brain spends one third of the day inoperable while computers can work constantly without rest.

yup, I can let my computer search for virii while I take a nap, snack, and talk to friends...and if my computer activates screensaver or sleep mode, it keeps working.

I'm not seeing a problem. or proof that thats proof of computer superiority.

That doesn't necessarily refute your basic point, but AI advocates could easily point to those as glaring inefficiencies of the human brain vis-à-vis artificial intelligence.

they could? but do they?

keenir wrote: ↑Fri Mar 07, 2025 3:48 pmyup, I can let my computer search for virii while I take a nap, snack, and talk to friends...and if my computer activates screensaver or sleep mode, it keeps working.

I'm not seeing a problem. or proof that thats proof of computer superiority.

Almost by definition, the ability to work constantly represents an advantage over having to take long breaks between work. Regardless of how broad you consider the gulf between human and computer intelligence, the ability of computers to work far harder than humans represents a massive point in their favor.

Meanwhile it becomes increasingly difficult to name human-exclusive cognitive abilities by the day...