Zompist Bboard Again

zompist wrote: ↑Sat Sep 03, 2022 7:02 pm

Curlyjimsam wrote: ↑Sat Sep 03, 2022 2:37 pm I think, at least to start with, AI art will mostly be restricted to cases where people wouldn't have paid for it anyway. I'm never going to pay someone to create art for my conworld for me - but I'd happily let a computer do it for free (or maybe for a small fee). Nobody actually loses out in that case.

I have some mixed feelings about this. I think it'd be fun to play with myself. On the other hand, I doubt I could get so much as a good picture of an iliu, much less (say) an iliu city. I would still encourage conworlders to do their own visualizations; it's not that hard.

I expect for me, at least when the tech is rather better than it is now but nowhere near as good as one can imagine it one day might be, it will be a case of getting the AI to generate images which I then heavily edit myself. It will overcome some of the difficulties but not all of them.

This tweet is quite spot-on:

https://twitter.com/Carnage4Life/status ... 1208768513

Seeing the ways ChatGPT & generative AI are changing so much about how people create in just a few months, it’s crazy to think tech spent years convinced the future was selling JPEGs stored in a very slow distributed database.

Raphael wrote: ↑Sun Feb 05, 2023 10:22 am This tweet is quite spot-on:

https://twitter.com/Carnage4Life/status ... 1208768513

Seeing the ways ChatGPT & generative AI are changing so much about how people create in just a few months, it’s crazy to think tech spent years convinced the future was selling JPEGs stored in a very slow distributed database.

Oh, just to be a contrarian: the people most enthusiastic about AI also happen to be overly enthusiastic about crypto, NFT and blockchain. This is yet another factor leading me to suspect AI is fairly overhyped.

Relatedly: at work, we're supposed to use AI now. I'm fine with it -- I like cool toys. What are we actually supposed to use AI for, though? They didn't say.

make it design your powerpoint presentations. come to think of it, I should do that.

it's true that, between blockchain and genererative language / image / whatever models, only one of the two has actually proven to be useful. I've heard of people automating their job with AI, I've not heard of blockchain doing anything other than pump and dumps and other schemes the main appeal of is that they use crypto/blockchain (for stuff normal tech works better). Tbh I think the worst thing about crypto is that it sucked all the air out of the p2p room: many things would be much better with p2p, but since they don't use b l o c k c h a i n....

Um. Good idea! ChatGPT can probably do that.

I agree with you on the rest. AI is genuinely useful though overhyped while blockchain is useless except as a scam.

To be scrupulously fair to blockchain, I do hear that it’s still rather useful for transferring money without a central authority, which of course was its original purpose. Apparently Bitcoin is rather popular in certain areas (Argentina is the one I recall) for precisely this reason. Of course, pretty much every other application of blockchain has proved to be a scam.

sure, but there's perfectly good systems to do that (hawala or gold) that don't require thousands of GPUs solving pointless math puzzles so hard they output the energy of a small meteor impact every week. no, really, a single transaction takes as much energy these days as cooking a bowl of soup. much cheaper to call uncle hassan over the phone, or smuggle a gold coin or two inside like an external USB battery case.

Apparently there's a guy who's making an open source p2p location services framework or something... man, open source uber would be nice.

There is the well-known hype cycle - Stage 1: "Technology X will never happen / never be practical", Stage 2: "Oh my god it works! This will change everything!!!! The applications are boundless!!!!!!", Stage 3: "Meh, it all is a dud and / or a scam...", Stage 4: Picking up the pieces and create some boring but useful applications; maybe some even becoming indeed part of everyday life. Blockchain is now in Stage 3, AI in Stage 2. Let's see where they go.

hwhatting wrote: ↑Tue Feb 07, 2023 4:10 am There is the well-known hype cycle - Stage 1: "Technology X will never happen / never be practical", Stage 2: "Oh my god it works! This will change everything!!!! The applications are boundless!!!!!!", Stage 3: "Meh, it all is a dud and / or a scam...", Stage 4: Picking up the pieces and create some boring but useful applications; maybe some even becoming indeed part of everyday life. Blockchain is now in Stage 3, AI in Stage 2. Let's see where they go.

Nah, AI is in stage 4. Stage two happened ~40 years ago, when people were getting all excited about ‘expert systems’ and sci-fi authors were fretting about robots. Stage 3 was the ‘AI winter’.

Another way of thinking about this: Roy Amara once said that ‘we tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run’. We are now in the late underestimation — remember, even 5 years ago, no-one predicted that something like ChatGPT would be possible! (Giving full credit, I first saw this in https://news.ycombinator.com/item?id=34074222 and subsequent discussion.)

bradrn wrote: ↑Tue Feb 07, 2023 4:39 am Nah, AI is in stage 4. Stage two happened ~40 years ago, when people were getting all excited about ‘expert systems’ and sci-fi authors were fretting about robots. Stage 3 was the ‘AI winter’.

(Shrugs) Maybe I shouldn't have said AI, it's too large a field. In any case, hype cycles can repeat when new incarnations of a technology come along.

bradrn wrote: ↑Tue Feb 07, 2023 4:39 am

hwhatting wrote: ↑Tue Feb 07, 2023 4:10 am There is the well-known hype cycle - Stage 1: "Technology X will never happen / never be practical", Stage 2: "Oh my god it works! This will change everything!!!! The applications are boundless!!!!!!", Stage 3: "Meh, it all is a dud and / or a scam...", Stage 4: Picking up the pieces and create some boring but useful applications; maybe some even becoming indeed part of everyday life. Blockchain is now in Stage 3, AI in Stage 2. Let's see where they go.

Nah, AI is in stage 4. Stage two happened ~40 years ago, when people were getting all excited about ‘expert systems’ and sci-fi authors were fretting about robots. Stage 3 was the ‘AI winter’.

Not really-- AI was more or less a cute toy for half a century, so stories about intelligent robots were not very different from ones about FTL travel.

Deep learning really is a game changer, I think, because really good pattern recognition can be used all over. It's probably a little overhyped; I wonder if the next frontier is interpreting and simplifying neural network discoveries. This article, on interpreting exactly what a network is doing, is a good start. In short: a neural network is not doing something procedural code couldn't do... after all, it is run on procedural code. You can tease out the relationships it finds and, if you choose, express it as a far simpler algorithm. (I'm thinking here of things like networks trained to do diagnosis from X-rays, or fold proteins, or translate texts.)

As many people have pointed out, ChatGPT is a bullshit generator. It's like a perfected Markov generator. From a capitalist viewpoint, this is not a drawback, because there is an enormous market for bullshit. It's pretty useless if you want non-fiction; but there are a lot of uses for readable, semi-plausible bullshit. One CEO even started writing e-mails with it, which is a telling self-own: CEO e-mails are a special brand of bullshit.

zompist wrote: ↑Tue Feb 07, 2023 5:08 amOne CEO even started writing e-mails with it, which is a telling self-own: CEO e-mails are a special brand of bullshit.

This sounds like it might be fun; do you have a link?

alice wrote: ↑Tue Feb 07, 2023 2:06 pm This sounds like it might be fun; do you have a link?

Yep!

And the company that guy runs is an online learning provider? Priceless

zompist wrote: ↑Tue Feb 07, 2023 3:34 pm

alice wrote: ↑Tue Feb 07, 2023 2:06 pm This sounds like it might be fun; do you have a link?

Yep!

[The CEO of Coursera] … uses [ChatGPT] to help break down big strategic questions — such as how Coursera should approach incorporating artificial intelligence tools like ChatGPT into its platform.

“I use it as a writing assistant and as a thought partner,” Maggioncalda told CNN.

Amazing. This kind of attitude goes some way towards explaining my last few experiences with Coursera. (Bad teaching and my own laziness account for the rest, but it’s good to know it wasn’t just me.)

Using ChatGPT as a thought partner is like using a cup as a drinking buddy.

I have doubts that procedural methods can match the performance of deep learning. In theory, neural network models aren't doing anything that can't be hand engineered. In practice, this requires the programmer to be effectively omniscient.

Imagine the space of all possible systems of linear equations with a fixed number of variables. Basically, the deep learning model tries to select from this space one system that matches the given inputs to the corresponding outputs. (Deep Learning is a supervised method that requires fully labeled data in the training phase.) By "one system of linear equations", I mean it learns the parameters of the equations. That is to say, the constant values by which the variables are multiplied. For example, in ax + by + c = 0 and dx + ey + f = 0, the learning process determines the values a, b, c, d, e and f from the space of all values these could have.

In pre-college math, I was taught to demarcate regions of space by saying it's over line 1, under line 2, and so on. Now scale this up to m variables in n dimensions satisfying millions of data points.

I can't imagine a low level programmer pulling that off with any reliability even after months of research. Of course, once you've determined the system of linear equations, you can study it, combine it, and do all sorts of other things to it. But how would you obtain that oracle in the first place if not by training a network?

Notes:

1. The number of dimensions may be arbitrarily large, but the neural network technique always works with systems of linear equations denoting lines in n dimensional space, not quadratic or cubic equations, let alone anything fancier.

2. We all know the data is insufficient to determine the parameters precisely. Honestly, it makes no theoretical sense that so many problems can be solved so well with n-variable linear equations when the asymptotic value n is nowhere close to infinity. Every time a nontrivial neural network model works as intended, it's in ungodly defiance of all reason and philosophy.

rotting bones wrote: ↑Tue Feb 07, 2023 10:07 pm Imagine the space of all possible systems of linear equations with a fixed number of variables. Basically, the deep learning model tries to select from this space one system that matches the given inputs to the corresponding outputs.

It gets worse… from what I can recall, neural networks can learn arbitrary nonlinear functions! (Even shallow networks, IIRC.)

bradrn wrote: ↑Tue Feb 07, 2023 11:50 pm

rotting bones wrote: ↑Tue Feb 07, 2023 10:07 pm Imagine the space of all possible systems of linear equations with a fixed number of variables. Basically, the deep learning model tries to select from this space one system that matches the given inputs to the corresponding outputs.

It gets worse… from what I can recall, neural networks can learn arbitrary nonlinear functions! (Even shallow networks, IIRC.)

To be exact, each layer is one system of linear equations followed by one nonlinear activation function. Without the nonlinearities artificially introduced after each layer, a simple neural network is unable to learn the XOR function IIRC. But with the nonlinearities, the network approximates a pixellated version of arbitrary nonlinear functions, seeing the data as a sort of Minecraft world.

I can't think of a way to explain that right now. But I don't think humans can manually find even one system of linear equations with 500 variables.

Another point of clarification is that a 3D line is not a "line" but a "plane". Similarly, the n dimensional lines are "hyperplanes".

rotting bones wrote: ↑Tue Feb 07, 2023 10:07 pm I have doubts that procedural methods can match the performance of deep learning. In theory, neural network models aren't doing anything that can't be hand engineered. In practice, this requires the programmer to be effectively omniscient.

I can't imagine a low level programmer pulling that off with any reliability even after months of research. Of course, once you've determined the system of linear equations, you can study it, combine it, and do all sorts of other things to it. But how would you obtain that oracle in the first place if not by training a network?

Did you read the transcript I provided? The idea is not that a human should come up with the neural network themselves. The idea is that once it exists, it can be analyzed, and often simplfiied.