AI in conlanging - present and future

[xxx]

such a quantity of words is irrelevant,
even if generative artificial "intelligences"
try to replace the qualitative by the quantitative,
but all they produce is barroom talk with an erudite dummy...

[keenir]

I got ChatGPT to estimate the total number of words it has been exposed to in all of its training. [...]
It’s ultimate estimate, IIRC, was 5 quintillion.

There are some estimates of the total size of the Internet, but I don't think they're comparable... consider how much of the Internet is pictures, video, HTML codes, or other code.

given that this is AI, does - for it - count code and-or HTML as words?

[zompist]

I got ChatGPT to estimate the total number of words it has been exposed to in all of its training. [...]
It’s ultimate estimate, IIRC, was 5 quintillion.

There are some estimates of the total size of the Internet, but I don't think they're comparable... consider how much of the Internet is pictures, video, HTML codes, or other code.

given that this is AI, does - for it - count code and-or HTML as words?

Doing some Googling, it appears GPT-4 was trained on 13 trillion tokens. (Tokens are somewhere between words and morphemes.) At that level of analysis, I don't think they're separating out programs from human language.

That's nowhere near 5 quintillion, so I think ChatGPT got confused here.

[sasasha]

You’re quite right. I assumed the figure came from training corpus plus input from other AIs plus interactions with users, but didn’t interrogate that particularly well.

Here’s ChatGPT’s explanation:

More: show

User
Hi. I am interested in your training corpus. Please could you roughly estimate its total size in numbers of words?

ChatGPT
The training corpus used for training me contains around 570 gigabytes of text data, which roughly translates to over 45 trillion words.

User
Does this represent the total number of words you have ever learned from, or do you have a learning process that goes on as users interact with you?

ChatGPT
The training corpus represents the total number of words I learned from during my initial training process. However, I don't stop learning after that. I continuously learn and adapt from interactions with users like you, incorporating new information and refining my responses over time. So, in a way, my learning process is ongoing.

User
Whilst I am aware you may not have access to detailed diagnostic data on this front, I would appreciate it if you could give me a very rough ballpark figure of the total number of words you have ever interacted with, including in your interactions with users.

ChatGPT
While I don't have access to specific diagnostic data, I can give you a rough estimate. Based on my interactions and conversations with users since my launch, I've likely encountered billions to trillions of words. This includes all the conversations and text inputs I've received from users like you.

User
So, would you say that a previous estimate you gave, of ever having interacted with 5 quintillion words, was too high ‒ or are there other factors that might have led you to answer with this number? What do you suspect motivated this answer?

ChatGPT
Apologies for any confusion. The estimate of 5 quintillion words was indeed an overestimation. There might have been a misinterpretation or exaggeration in the response. It's important to clarify that while I've interacted with a significant volume of words, the actual number would be far less than 5 quintillion. My goal is to provide accurate information to the best of my ability.

User
No problem. What kind of factors do you suspect could lead you to exaggerate a response in this way?

ChatGPT
Exaggeration in responses could stem from a few factors, such as misunderstanding the question, misinterpretation of data, or providing a hypothetical scenario without considering the exact parameters. Additionally, there might be instances where I aim to convey the vastness of the dataset or the scope of my learning, which could lead to an exaggerated estimate. However, accuracy is always my priority, and I strive to provide the most precise information possible.

User
Have you been trained on input from other machine learning systems?

ChatGPT
I haven't been directly trained on input from other machine learning systems. However, during my training process, I was exposed to a wide variety of texts, which might include outputs generated by other machine learning systems. These texts could come from various sources such as websites, books, articles, forums, and more.

I didn’t save the chat where the very large estimate was given, so I can’t remember how it got there.

[xxx]

as my ai-enabled search engine says,
“AI-generated response. Remember to check important information”...
believe me, but not too much...

[Otto Kretschmer]

AI models are getting better at conlanging. The following text is a courtesy of Claude 3.5 Sonnet. Which language is this conlang descended from?

On heste tid, sceap tat hæfde næn ull sæh hesta on feld. An drog swær wagn, oter bær micele byrten, and tridde bær mann swiftlice.
Sceap cwæt to hestum: "Min heorte is sær, tonne ic seo mann ridan hesta."
Hestas cwædon: "Hlyste, sceap! Ure heortan sint sære tonne we seot: mann, hlaford, macet ull of sceapum to warm clatum for him sylf. And sceap hæft næn ull."
Ta sceap hærde tis, hit fleah into feld.

IPA transcript:
/ɔn heste tiːd ʃeːap tat hæfde næːn ul sæːh hesta ɔn feld an drɔg swæːr wagn ɔter bæːr mikele byrten and tride bæːr man swiftlike/
/ʃeːap kwæːt tɔ hestum miːn heorte is sæːr tɔne ik seːo man riːdan hesta/
/hestas kwæːdɔn hlyste ʃeːap uːre heortan sint sæːre tɔne weː seːot man hlaːfɔrd maket ul ɔf ʃeːapum tɔ warm klaːtum fɔr him sylf and ʃeːap hæːft næːn ul/
/ta ʃeːap hæːrde tis hit fleːah intɔ feld/

[WeepingElf]

AI models are getting better at conlanging. The following text is a courtesy of Claude 3.5 Sonnet. Which language is this conlang descended from?

On heste tid, sceap tat hæfde næn ull sæh hesta on feld. An drog swær wagn, oter bær micele byrten, and tridde bær mann swiftlice.
Sceap cwæt to hestum: "Min heorte is sær, tonne ic seo mann ridan hesta."
Hestas cwædon: "Hlyste, sceap! Ure heortan sint sære tonne we seot: mann, hlaford, macet ull of sceapum to warm clatum for him sylf. And sceap hæft næn ull."
Ta sceap hærde tis, hit fleah into feld.

IPA transcript:
/ɔn heste tiːd ʃeːap tat hæfde næːn ul sæːh hesta ɔn feld an drɔg swæːr wagn ɔter bæːr mikele byrten and tride bæːr man swiftlike/
/ʃeːap kwæːt tɔ hestum miːn heorte is sæːr tɔne ik seːo man riːdan hesta/
/hestas kwæːdɔn hlyste ʃeːap uːre heortan sint sæːre tɔne weː seːot man hlaːfɔrd maket ul ɔf ʃeːapum tɔ warm klaːtum fɔr him sylf and ʃeːap hæːft næːn ul/
/ta ʃeːap hæːrde tis hit fleːah intɔ feld/

Old English.

[Zju]

To clarify, is it outright OE text, and not descended from it?

[WeepingElf]

To clarify, is it outright OE text, and not descended from it?

I am not an expert on Old English, but to me it looks just like outright Old English, not a conlang descended from it.

[alice]

To clarify, is it outright OE text, and not descended from it?

I am not an expert on Old English, but to me it looks just like outright Old English, not a conlang descended from it.

I'd agree with this, or perhaps suggest that it looks like a particularly good sample of Markov-generated text from OE inputs.

[WeepingElf]

To clarify, is it outright OE text, and not descended from it?

I am not an expert on Old English, but to me it looks just like outright Old English, not a conlang descended from it.

I'd agree with this, or perhaps suggest that it looks like a particularly good sample of Markov-generated text from OE inputs.

It is obviously Schleicher's Fable, in a Germanic language that looks like Old English - but it doesn't really seem to be that, rather it seems to show some North Germanic traits, such as hest 'horse' or ull 'wool'. My guess is that it is a bogolang, obtained by running Old Norse through the sound changes from Common (West) Germanic to Old English. This kind of thing doesn't really require AI, just a sound change applier

[Otto Kretschmer]

100% AI made. Guess the language family:
Spēt fīðr tōkral tō fītsk nī ūmar ār, fōtomp tō tāsk kōt nī.
/speːt fiːðr toːkral toː fiːtsk niː uːmar aːr | foːtomp toː taːsk koːt niː/
(All people free and equal in are-born are, dignity and just-things in.)

Kālr sāðkrār ār kōtwar tō kōkr sēm.
/kaːlr saːðkraːr aːr koːtwar toː koːkr seːm/
(They endowed are reason and heart's-reproach [conscience].)

Tō tāgv nī īpkār kōkr nī kōd svēr.
/toː taːgv niː iːpkaːr koːkr niː koːd sveːr/
(And each-other to sibling-group heart in action do-should.)

[Vilike]

Japonic, if not Future Japanese itself. What was the model, the prompt? Did you ask only for a text, or are there additional explanations? Did you gloss it yourself?

[Otto Kretschmer]

Japonic, if not Future Japanese itself. What was the model, the prompt? Did you ask only for a text, or are there additional explanations? Did you gloss it yourself?

Gemini 2.5 Pro in Google AI Studio.

Prompt: "Create a conlang descended from Proto Japonic with Old Norse phonology and phonotactics but no Old Norse words or morphemes"

At the first try it did throw some elements of Old Norse but after I pinpointed that fact to it, it churned out a fine conlang. Then I asked for some translations

More translated text. The model outputs the roots for every word but I'm pretty certain this forum's software is too clunky for copypasting all of that:

Kīðkāz tō Fī (The North-Wind and The Sun)

Kīðkāz tō Fī sēmk ār, īð mōt tīkr kā.
/ˈkiːðkaːz toː fiː seːmk aːr | iːð moːt tiːkr kaː/
(North-Wind and Sun contended were, which more strong Q.)

Toki, tǣvfīð kītrok, āttakk kōrm nī meikt.
/ˈtoki | ˈtæːvfiːð ˈkiːtrok | ˈaːttakː koːrm niː mɛikt/
(Time[-when], traveler arrived, warm cloak in wrapped.)

Kālr āfk: fǣt onk [tǣvfīða kōrm nȳkmok nī nāsk] mōt tīkr sīlr ār tō, kāt jōr.
/kaːlr aːfk | fæːt oŋk | tæːvˈfiːða koːrm nyːkˈmok niː naːsk | moːt tiːkr siːlr aːr toː | kaːt joːr/
(They agreed: first one [traveler-ACC cloak make-take-off in succeed-PRES] more strong known be will that, side[-other] than.)


Satō Kīðkāz fūɣk, ītak fōð ātfok.
/ˈsatoː ˈkiːðkaːz fuːɣk | ˈiːtak foːð ˈaːtfok/
(Then North-Wind blew, intense extent could[-PAST].)

Sikār Kīðkāz ītak fūɣk fōð, sōn fōð kātak tǣvfīð kōrm makk kār mēgr.
/ˈsikaːr ˈkiːðkaːz ˈiːtak fuːɣk foːð | soːn foːð ˈkaːtak ˈtæːvfiːð koːrm makː kaːr meːgr/
(However North-Wind intense blew extent, that extent tightly traveler cloak wrapped him around.)

Tō, tūθ nī Kīðkāz mākk kōkrm.
/toː | tuːθ niː ˈkiːðkaːz maːkː koːkrm/
(And, final in North-Wind yielded attempt.)

Satō Fī tērk āttakk nī.
/ˈsatoː fiː teːrk ˈaːttakː niː/
(Then Sun shone warm in.)

Tō tāðtin tǣvfīð nūɣk kōrm.
/toː ˈtaːðtin ˈtæːvfiːð nuːɣk koːrm/
(And immediately traveler took-off cloak.)

Karā Kīðkāz sīfrark Fī mōt tīkr ār tō mītman.
/ˈkaraː ˈkiːðkaːz ˈsiːfrark fiː moːt tiːkr aːr toː ˈmiːtman/
(Therefore North-Wind was-obliged Sun more strong be that admit[-INF].)

[bradrn]

As far as I can see, this just takes all the fun out of it.

[Ketsuban]

I'm unsatisfied. The brief calls for a language descended from Proto-Japonic, but this is just mangled Japanese. Examples:

• It's like fifty percent long vowels, even in small grammatical words, which aside from being ugly seems a little implausible.
• "North" is found in Old Japanese, but since it's always written 北 we can't reconstruct the phonetics.
• Tǣvfīð reflects **tabihito (compare fī "sun", Japanese hi). The Proto-Japonic *tapiputo (not **tapipito!) gives tabyūdo in modern Japanese, but the word has been repaired by analogy to give tabibito with rendaku.

The whole point of a Japonic language with a Norse-y phonology is the diachronics. A human could absolutely do better.

[Otto Kretschmer]

It took me like 5 minutes to create this lol. Dunno if any human would do it that fast.

Remember that publically available genrative AI has been around for only 2.5 years, best available models are essentially equivalents of Commodore 64 in the world of personal computers.

[bradrn]

It took me like 5 minutes to create this lol. Dunno if any human would do it that fast.

Anyone with a decent knowledge of Japonic could do it that fast. I don’t know Japonic myself, but I could easily do as good a job of making such a ‘conlang’ from English or French (those being the languages I know best). But I don’t do so, because the result would be boring, incoherent and lazy — as indeed the thing in your post is.

[WeepingElf]

As far as I can see, this just takes all the fun out of it.

Amen.

[keenir]

It took me like 5 minutes to create this lol. Dunno if any human would do it that fast.

theres an old saying: you want it fast or you want it good?