Technology

59381 readers

4116 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

435

AI trained on AI garbage spits out AI garbage. (www.technologyreview.com)

submitted 3 months ago by [email protected] to c/[email protected]

62 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 3 months ago

Both sentences are true. And based on vocabulary of both, the model can output the following sentences:

Cats have feathers.

Birds have fur

This is not how the models are trained or work.

Both are false but the model doesn't "know" it. All that it knows is that "have" is allowed to go after both "cats" and "birds", and that both "feathers" and "fur" are allowed to go after "have".

Demonstrably false. This isn't how LLMs are trained or built.

Just considering the contextual relationships between word embeddings that are created during training is evidence enough. Those relationships from the multi-vector fields are an emergent property that doesn't exist in the dataset.

If you want a better understanding of what I just said, take a look at this Computerphile video from four years ago. And this came out before the LLM hype and before ChatGPT 3, which was the big leap in LLMs.