this post was submitted on 14 Apr 2024
127 points (100.0% liked)

TechTakes

1430 readers
116 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago
MODERATORS
[email protected]
127
"Not all AI content is spam, but I think right now all spam is AI content." (www.theregister.com)
submitted 7 months ago by [email protected] to c/[email protected]
24 comments fedilink hide all child comments
all 28 comments
sorted by: hot top controversial new old
[–] [email protected] 37 points 7 months ago* (last edited 7 months ago)

plus: this aligns with my preconceptions, which feels nice

minus: the guy saying this is CEO of a company that sells supposedly ai detecting software and is therefore completely untrustworthy, because (1) his job is to say his company is as urgently needed as possible (2) his product is presumably fake which immediately classifies him as a grifter

[–] [email protected] 24 points 7 months ago (2 children)

correlation? between the rise in popularity of tools that exclusively generates bullshit en masse and the huge swelling in volume of bullshit on the Internet? it's more likely than you think

it is a little funny to me that they're taking about using AI to detect AI garbage as a mechanism of preventing the sort of model/data collapse that happens when data sets start to become poisoned with AI content. because it seems reasonable to me that if you start feeding your spam-or-real classification data back into the spam-detection model, you'd wind up with exactly the same degredations of classification and your model might start calling every article that has a sentence starting with "Certainly," a machine-generated one. maybe they're careful to only use human-curated sets of real and spam content, maybe not

it's also funny how nakedly straightforward the business proposition for SEO spamming is, compared to literally any other use case for "AI". you pay $X to use this tool, you generate Y articles which reach the top of Google results, you generate $(X+P) in click revenue and you do it again. meanwhile "real" business are trying to gauge exactly what single digit percent of bullshit they can afford to get away with putting in their support systems or codebases while trying to avoid situations like being forced to give refunds to customers under a policy your chatbot hallucinated (archive.org link) or having to issue an apology for generating racially diverse Nazis (archive).

[–] [email protected] 9 points 7 months ago* (last edited 7 months ago)

using AI to detect AI garbage

It's like an ass backward halting problem that pivots on actually taking time to choose and license training data... but where's the financial incentive in that?

Move fast and break everything, I guess.

[–] [email protected] -4 points 7 months ago (5 children)

it is a little funny to me that they're taking about using AI to detect AI garbage as a mechanism of preventing the sort of model/data collapse that happens when data sets start to become poisoned with AI content. because it seems reasonable to me that if you start feeding your spam-or-real classification data back into the spam-detection model, you'd wind up with exactly the same degredations of classification and your model might start calling every article that has a sentence starting with "Certainly," a machine-generated one. maybe they're careful to only use human-curated sets of real and spam content, maybe not

Ultimately, LLMs don't use words, they use tokens. Tokens aren't just words - they're nodes in a high-dimensional graph... Their location and connections in information space is data invisible to humans.

LLM responses are basically paths through the token space, they may or may not overuse certain words, but they'll have a bias towards using certain words together

So I don't think this is impossible... Humans struggle to grasp these kinds of hidden relationships (consciously at least), but neural networks are good at that kind of thing

I too think it's funny/sad how AI is being used... It's good at generation, that's why we call it generative AI. It's incredibly useful to generate all sorts of content when paired with a skilled human, it's insane to expect common sense out of something easier to gaslight than a toddler. It can handle the tedious details while a skilled human drives it and validates the output

The biggest, if rarely used, use case is education - they're an infinitely patient tutor that can explain things in many ways and give you endless examples. Everyone has different learning styles - you could so easily take an existing lesson and create more concrete or abstract versions, versions for people who need long explanations and ones for people who learn through application

[–] [email protected] 10 points 7 months ago (1 children)

Ultimately, LLMs don’t use words,

LLM responses are basically paths through the token space, they may or may not overuse certain words, but they’ll have a bias towards using certain words together

so they use words but they don't. okay

this is about as convincing a point as "humans don't use words, they use letters!" it's not saying anything, just adding noise

So I don’t think this is impossible… Humans struggle to grasp these kinds of hidden relationships (consciously at least), but neural networks are good at that kind of thing

i can't tell what the "this" is that you think is possible

part of the problem is that a lot of those "hidden relationships" are also noise. knowing that "running" is typically an activity involving your legs doesn't help one parse the sentence "he's running his mouth", and part of participating in communication is being able to throw out these spurious and useless connections when reading and writing, something the machine consistently fails to do.

It’s incredibly useful to generate all sorts of content when paired with a skilled human

so is a rock

It can handle the tedious details while a skilled human drives it and validates the output

validation is the hard step, actually. writing articles is actually really easy if you don't care about the legibility, truthiness, or quality of the output. i've tried to "co-write" short-format fiction with large language models for fun and it always devolved into me deleting large chunks -- or even the whole -- output of the machine and rewriting it by hand. i was more "productive" with a blank notepad.exe. i've not tried it for documentation or persuasive writing but i'm pretty sure it would be a similar situation there, if not even more so, because in nonfiction writing i actually have to conform to reality.

this argument always baffles me whenever it comes up. as if writing is 5% coming up with ideas and then the other 95% is boring, tedium, pen-in-hand (or fingers-on-keyboard) execution. i've yet to meet a writer who believes this -- all the writing i've ever done required more-or-less constant editorial decisions from the macro scale of format and structure down to individual choices. have i sufficiently introduced this concept? do i like the way this sentence flows, or does it need to go earlier in the paragraph? how does this tie with the feeling i'm trying to convey or the argument i'm trying to put forward?

writing is, as a skill, that editorial process (at least to one degree or another). sure, i can defer all the choice to the machine and get the statistically-most-expected, confusing, factually dubious, aimless, unchallenging, and uncompelling text out of it. but if i want anything more than that (and i suspect most writers do), then i am doing 100% of that work myself.

[–] [email protected] 4 points 7 months ago

this is about as convincing a point as “humans don’t use words, they use letters!” it’s not saying anything, just adding noise

I'm sorry I communicate exclusively in mouthnoises, optionally delivered as Ritual Sigils Riding Beams Of Light

[–] [email protected] 9 points 7 months ago (2 children)

nodes in a high-dimensional graph

for people without a technical background: this is gibberish

[–] [email protected] 6 points 7 months ago

at least if it was "vectors in a high-dimensional space" it would be like. at least a little bit accurate to the internals of llm's. (still an entirely irrelevant implementation detail that adds noise to the conversation, but accurate.)

[–] [email protected] 3 points 7 months ago

for people with a technical background, it's 120d8 psychic damage

[–] [email protected] 9 points 7 months ago (1 children)

The biggest, if rarely used, use case is education - they’re an infinitely patient tutor that can explain things in many ways and give you endless examples.

No. They're not.

[–] [email protected] -5 points 7 months ago (2 children)

They're famously terrible at math, you can relatively easily offload that to a conventional program

I didn't mean for children (aside from generating learning materials). They can be wrong - it's crippling to teach the fundamentals wrong, and children probably lack the nuance to keep from asking leading questions

I meant more for high school, college, and beyond. I've been using it for programming this way - the docs for what I'm using suck and are very dry, getting chat gpt to write an explanation and examples is far more digestible. If you ask correctly, it'll explain very technical topics in a relatable way

Even with math, you could probably get a better calculus education than I got... It'll be able to explain concepts and their application - I had zero interest in calculus because I little explanation on why I should learn it or what it's good for, I only really started to learn it when it came up in kerbal space program and I had a reason

But you should never trust its math answers lol

[–] [email protected] 5 points 7 months ago

So, you're fine with psychologically torturing Black people because software manuals are too dry.

Good to know.

[–] [email protected] 8 points 7 months ago* (last edited 7 months ago) (1 children)

education

lol what

[–] [email protected] 0 points 7 months ago (2 children)

Try reading something like Djikstra's algorithm on Wikipedia, then ask one to explain it to you. You can ask for a theme, ask it to explain like you're 5, or provide an example to check if you understood and have it correct any mistakes

It's fantastic for technical or dry topics, if you know how to phrase things you can get quick lessons tailored to be entertaining and understandable for you personally. And of course, you can ask follow up questions

[–] [email protected] 8 points 7 months ago

Try reading something like Djikstra’s algorithm on Wikipedia, then ask one to explain it to you.

I did! I feel entitled to compensation now!

[–] [email protected] 8 points 7 months ago

Education? Really? You think that a good use for the essentially-unverifiable synthesis engine that generates without provenance or reference is good for education? Really?

I guess you must’ve learned that stance from a LLM