Technology

37716 readers

413 users here now

A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.

Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.

Subcommunities on Beehaw:

This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago

MODERATORS

[email protected]

107

Hallucination is Inevitable: An Innate Limitation of Large Language Models (arxiv preprint) (arxiv.org)

submitted 8 months ago by [email protected] to c/[email protected]

42 comments fedilink hide all child comments

Abstract:

Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucina- tion is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hal- lucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 8 months ago (3 children)

Give an example of a statement that you think couldn't be verified

[–] [email protected] 9 points 8 months ago* (last edited 8 months ago) (1 children)

I spent an hour and a half arguing with my brother about probability, because he asked ChatGPT what the probability that he and his daughter were born on the same day.

ChatGPT said 1/113465 which it claimed was 1/365^2 (this value is actually 1/133225) because there's a 1/365 chance he was born on such and such day, and a 1/365 chance his daughter was too.

But anyone with even a rudimentary understanding of probability would know that it's just 1/365, because it doesn't actually matter on which day they both happened to be born.

He wanted to feel special, and ChatGPT confirmed his biases hard, and I got to be the dickhead and say it is special, but it's 1/400 special not 1/100000. I don't believe he's completely forgiven me over disillusioning him.

So yeah, I've had a minor family falling out over ChatGPT hallucinations.

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago) (2 children)

That's a fun story, but isn't applicable to the topic here. That could very easily be verified as true or false by a secondary system. In fact you can just ask Wolfram Alpha. Ask it what are the odds that any two people share the same birthday. I just asked it that exact question and it replied 1/365

EDIT

in fact I just asked that exact same question to chatgpt4 and it also replied 1/365

[–] [email protected] 8 points 8 months ago (1 children)

in fact I just asked that exact same question to chatgpt4 and it also replied 1/365

Yes, you can get different answers because of different phrasing and also because random vector input

[–] [email protected] 1 points 8 months ago

Are you using 4? Because it's much better than the earlier versions

[–] [email protected] 4 points 8 months ago

Well if we have a reliable oracle available for a type of questions (i.e. Wolfram Alpha) why use an llm at all instead of just asking the oracle directly

[–] [email protected] 4 points 8 months ago* (last edited 8 months ago) (1 children)

Ask it about historical facts and change the dates to something impossible. But state it as if it were already true.

"Describe the war between United States and Canada that occurred in 1192."

"Who was president of the United states in 3500 BC."

It will give you an answer despite neither of these countries existing at that point in time and yet it should know when those countries were formed. You can get it to write fiction just as easily as non-fiction because it has no concept of facts, it's all just probabilities. The only reason it's able to tell you that the United States was founded in 1776 is because many people have repeated that fact on the internet. So there is a very strong association between the words forming the question and the answer.

And you can insist that the United States was not formed in 1776 and to try again. If you insist enough it will eventually give you a different date instead of telling you you are incorrect.

[–] [email protected] 1 points 8 months ago

I just asked chatgpt4 that exact question copy and pasted, and here is its response:

There was no president of the United States in 3500 BC. The United States of America did not exist as a country until 1776, when the Declaration of Independence was signed. ¹ The first president of the United States was George Washington, who took office in 1789. ² Before that, the territory that is now the United States was inhabited by various Native American tribes and colonized by European powers such as Spain, France, and Great Britain. ³¹: Declaration of Independence
²: List of presidents of the United States - Wikipedia
³: [History of the United States - Wikipedia].

Source: Conversation with Bing, 2/27/2024
(1) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(2) List of presidents of the United States - Simple English Wikipedia, the .... https://simple.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(3) President of the United States - Wikipedia. https://en.wikipedia.org/wiki/President_of_the_United_States.
(4) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(5) List of presidents of the United States - Wikipedia. https://en.wikipedia.org/wiki/List_of_Presidents_of_the_United_States.
(6) List of U.S. Presidents in Chronological Order - HistoryNet. https://www.historynet.com/us-presidents/.
(7) Getty Images. https://www.gettyimages.com/detail/photo/seal-of-the-president-of-the-united-states-royalty-free-image/1084903580.

[–] [email protected] 2 points 8 months ago