Technology

Come on now, next you’ll be saying the tech industry consistently overplays its incremental improvements as Earth-shattering paradigm shifts purely for the investment money!

This message posted from the metaverse

Reading this comment section is so strange. Skepticism about generative AI seems to have become some kind of professional sport on the internet.

Consensus in our group is that generative AI is a great tool. Maybe not perfect, but the comparison to the metaverse is absurd: no one asked for the metaverse or needed it for anything, as opposed to several cases where GPT has literally bailed us out of a difficult situation. e.g. some proof of concept needed to be written in a programming language that no one in the group had enough experience with. With no GPT, this could have easily cost someone a week. With GPT assistance -- proof of concept ready in less than a day.

Generative AI does suffer from a host of problems. Hallucinations, jailbreaks, injections, reality 101 failures, believe me I've encountered all these intimately as I've had to utilize GPT for some of my day job tasks, often against its own better judgment and despite its own woefully lacking capacity to deal with the task. What I think is interesting is a candid discussion: why do these issues persist? What have we tried? What techniques can we try next? Are these issues intractable in some profound sense, and constitute a hard ceiling for where generative AI can go? Is there an "impossibility theorem for putting AI on autopilot"? Or are these limitations just artifacts we can engineer away and route around?

It seems like instead of having this discussion, it's become in vogue to wave around the issues triumphantly and implicitly declare the field successfully dunked on, and the discussion over. That's, to be blunt, reductive. Smartphones had issues, the early internet had issues. Sure, "they also laughed at Bozo the clown" and all that, but without a serious discussion of the landscape right now, of how far away we are from mitigating these issues and why, a lot of this "ha ha suck it AI" discourse strikes me as deeply performative. Like, suppose a year from now OpenAI solves hallucinations. The issue is just gone. Do all the cool kids who sneered at the invented legal precedents, crafted their image as knowing better than the OpenAI dweebs, elegantly implied how hallucinations are a cornerstone in how the entire field is a stupid useless dead end -- do they lose any face? I think they don't. I think this is why this sneering has become such a lucrative online professional sport.

3 months ago: Everyone’s going to lose their jobs!

Today: Generative AI’s dead!

More realistically: Generative AI is a tool that will gradually get better over time. It is not universally applicable, but it does have a lot of potential applications. It is not going to take over the world, nor will it just suddenly go away.

"If hallucinations aren't fixable, generative AI probably isn't going to make a trillion dollars a year," he said. "And if it probably isn't going to make a trillion dollars a year, it probably isn't going to have the impact people seem to be expecting," he continued. "And if it isn't going to have that impact, maybe we should not be building our world around the premise that it is."

Well he sure proves one does not need an AI to hallucinate...

In the early 1980s, a teacher refused to let me word-process my homework (my penmanship was shit) on the grounds that I shouldn't be able to produce a paper at the touch of a button.

Upper management look at AI end results and imagine a similar scenario: they don't see the human effort behind the dumb-waiter and imagine a clerk can just tell an LLM to make me a sequel to Dumbo without getting very specific and then having a team of reviewers watch hundreds of terrible elephant films to curate the few good ones.

But what is telling is how our corporate bosses responded to the prospect of automated art. Much like the robot pizza company who did not automate the process and pass the savings on to you! (his offerings were typical pizza at typical prices and he kept all the savings for himself) our senior execs imagine ways to replace workers with cheaper automation rather than producing better stuff or cheaper movie tickets for their customers.

So maybe we should growl at them and change the system before they figure out how to actually pay fewer people while keeping more profits.

I can't believe this tech bubble will burst. All the other ones have fared so well.

Feels like a minds war waged by billionaires in this space when you're actually playing with this stuff. All this hype is a joke as are the proprietary junk. Get a decent comp and try offline AI yourself and see what it can do. Try Llama2 70B Q4 GGML. You need a machine with more than 10-12+ cores and at least 64GB of system memory. It really helps to have a Nvidia GPU with 16GB+ but you don't have to have that here. This model can write python snippets like you're searching stack overflow but an order of magnitude faster. If you know basic code elements, branching, and looping, this model can code, and resolve its errors when it gets something wrong by prompting it with the error message. A 30B like WizardLM or Vicuna are almost technically useful, but the 70B is a beast.

Kinda is, sure. The problem is when you become overly reliant on the tech without it being reliable. It's also kinda bad when it causes you to lose skills that you need to maintain it or further it.

AI doesn’t seem to do well when it trains on its own data so I do think there’s a possibility it’s a one trick pony. Once there’s too much AI content in the data it’s trained on it will devolve into nonsense.

Isn't ChatGPT's launch only less than 6 months old or something...

I’m curious about the development of artificial intelligence in the future, and I’m looking forward to seeing what GPT-5 can do. If it’s a huge leap forward, then I will agree that the future will be very different from what we have now. But if it’s only a slight improvement, like Llama 1 vs Llama 2, then large language models (LLMs) might face the same challenges as self-driving cars. They are somewhat functional, but not reliable enough to let you sleep on your commute, and they won’t be for a long time.
It might be impossible to eliminate all the hallucinations from LLMs, but if the next versions are incredibly useful, then we will learn to live with them. For example, currently 30% of chips fail on a wafer, but we still produce more CPUs and they are groundbreaking technology. But even GPT4+ will have a significant impact on our future, especially in education. Every kid will have an AI in their phone that is ready to answer all their questions with minimal effort. This will greatly enhance the intelligence of future generations and make education accessible to almost everyone on earth at a similar high level. But this will not make us all lose our jobs in 10 years.

Ultimately, generative AI are tools, not magic. We're now past the hype phase and are now at the leveling out phase of the S-curve as people realizes that these things are limited.

I think ChatGPT is mostly going to be used as an automated copywriter for emails and resumes and such, whereas diffusion models will find their way into digital artists' workflow.

Life goes on.

I wonder, could AI actually "collapse"? As in, once companies and people start leaving the AI hype space, could the external input become small enough so that the AI to AI input takes over to such a degree that all trained models become essentially useless?

Genuine question: How hard is it to fix A.I. Hallucinations?

Oohh i really love when im listening to music and click on an article and it starts autoplaying it -_-