this post was submitted on 06 Jul 2024
113 points (91.2% liked)
Technology
59285 readers
4685 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
That's not what an algorithms researcher means when we talk about "understanding". Obviously we know the mechanism by which it operates, it's not an unknown alien technology that dropped into our laps.
Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?
Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.
I don't believe I've suggested anywhere that I think it will, but I'll play around with this concern anyway... There's a lot of discussion going on about having models feed back on themselves to learn from their own output. I don't find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it's not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don't think it's all that far fetched to consider the possibility of it happening.
There are plenty of nondeterministic algorithms. It's not a special trait. There are plenty of algorithms with actual emergent behavior, which LLMs don't have to any meaningful extent. We absolutely understand how LLMs work
The answer to both of your questions is not some unsolved mystery. It's "of course not". That's not what they do and fundamentally requires a much more complex architecture to even approach.
Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.