this post was submitted on 02 Dec 2024
384 points (99.0% liked)

Technology

60116 readers
2842 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
384
Intel Announces Retirement of CEO Pat Gelsinger (www.intc.com)
submitted 3 weeks ago by [email protected] to c/[email protected]
142 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 1 points 3 weeks ago* (last edited 3 weeks ago) (1 children)

Also that is a very low context test. A longer context will bog it down, even setting aside the prompt processing time.

...On the other hand, you could probably squeeze a bit more running openvino instead of llama.cpp, so that is still respectable.

[โ€“] [email protected] 2 points 3 weeks ago

text test. A longer co

yeah, it's definitely not good enough for user-facing work, but if I'm working on development for something like translations, being able to see the 70b output to compare it to other models, it's super useful before I send it off to something that costs more money to run.

9/10 times, the bigger model isn't significantly better for what I'm trying to do, but it's really nice to confirm that.