this post was submitted on 12 Nov 2024
13 points (100.0% liked)

Hardware

656 readers
190 users here now

All things related to technology hardware, with a focus on computing hardware.

Rules (Click to Expand):

  1. Follow the Lemmy.world Rules - https://mastodon.world/about

  2. Be kind. No bullying, harassment, racism, sexism etc. against other users.

  3. No Spam, illegal content, or NSFW content.

  4. Please stay on topic, adjacent topics (e.g. software) are fine if they are strongly relevant to technology hardware. Another example would be business news for hardware-focused companies.

  5. Please try and post original sources when possible (as opposed to summaries).

  6. If posting an archived version of the article, please include a URL link to the original article in the body of the post.

Some other hardware communities across Lemmy:

Icon by "icon lauk" under CC BY 3.0

founded 1 year ago
MODERATORS
[email protected]
[email protected]
13
The NPU: Neural processing unit or needless pricey upsell? (www.theregister.com)
submitted 3 days ago by [email protected] to c/[email protected]
6 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 2 points 2 days ago* (last edited 2 days ago) (1 children)

I would like to know how well the NPUs would work on more specialized tasks.

AI up-scaling of video takes about ~30 minutes for a 5 min SD min video (sometimes it can be longer) on my 3080.

I've seen reviews reference the 3080 having ~238 TOPS, which implies that current gen NPUs would take significantly longer in upscaling.

I would be willing to get a dedicated NPU if it offered significantly faster AI upscaling relative to high-end GPUs. Having software and an ecosystem provided by the NPU manufacturer would also be nice.

[–] [email protected] 2 points 2 days ago (1 children)

I see you seem fascinated by NPUs, here is probably one of the most recent publications that discuss performance comparison between NPU, GPU, and CPU for Edge computing case using YoloV5 model. And here's another benchmark provided by Google for its TPU product (Google Coral).

Given the circumstances, it's unlikely for current generation NPU to fulfill those heavy data tasks like upscaling video or generating video but for something like Edge computing they provide fairly performant solutions at affordable prices (in the case of NPU you can get them on Single Board Computer using Rockchip SoC especially RK3588 one). If you're using Raspberry Pi 5 or other SBC, you can get Google Coral (M.2 or USB version).

Lastly, as recent as July 28th 2024, Anandtech AI benchmark on the new Ryzen AI 9 HX 370 still using CPU because the software to benchmark the NPU modules isn't available yet for desktop environment to prevent any bias in result.

[–] [email protected] 1 points 2 days ago (1 children)

Thanks for those links. Will take a look.

I am honestly surprised we don't see more "power user" type benchmarks for measuring AI Compute for use cases such as video upscaling, local LLM (my next project, I use ChatGPT as an elaborate spell checker, but I want to switch away from corporate cloud solutions).