this post was submitted on 19 Dec 2024
21 points (92.0% liked)

homelab

6729 readers
2 users here now

founded 4 years ago
MODERATORS
[email protected]
21
Why can't we have iGPU AND PCIe lanes? (lemmy.world)
submitted 1 week ago by [email protected] to c/[email protected]
15 comments fedilink hide all child comments
 

I'm currently running a Xeon E3-1231v3. It's getting long in the tooth, supports only 32GB RAM, and has only 16 PCIe lanes. I've been butting up against the platform limitations for a couple of years now, and I'm ready to upgrade. I've been running this system for ~10yrs now.

I'm hoping to future proof the next system to also last 8-10 years (where reasonable, considering advancements in tech and improvements in efficiency), but I'm hitting a wall finding CPU candidates.

In a perfect world, I'd like an Intel with iGPU for QuickSync (HWaccel for Frigate/Immich/Jellyfin), AND I would like the 40+ PCIe lanes that the Intel Xeon Scalable CPUs offer.

With only my minimum required PCIe devices I've surpassed the 20 lanes available on desktop CPU's with an iGPU:

  • Dual m.2 for Proxmox ZFS mirror (guest storage) - in addition to boot drive (8 lanes)
  • LSI HBA (8 lanes)
  • Dual SFP+ NIC (8 lanes)

Future proofing:

High priority

  • Dedicated GPU (16 lanes)

Low priority

  • Additional dual m.2 expansion (8 lanes)
  • USB expansions for simplified device passthrough (Coral TPU, Zigbee/Zwave for Home Aassistant, etc) (4 lanes per card) - this assumes the motherboard comes with at least 4-ports
  • Coral TPU PCIe (4 lanes?)

Is there anything that fulfills both requirements? Am I being unreasonable or overthinking it? Is there a solution that adds GPU hardware acceleration to the Xeon Silver line without significantly increasing power draw?

Thanks!

top 15 comments
sorted by: hot top controversial new old
[–] [email protected] 14 points 1 week ago* (last edited 1 week ago) (1 children)

iGPU is just integrated GPU on the CPU die. That is going to use pcie lanes for communication.

Wiring up a iGPU, as a cpu architect, you have two options:

  • direct interconnects (low latency, no space, no extra heat)
  • MUXed interconnects (latency, complexity, space, and heat on die), but even then you would have to choose between using the iGPU and having external PCIe lanes anyway

I think most designers have gone with direct interconnects

Sounds like your real requirement is just more pcie lanes, I believe epyc chips will provide in abundance

https://www.amd.com/en/products/processors/chipsets/am5.html

You can look at pci-e lanes available by model here.

Also you can use newegg to search moterboards by usable pci-e lanes.

[–] [email protected] 3 points 1 week ago (1 children)

Thanks. I'll be the first to admit a lack of knowledge with respect to CPU architecture - very interesting. I think you've answered my question - I can't have QuickSync AND lanes.

Given I can't have both, I suppose the question pivots to a comparison of performance-per-watt and number of simultaneous streams of an iGPU with QuickSync vs. a discrete GPU (likely either nVidia or Intel ARC), considering a dGPU will increase power usage by 200W+ under load (27c/kWh here). Strong chance I am mistaken though, and have misunderstood QuickSync's impressive capabilities. I will keep reading.

I think the additional lanes are of greater value for future proofing. I can just lean on CPU without HWaccel. Thanks again!

[–] [email protected] 4 points 1 week ago (1 children)

If power consumption is an issue then I'd recommend the arc a310 which can only draw up to 30 watts. I've been using one for a while and it can easily handle several 4k streams without issue.

[–] [email protected] 2 points 1 week ago

Thanks. This is a pretty compelling option. I hadn’t looked at the entry level arc, but when it comes to encode/decode it seems all the tiers are similar. 30W is okay, and it’s not a hard limit or anything, just nice to keep bills down!

[–] [email protected] 7 points 1 week ago (2 children)

Sounds like it may be time to consider threadripper, combined with an Intel arc/battlemage GPU, if you want more pcie lanes and quicksync

[–] [email protected] 4 points 1 week ago* (last edited 1 week ago)

Mine's got all the slots filled and I think I still have spare PCIe lanes, Threadrippers are nuts.

[–] [email protected] 2 points 1 week ago (1 children)

I think this is where I'm headed. Is there anything to consider with Threadripper vs Epyc? I'm seeing lots of CPU/MOBO/RAM combo's on ebay for 2nd gen Epyc's. Many posts on reddit confirming the legitimacy of particular sellers, plus paypal buy protections have me tempted.

[–] [email protected] 2 points 1 week ago (1 children)

I think the 3rd Gen epic and threadripper were the first ones with pcie gen4. You might want to look at something a little newer if you're wanting to get 10 years out of it. I'm personally looking at trx50 and a 7960x, but still saving up for that $1300 processor

[–] [email protected] 1 points 1 week ago (1 children)

The options I’m looking at have PCIe 4 and seem to be gen 2? Epyc 7282 or 7302.

[–] [email protected] 2 points 1 week ago

I didn't realize that, and it looks like you're right, that could be a good deal

[–] [email protected] 6 points 1 week ago* (last edited 1 week ago) (1 children)

Have you considered AMD? The 70 class boards basically have an extra I/O die as the chipset on the board. I have this board with a 7700x which is way overkill. I put an A380 in the bottom slot for frigate and Plex, but honestly the amd igpu handles Plex just fine. I never tried frigate with the igpu.

https://pg.asrock.com/mb/AMD/X670E%20PG%20Lightning/index.asp#Specification

[email protected]
[email protected]
[email protected]
[email protected], full 16 slot

E-key for coral

M-key gen5x4
M-key gen3x4
M-key gen4x2
M-key gen4x4

4 x SATA3

[–] [email protected] 3 points 1 week ago (2 children)

I hadn’t considered AMD, really only due to the high praise I’m seeing around the web for QuickSync, and AMD falling behind both Intel and nvidia in hwaccel. Certainly will consider if there’s not a viable option with QS anyway.

And you’re right, the south bridge provides additional PCIe connectivity (AMD and Intel), but bandwidth has to be considered. Connecting a HBA (x8), 2x m.2 SSD (x8), and 10Gb NIC (x8) over the same x4 link for something like a TrueNAS VM (ignoring other VM IO requirements), you’re going to be hitting the NIC and HBA and/or SSD (think ZFS cache/logging) at max simultaneously, saturating the link resulting in a significant bottleneck, no?

[–] [email protected] 4 points 1 week ago (1 children)

The chipset link is 4x4.0 and daisy chained so 8GB per second. My use case is way more casual than you're looking for.

I think what's your up against is Intel locking features behind a paywall, like they have with desktop ECC and hyper threading thru the years.

[–] [email protected] 3 points 1 week ago

Thanks, I'll need to have a look at how the chipset link works, and how the southbridge combines incoming PCIe lanes to reduce the number of connections from 24 in my example, to the 4 available. Despite this though, and considering these devices are typically PCIe 3.0, operating at the maximum spec, they could swamp the link with 3x the data it has bandwidth for (24x3.0 is 23.64GB/s, vs 4x4.0 being 7.88GB/s).

[–] [email protected] 2 points 1 week ago* (last edited 1 week ago)

You could always add an Intel GPU in a PCIe slot, if you're going for that kind of high end build.

Alternatively if you run an Intel iGPU you don't need a Coral TPU either, as Frigate can use OpenVINO and it works as good as the Coral or better anyways.

Also if the LSI HBA is connecting to HDDs, it won't need very much bandwidth so I'm not sure if the lane restriction there would matter?