my brother came up with a great analogy - say you’re falling and there’s a portal below you also falling, just slightly slower than you. when you eventually fall through it - do you come out falling slowly or quickly?

it would have the be quickly. even though you and portal are moving slowly relatively to each other, your individual momentum is conserved, the movement of the portal is irrelevant.