Seeds is a great idea. If in building a big base and I can't easily portal back to my main one, I like to build a cabin to live out if while building. Make sure it has the essentials like chests and crafting station and bed, and you feel way less rushed while building.
Also, plant your garden as soon as you land your boat, let your first harvest start growing while you build.