RetroGaming

Miyoo Mini plus and Ambernic RG35xx are probably what you want. I have these two devices and the 353V, all three are quite nice but the MM+ and RG35xx are better to your needs and are probably better for beginners.

Personally I would go for the MM+. Definitely install OnionOS if you get a MM+ or GarlicOS if you go for thr 35xx. (JELOS is my pick for the 353V) These custom firmwares are far more polished and have more performance than the default OS and have tons of great features and community applications.

Every handheld made in the past few years plays GB/GBC/GBA at 100%.

I have both am R4 card for my 3DS and a 353V. Loving the 353 as it covers so many systems. Ripping all the games from my Dreamcast from back in college has been awesome. GB, GBA are flawless on that device

I'm surprised no one mentioned the Analogue Pocket, especially if you had cartridges. It does have support for ROMs as well. One of the best features is that you can sleep your device on even original cartridges

I find playing Nintendo games is always best on Nintendo systems. I use a hacked 3ds for all my handheld games including Gameboy, a hacked Wii U for Gamecube, Wii, and Wii U games, and a hacked Switch for everything else.

All Gameboy Advances play GB, GBC, and GBA. The best GBA is the SP model AG-101 which has a factory backlight screen. However, there are tons of modifications available for Gameboys out there. I say get a used GBA and go to https://www.retromodding.com/ and go to town on it. Customize it the way you want

For ROMs only, the absolute best way to play is to get a Nintendo 3DS or 2DS and install home brew software. And as a bonus you can play the entire DS and 3DS library as well.

FYI the Nintendo DS can play GBA games, but it does not support GB or GBC.

I know you said you wanted handheld, but for anyone else reading my absolute favorite way to play is with a Gamecube and a Gameboy Player! It actually plays all GB, GBC, and GBA games! Gameboy Players can be cheap, but the boot disc is expensive. To get around this there is home brew software for the Gameboy Player that is very good, although I haven't tried it myself. To use it you need an Action Replay and a cheap SD card to GC memory card adapter. The software will boot the Action Replay first, then boot whatever is on the SD card. (I bet there are ways to load roms too but I haven't looked). I play with a Raphnet SNES to Gamecube adapter.