Successor, your one and only choice is the PSP from what you've said. If you buy a PSP and a copy of Lumines now, there will soon be a downgrader exploit released. This will allow you to turn back the clock on your PSP to an earlier revision, and then reupgrade with security broken so you can run homebrew and pirated games too. You will need the PSP (I think that's 150$ now) and a Memory Stick which varys in cost depending on the size you want, you should probably get a 1GB stick for emulation and general game saving. You'll want something like a 2GB or 4GB stick for playing PSP games off memory stick. You'll also need a PSP usb transfer cable to put files from your PC on your PSP's memory stick.
All together you should expcet to spend around 200 to 250 to get a PSP, the game with the exploit, a game or two you actually want, memory stick, transfer cable.
The DS is just bad. You have a nice NES emulator, an impressive (technically speaking) SNES emulator, I think a very primative Genesis emulator. The main issues are cost, it'll cost you a bit of money to really get what you should for Ds, about 90$ for this GBA cart that lets you insert flash memory cards into it, a special DS gamecard to bypass the DS slot to the GBA cart device, then you need more money to buy the memory card to put into that. The other problems are the DS has a screen resolution of 256x192 per screen. The NES from 1985 has a resolution of 256x224 (roughly for NTSC). So does SNES. Genesis can do the same for games but also a 320x224 way too large for the DS. Also the DS has too slow of a CPU for nice emulation like you would want and too little memory.
The GP2X is neat, it has an advantage of more memory than the PSP, 64MB instead of 32MB which is handy for NeoGeo emulation. The GP2X is no hassles for running emulators since it's intended for that, the PSP you have issues where every so often you need to go to a site to upgrade to the latest hacked firmware to play newer games. Not a real big issue but it's there. My main concerns about GP2X are that the D-Pad, isn't a dpad. It's a weird thumbstick. Also I'm not sure there's as much support for the device as the PSP for emulation. That may have changed since I last checked. They should both emulate about the same things.
Anyway, I recommend you get a PSP, though you should try one out first. The PSP can emulate your NES, SNES, and Genesis very well with few exceptions. The exceptions would be SNES games that use special enhancement chips like StarFox, any SuperFX game, Super Mario RPG, Kirby Dreamland 3, Kirby SuperStar. Mega Man X2 and X3 are ok. You could also emulate PSX games, NeoGeo, Capcom CPS1 and CPS2, and others.
Really there's nothing else better than it. The GP2X has its advantages over the PSP and the PSP has its over GP2X. Checkout the PSP sites and GP2X sites. There isn't any central hub I've noticed for DS homebrew. DS isn't that notable anyway.
DS emulation on PC has progressed. I believe it's still pretty slow but some games are playable. I can't remember how well it all worked but I was able to play some DS roms.