Many people just want to download an executable that works, so gfoot has kindly provided his binaries in case could be helpful to others.
This archive contains a binary of gdbserver for the GP2X, and a binary of arm-linux-gdb.exe for Windows, targeting the GP2X. No plain gdb on GP2X sadly. He didn't build gdbserver himself, this is the binary that luteijn is distributing - hopefully he doesn't mind it being included here.
Building it yourself
It wasn't very hard to build arm-linux-gdb.exe. Simply follow Dan Kegel's instructions  - download the source, configure it for arm-linux (minus his magic switch, g didn't apply his patch), and building it with make.
As stated previously, Gfoot didn't build gdbserver himself, but Dan describes the simple configuration process in his page, linked above. luteijn actually built this binary so maybe he can help out if anyone has trouble with it.