These are a few notes on sourcing gdb. See gdb for notes on using it on the GP2X.
Many people just want to download an executable that works, so gfoot has kindly provided his binaries in case could be helpful to others.
This archive contains a binary of gdbserver for the GP2X, and a binary of arm-linux-gdb.exe for Windows, targeting the GP2X. No plain gdb on GP2X sadly. He didn't build gdbserver himself, this is the binary that luteijn is distributing - hopefully he doesn't mind it being included here.
Building it yourself
It wasn't very hard to build arm-linux-gdb.exe. Simply follow Dan Kegel's instructions  - download the source, configure it for arm-linux (minus his magic switch, g didn't apply his patch), and building it with make.
As stated previously, Gfoot didn't build gdbserver himself, but Dan describes the simple configuration process in his page, linked above. luteijn actually built this binary so maybe he can help out if anyone has trouble with it.