Cheaper Android phones are generally of poorer build quality and slower hardware. Other than that, the entry level models from trusted vendors like HTC, Samsung etc are still perfectly usable. I would avoid buying some cheap thing from a Chinese web store, they're cheap knock offs and give Android a bad name.
Pick one with Android 2.1 if you can help it, earlier versions can't use all of the apps on the market. Plus it's good to be up to date with the OS.