Is your GPS on? Literally, I mean. When my Droid first tried to find my location it was quite a bit off, but once I turned the GPS on, it was perfect. There's a widget that keeps track of wi-fi/bluetooth/GPS/sync/brightness. (It's a good widget to have to keep track of battery life.) I suggest installing that little dude and make sure your GPS is on.

 

what adara said. pretty sure what is happening is it is using Tringulation of towers to pin point you to a certain area.