I thought my GPS just plain didn't work, but after leaving it alone for 3 hours I came back to see that it found my location. However, 3 hours is a wee bit much for turn-by-turn navigation.
Hey man... step back from the phone and take a deep breath.
Even car GPS units work like this... just not as slow. Here is why:
When you first get your phone, it has absolutely NO idea where to search for satellites. It has to check 360 degrees on nearly every axis all around it until it finally locks in to your location.
This only happens the first time. From here on out, it will pick up your signal within a minute or two after you turn your GPS on... assuming you are in the same area (state etc.)
Dedicated car GPS units get around this by pre-programming units with what is called "GPS quick fix" files before being shipped for sale. The GPS quick fix is nothing more than a file which eliminates any of the lines the GPS would normally search that are unneeded. (if it were to try to find GPS signals in Japan, for example... the lines would be different than in Ohio). This allows the units, on "cold boot" (no basic idea of the signal), to find it within 5-6 minutes.
Once you let the phone find your GPS location for the first time, it will have your basic area. If you turn it off and fly across the country... it may take 10-20 minutes when you activate it there since you are so far away. Otherwise, you should be fine.
This is the exact reason your phone asks for your wifi connection, as well. it uses your wifi to narrow down your location, and thus, make the connection time shorter. If you don't have wifi, or if it has a shoddy connection, this won't help. Perhaps why yours took more than 3 hours, and mine took 30 minutes?
In any event... like I said... I wouldn't worry about it too much once you've done it for the first time.