Quick update, many people are reporting the exact same poor quality VoIP calls on the Nexus One phone compared to other phones (like the Droid, for example):
VoIP quality on the Nexus One is terrible. - Google Mobile Help
If anyone has similar problems please post so people don't get misinformed that a data only T-Mobile plan will allow them to to use VoIP calls on the Nexus One. The VoIP quality on the Nexus One is terrible.