A lot of things may be contributing to that issue. Anything from the carrier to the strength of the signal and the voice codes used to put the voice on the air. There are many factors that could be causing it.
The more people on a tower with active phone calls the more they compress the voice data stream. Verizon is known to compress the heck out of the voice stream if there are a lot of people on the tower.