Is there a "clear" button on your task bar when you pull it down? It hink I'd try rebooting the phone...I've never had that problem.

 

wait, so you have deleted your voicemails? because it usually leaves that icon when you have voicemails left in your inbox. try deleting them all when you call your voicemail then see if it is still there.

 

maybe theres a setting in device management that deals with your voicemail.. but other than that there has to be something wrong with the phone

 

I used this post from iOlivia for my EVO and it worked:
THIS 100% WORKS!! and its completly easy. (HTC Hero)
Go to your settings, then go to applications, then go to manage applications, then scroll all the way down to voicmail, tap voicemail, then scroll down to force stop. Then tap force stop, and the icon goes away. It temporaraly stops your voicemail, but once you tap on the voicmail app, it starts again without the icon. www.olivia-knowstechnology.blogspot.com