Ugh..this happened to me a while back. I lived with it for about two weeks and then it "sorta/kinda" went away. I'll see if I can recall what I did (other than wait it out)

 

Have you tried simply restarting your phone? (Power off, power on)

 

Have someone call you and leave another voicemail. Once you delete the new one, the notification should go away like it normally does.

 

I would definitely try a factory reset, and install no additional apps until you can determine that the problem is gone. Hope that fixes it. Weird problem for sure.

 

scratch the factory reset...

are you using any other voicemail service, such as google voice?

if not, try going to settings> apps> all> voicemail> force stop and clear data.