I didn't see this explained anywhere else, but thought it would be helpful.
In GB, I had created a homescreen voicemail icon with my passcode and "new messages" button. Here's how I did it on ICS.
The default icon is a preset contact and cannot be customized. Not only can you customize one with your passcode, you can also add text to the icon.
1. Edit your voicemail contact. Add a new phone number that is *86,[your passcode]
#1 and the '#1' is optional. This takes you immediately to new messages.
2. Then click in the category field just to the right of the number and scroll down to Custom. Add a label name. I used VZW as I want to distinguish this direct dial voicemail from my home voicemail. Click Save at top.
3. Now add a Direct Dial widget to your home screen, and it will pop up your contacts for you select one. Select the new voicemail number with custom category you just created.
The icon will be called Voicemail and have the custom label you added embedded on it!