Settings > Sound > Vibrate on Touch
Turns haptic feedback on and off from the 4 capacitive buttons and some on screen functions. I have never used the stock ICS keyboard so I am not knowledgeable about that.
THAT LITTLE KEYBOARD DRIVES ME NUTS.
SwiftKey has its own haptic feedback settings in Apps > SwoftKey 3 > Advanced.