Ok, I was shocked to find out there was no native option to do this in the camera app.
So after searching, I found this
xda-developers - View Single Post - [MOD] Audio Tweaks (Including Silent Camera)
You have to be rooted to do this.
Using your favorite file explorer, go to system/media/audio/ui folder
Locate these files:
cam_start.ogg, cam_stop.ogg, camera_click.ogg, camera_click_short.ogg, shutter.ogg, shutter_multiple.ogg, and videorecord.ogg
rename the files and change .ogg > .old
After that you are done.
If you are a flashoholic, then you will need to do this after each flash of a rom as the files are fixed.
Takes ~1 min to do.