1) Factory reset means wiping the phone clean close to factory condition. Doesn't touch the USB storage and microSD but it will clean the 2GB internal phone memory. You lose all apps, settings and contacts. So if you want to do this, you have to backup as much as you can with any available backup methods or apps.
Although if your problem is not serious enough to require this solution, leave it as last resort.
2) Not sure above handcent, but GoSMS have lots of settings. There is a setting for pop up notification which you can off. You should also off the notification from the stock sms or the other sms app if you don't want double notification. Just go through the settings for handcent and GoSMS carefully. Its all in there.
Good luck