I've had my S4 for awhile now and just disabled S-Voice, so much nicer. I wish I had done it sooner.
I'm one of those people that try to enjoy what the phone comes with. So I'm still using TW and the stock build on the phone. No root, no flash, nothing. I try to enjoy it how it is. I also do a lot of research before I buy, to see if I'll like it or not. But that doesn't mean I didn't make any changes.
First things I did were disable/uninstall any of the bloatware that would allow it. Coming from the iPhone and Nexus 4, it's been awhile since I've had to deal with AT&T bloatware.
I then turned off some of the gestures and air view features I wasn't going to use, along with turning smart screen off completely. I also turned off the multi-window feature. I didn't care for it.
I then cleared off the entire homescreen. All the widgets, apps, etc, and then organized it how I wanted. I have 5 panels turned on, but I only use 3 of them. Haven't quite figured out what to do with the other 2, lol.