Non-removable battery is an instant no-go for me. Batteries do lose their ability to hold a charge over time and when that happens, what will you do? You're at the mercy of your carrier/insurance provider/warranty provider to replace the battery for you. And that usually involves replacing the whole phone. Nope. Sorry, that doesn't fly with me.
If and when that happens with the S4, all I have to do is walk into my carrier's retail store, buy a new battery, pop the back of the phone off, remove old battery and install new battery. Instant new battery!
Much easier than having to rely on getting a new phone just for a new battery.