Agreed. I'd have a hard time going smaller. Or bigger, at least for an every day carry device. So this is the sweet spot for me too on phone size. I knew going in I would not have any issue with it, though. My wife has an S3 and I upgraded from an HTC EVO 4G LTE, both are about the same size.
But size is a personal thing. Small hands, small pockets, personal prefrence, brand bias, etc all enter into the "right size" decision.