Both my friend and I have the same issue with the S5 and I'm on AT&T and he's on Verizon. The phone rings, you swipe the screen to answer, and more often than not the first, second or, sometimes, even third swipe doesn't do it.
He told me this morning, he missed 2 calls just trying to...