I too was a little concerned about the size.
I have found that the One X+, although larger, is no more uncomfortable in my pocket. I would often jigger about with the Desire when I sat in my car or take it out my pocket completely. You can't get away from it though that it is bigger. I have one pair of jeans that it will not be comfortable in as my Desire only just fitted in.
I think it's a trade off though. The bigger screen is a dream to use. My wife has my old Desire and it now seems like an old brick in comparison and that's after only a few days.