I was facing the same issues. It used to take a lot of time to load the home screen and sometimes once the screen got loaded further time to show all the icons. At times the touch became delayed almost to the point that for a moment i thought it was unresponsive.
I followed your advice and unchecked the "Dont keep activities" option and now all things seem to be working fine.
Since i am new to Androids (HTC OneX was my first), what i would like to know is what is the default setting for "Dont keep activities". If it was supposed to be checked by default then something is wrong with my phone and the current change is just a workaround, so i would still have to figure out and fix the root cause. If it is supposed to be unchecked by default, then maybe i changed it accidentally, so all fine
But that makes me wonder why we have such a deadly option in the first place.