I have no real lag. It is no less smooth than my iPad or my Nexus 7. The only hint of lag I ever see is that I will sometimes get a screen redraw. Otherwise, everything is fast.
I do not use Touchwhiz at all, and I disabled all the Samsung features and VZW crapware. Otherwise, I have not made many alterations.
I suspect the problems you guys are having could be cured by Apex or Nova.