As some folks here might know, I am kind of picky.... That said, my S4 32gb has not showed any signs of lag with Nova or Wiz and with both firmware versions. It was zippy fast when I first used it and the same with 260 apps installed and new firmware. I do agree the demo units I tested wer a little laggy, but not having any issues with lag. Easily the fastest device I have used- including compared to the speedy iPad 4 and S3.
Of course, like with every device I have- animations are off, extra sensors are off and no live wallpaper. That stuff seems frilly- especially all the eye and hand wave sensor stuff. Turned it all off since just drains battery/resources more.
Wish I did not update though, since root is now currently (or always) not an option.