the only time i get lag on my Note or my S4 is when i background/minimize a few dozen heavy apps (hitting home vs. back. back often closes the foreground app leaving only the services running, hitting home leaves the foreground app going, so it's faster to load when you load it back up again later) but i only keep around 5-6 open on my S4 at one time (no lag at all). there is a slight delay if you back out of an app that's heavy on resources vs. leaving it running though (S-Voice takes a bit longer to load if you back out of it, but hitting home and launching it later has it instantly respond).
I think it has to do with what kind of apps you run and how many you leave going. my S3 lagged like heck with the 4.3 update, and it reminded me how slow my iPhone 4 became with iOS 7. i'm still on 4.3 on my S4, which feels more designed to run it, and runs extremely well in my use, so i wouldn't chance it with the updated KitKat.
Turning off Google Account Sync/deleting Google Play can increase a lot of speed, and perhaps that is why i never see lag now. i never use it. when i do have Play apps going, the sync is off, notifications are off so they just sit there, and never use anything up. i am not sure why, but removing or turning off Google Sync can help tons. Google Now tends to eat a bit of battery too, if you have it going.
in my view, obviously not shared by the majority here, TouchWiz Nature UX on Android 4.3 on the S4 really is beautiful. it combines my love of nature with a half-skeuomorphic, half-flat UI with never too much of either one, and does the high-resolution 1080p display in the S4 justice. it seems the S4 has enough juice to run Android 4.3 with TouchWiz just fine. it really feels like my old iPhone when it ran iOS 6. but i got rid of Google Play which i discovered caused some lag and stability issues on my previous S3, and disabled or turned off any apps or notifications i never want. i hardly have installed anything as everything i use is built-in. that might help too. not sure. i have power saver on but i never get lag.
not sure if my definition for 'lag' is the same as yours either. for me 'lag' is when you open an app, and nothing happens, then it becomes unresponsive and you get a '[app name] is not responding, do you wish to close it?' [wait] [close] prompt, or you turn on your phone, and the lockscreen either does not immediately display, or displays but does not respond to touch for a few seconds, or when you hit the home button and your launcher redraws itself, and half your widgets haven't loaded or need to be restarted. a phone that feels a bit like a 486 running Windows 98SE with tons of malware is lag.
I do not consider microstutter or a bit of un-smooth scrolling in the web browser as lag.