I'm using 4.4 on my Verizon Galaxy Nexus (Toro) as I type this. Overall, it's very fast and smooth - and although it's purely subjective, 4.4 certainly 'feels' like an improvement over 4.3 to me.
One clear area of improvement is memory usage. With every prior Android version, I never saw free memory much over 200-230 meg with my usual app load. In 4.4, I'm seeing 345 meg free at this moment! And say what you will about Android memory management, but I've always had better performance with more free memory.
All the basics seem to work very well. The only real issue (and it's a big one) comes when you're using any app that relies on a web view: you get large black rectangles of various dimensions that cover sections of the output. Sometimes scrolling around in a long page and/or reloading the page will cause the blocks to move around or become smaller, but they don't disappear. Apps that use a web view include browsers (it happens with both Chrome and Dolphin), email apps that display HTML-formatted messages, etc. Unfortunately, this problem is so severe that it makes 4.4 unusable as a daily driver on a Galaxy Nexus.
My understanding is that the issue stems from the outdated drivers. It's a huge shame, because it looks like everything else either works already, or will soon given the usual debugging needed with any new ROM.
Since TI is out of the chip business for these phones anyway, I can't see why they couldn't at least release the driver code as open source so ROM developers could try to fix the problems themselves. But it looks like we're out of luck unless someone finds a way to resolve the driver issues.