I've been using 5.0 since the factory images were published, both on my Nexus 4 and Nexus 7. No showstoppers on either. It's a a huge change to Android so there will of course be some bugs and minor issues, but I think the benefits of the new OS easily outweigh those small problems. So I would strongly encourage you to try it out.
That said, the change from 4.4.4 to 5.0 is a HUGE one. The entire OS got an overhaul; even the underlying runtime changed from Dalvik to ART. That's a fundamental shift that does have the potential to cause problems under certain circumstances. Any existing file system problems or user data corruption could potentially cause the upgrade to fail.
The biggest thing to be aware of is that every one of your apps will have to be recompiled to work with ART. That will happen as one of the final steps of the upgrade. The nature of ART means that the application cache takes up a lot more space than it did on Dalvik. If you run out of space while that app cache is being rebuilt you'll likely get a bootloop (been there, done that). You can help minimize potential upgrade problems by making sure you have plenty of free space. I think the update says it needs 1GB; I'd encourage you to try for 3 or more. You may also want to run through your list of installed applications and remove any you no longer use or need. Each app you can delete pre-upgrade is one fewer potential problem.
By far the safest way to make sure you don't run into issues is to flash the factory image, complete with a full wipe. You will lose your data and have to set everything back up, but you won't have to worry about any stale data causing issues. The image overwrites the file systems as a whole which also makes sure that all the partitions are in order.
If you do run into issues after the upgrade, a factory reset should generally clear it up. You'll lose your data, but at least get back up and running in a hurry. Any really stubborn issues will be resolved by flashing the factory image.
