Do you know what the exact error through recovery was?
I'd have to see the output from the recovery to try and help with better answer.
I think it's because you wiped the system partition. So follow these instructions and you should be OK.
Quote from cataclysm -
- Don't wipe system. Don't mount system. If you did, reboot again to recovery before flashing.
If you haven't already, I'd also recommend updating your N5's bootloader, radio/modem and recovery to the newst versions for 5.0+ compatibility. (TWRP 2.8.1 is a good one)
You can either fastboot flash the newest bootloader, radio/modem or TWRP through your PC or you can also find all the files in a flashable zip that you can flash through your current recovery.
If you haven't, I would make sure that you're updated in these areas first.
If you need any help flashing the bootloader, radio or TWRP just ask and hope this helps