Not exactly sure what you mean (so this may or may not be of much help - fair warning... and I've had a few beers tonight ).

To enable USB debugging, assuming you're using the stock ROM, you need to enable the "hidden" developer menu by going into the settings menu, about phone, then tapping repeatedly on the build number. Then the developer options menu should show up in the settings menu. You can enable USB debugging from there.

Connecting up to your PC, I think you also need to install the drivers from Google for it instead of just the normal default drivers that Windows loads. So, while in debugging mode, connect the Nexus 5 up to your PC and it should try (unsuccessfully) to install drivers for it. You'll want to find this in your device manager in the control panel in Windows. Choose to install the custom drivers that you downloaded for it.

The Google drivers can be found here: Google USB Driver | Android Developers

There's likely more you can do with the ADK... kit (yes, I realize this is redundant - it is intentional) interface, though I'm not as familiar with this as I don't really have much use for it for my needs.

I know to get the Nexus 5 set up for rooting using some methods in fastboot mode, et cetera, you need the Google drivers as well. (Edit: Upon reading your post again, this may be more relevant to your needs.)

Hope this helps a little, and sorry if it doesn't. Given the latter, I'll leave this in the capable hands of the other pros here, though you may have wanted to post in the rooting section of the forum for the Nexus 5 for that.

Cheers!

 

Welcome to the forums!

I'm a little confused by what you are trying to accomplish. Do you have files on the phone that you are trying to access/save? Why won't the device boot?

When the device is booted to recovery or fastboot, it shouldn't matter what the setting of the phone OS are, as that area isnt running. You should still be able to do ADB/Fastboot just fine

 

Edit: Just tried every helpful fastboot command on my device (locked the bootloader) and no luck. If only you had a custom recovery before this bootloop happened then things would've been a lot easier. Since you have no access to adb commands via os or recovery, I'm afraid it's completely impossible to retrieve anything from your n5. To also answer your question on how to enable USB debugging without access to the OS itself, you would need a unlocked bootloader to do that...(my guess would you need to push a already tick usb debug version of the data of system.apk to it's respected place). Even if you had usb debugging on you'd still can't access the commands until the device performs a successful boot (drivers are accessed after that slight little vibration the device emits after or during the boot animation screen).

I'm not so familiar with deep recoveries of deleted files from a flash storage space but maybe it's possible but I'm at a lost on what tool/software you'd use for that.

 

I see now. Unfortunately with a locked bootloader fastboot isnt going to be any help, and the device needs to be booted for ADB to work.

I can't see any way out of this with the data in tact, but maybe someone else will think of a solution.

 

I"m in the exact same boat. I have a replacement Nexus 5 coming so this broken one would become a pet project