Well, you could certainly try another music player like WinAmp. There are lots of music players in the market, many of them free.
Maybe you can't uninstall Music - I haven't tried. Just was thinking out loud.
No, there's no specific folder you need to have the files in. For instance, I have some files in /Music/, but all of my podcasts (which Google Music sees) are in /DoggCatcher/EpisodeEnclosures/##/ - clearly not a folder that you would expect Google Music to be looking for. It just scans your entire storage folder for compatible files.