I have found a temporary solution to the multiple music/photo problem.
Settings->Applications->All Tab->Media Storage
Clear the data and then restart your phone.
This will restore gallery and your music player with single files again. However, I've found, after a time something happens with my phone and it again starts duplicating media files until I have several copies of the same song/photo.
If anyone knows what is going on with my phone and how to prevent this process in the first place, I'd appreciate any help.