The OOB install is a very generic install, and not all that bad, but also really not all that much better than, say, XP on the same machine.
The beauty of Linux is that you have direct access to the kernel. You can recompile a new kernel all on your own, and instead of having a kernel that polls all the hardware that you ahve nad then tries ot match a corresponding module so that hte hardware will be usable, you can manually specify which ones to laod so that the boot takes much less time. After that, the performance will also be determined upon how your partitions are set up, recompiling for your specific processor and hardware (which makes a huge difference in access times and whatnot) as well as the actual hardware used - you can use the most modern processor, with gobs of RAM, but you throw in a couple of 5400 rpm IDE drives you'll hit a bottleneck faster than you can say uncle. SATA III is the new gold standard, while SATA II is the current de facto standard, but even then HDs have different levels of internal cache and differnt spindle speeds. Each of these is going to affect performance. Faster drives get access to data faster. more cache means that more data can be loaded from the drive and then shunted over to RAM / CPU at any given time. Your motherboard's BUS, the CPU and RAM interface type, all sorts of things will factor in to how 'fast' or slow an app in any given OS will be.
For all we know, it could be that the MP3 encoding used on your files are problematic for whatever reason, and so your computer is working double time to try to 1) decode, and 2) fix any issues, before 3) playing the decoded audio. if this were to be the case it would manifest in all OS, not just XP.
However, transmitting data wirelessly is adding anywhere from 3 to 5 more points of failure, so when transferring data to a new SDCard, you're better off not going that route unless it is the only way possible (IOW, you do not have an SDCard reader in your computer).