I was just playing with my MediaMonkey for Android app last night (in my house on the same WiFi router as my PCs) and noticed that when I clicked on the "UPnP" feature, the Windows Media Player (WMP) servers for all of my family's PCs appeared as available media servers-- even though the WMP apps weren't running. (That explains why I see the "WMPlayer.exe" process always running when I look at the active processes under the Windows Task Manager.) MediaMonkey for Windows (MMW) also appears as a media server, but only when MMW is actually running on my PC. I was able to stream media from any of them (though my crappy old phone couldn't display high-bitrate videos). Selecting media from MMW is a LOT easier and faster.
If you just want to use the built-in (WMP) media server in Windows XP/7/8, I'm pretty sure that you have to turn on the media sharing. I don't remember exactly how you do that, but I always do it when I initially set up my PCs. I think you can access it from the Network Settings under "Advance Sharing Properties". There are probably 3 other ways to turn it on. You almost certainly have to point your media server (WMP, MediaMonkey, et al) to your media folders (and, if desired, create playlists).