I'm with Teufeltexan... battery life is not an issue for me, so I haven't worried about this setting. However, maybe I can shed some light as to what this setting does with another analogy (I seem to be full of them today).
This analogy will require some of us to show our age...
you need to remember the giddy days when dialup Internet access was the only game in town, several years back.
Back then, you (because you didn't want to leave your dialup connected all the time) would have your computer configured to automatically dial and connect when you accessed the Internet, or checked your email. You'd open your browser, your little dialup window would come up, and you'd do whatever you needed to do on the Internet. When you were done, you'd either manually disconnect, or (more likely) walk away from your computer. After a few minutes of inactivity, your computer would disconnect from your dialup service. When you needed to go back online, you'd have to go through the dialup process again.
Nowadays, we all have "always on" connections through cable or DSL or whatever. Your computer doesn't need to connect, because it's connected as soon as you boot it up.
That's kind of what this setting does. Unchecking the box tells your phone to not connect the data unless it actually needs to. A lot of apps (POP or IMAP mail and Internet browsing in particular) will automatically "dialup" or connect when they need to, and then the phone disconnects after they are done. Others may not connect automatically. Push email in particular won't work, because the connection is initiated by the mailserver, not by your phone.
So, the effect is that the phone has to go through the extra connect step when it needs to, which takes several seconds. Push services won't work. And it's debatable whether it increases battery life, because it does take a certain amount of power to make the connection. Supposedly, an idle "always on" connection doesn't draw much power if any.
I'd suspect that those of you who use data very rarely (no push or auto sync services) would see a slight improvement, but those of you whose phones access the data network regularly (for example, I have my phone checking 4 IMAP email accounts every 30 minutes, which means that on average it's checking one of them every 8 minutes or so) probably wouldn't notice a difference, because the power saved by going idle would probably be negated by the power needed to make the connection when needed.
In short... the less the particular apps you have installed actually use the phone's data connection, the more of a difference you'd notice.
Just speculation, but quite educated based on 6+ years in the Windows Mobile world, and I am a network engineer by trade.
