More Info!
I caught an entirely different audio setup (speakers this time, not the iTrip I reported problems with earlier) causing the same problems. Here's what I found:
First: If I plug the 3.5mm cable in, it sometimes gives me a headphone icon, and sometimes a headset-with-microphone icon in the top notification bar. (One icon looks like a standard pair of cans, the other's the same but with an obvious third thing sticking out towards where the mouth would be.)
I've found if I plug my 3.5mm cable in quickly, it goes to the headphone mode. If I plug it in slowly, it goes to headset-with-mic mode before I've finished plugging it in. (Obviously these are speakers, and so no mic, and so the headphone-without-mic mode is correct.)
a) If it's on headphone (no-mic) mode, it's fine.
b) If it's on headset-with-mic mode, it may or may not ffwd/rewind randomly.
b1) If it's in the headset-with-mic mode and IS fast-forwarding or rewinding randomly, I need to unplug and re-plug back in the 3.5mm cable. Ideally resetting it to the headphone-without-mic mode.
b2) If it's in the headset-with-mic mode and is NOT jumping around randomly, I can easily force it to do so by wiggling the connection. It's not obvious if wiggling the cable or the internal connection is the problem, as both easily seem to trigger it; it may be a small resistance/capacitance change is triggering this.
I can NOT trigger the headset-with-mic mode using my headphones. I've only been able to trigger it with powered speakers and my powered FM transmitter. This may be an important fact.
So my conclusion so far: make sure it's in headphone mode, not the one with the mic, and you'll be fine. Otherwise, you're likely to hit the problem.
@Baltimoretint: I couldn't find that setting -- where is it in the menus? FYI, I have bluetooth turned off, but I still didn't see it when I turned it on.
@yolky: That's close -- there's a different headset vs. headphone mode, and if it could force headphone, that would do it. (It doesn't look like it can.)