I think as soon as we get mobile hardware that can match the gameplay experience of dedicated handhelds, mobile will take over a much larger share of the handheld market.
Handhelds are still around for two reasons: 1. The hardware gives a much better gaming experience with analog buttons, thumbsticks, larger screens, etc. 2. The games available, especially for the Nintendo handhelds, are better games, and provide unique gaming experiences that are exclusive to that hardware. These games are often more "traditional" as well.
Currently, mobile offers a much different gaming experience, consisting mainly of casual games with simple touch controls. This is fine for most people, but more serious gamers still want the "core" experience. I think mobile will eventually be able to satisfy those consumers as well as Tegra devices as able to run high end games, and blu-tooth controllers (MOGA, Nyco Playpad, etc.) can provide the desired tactile controls. I think the Xperia Play (psp phone) would have brought this about sooner if it had been more successful.
The real challenge for any gaming specific mobile phones/accessories is getting developer support.