On what are you basing this? I can definitely tell a difference, and it is the movement.
Your analogy takes into account a mouse pointer moving on a fixed screen, not a complete desktop moving across a screen.
Also, as FPS goes up blur is reduced because you are drawing more frames.
The pointer is touch controller, it determines how responsive the screen is to touch. When some one says the screens change quicker, that is not frame per second, but the responsiveness of the touch controller. If you turn the mouse sensitivity down and have someone move a window, it would seem really unresponsive, if you turn the mouse sensitivity up and have someone move the window it would seem really responsive. Responsive has nothing to do with frames per second, but hz.
If you think your phone is more responsive, it does not matter if it is 30 or 60 on a 4 inch screen. Because the number of frames it takes to cover the whole screen is more then the number of seconds it takes to cover the area plus 10.
If it takes one second to cover 4 inch screen, then you only need to have 20 frames or more per second to cover that distance.
You would have to cover a 4 inch screen in less then 1/2 second to make 40 screens per second seem slow.
Can you really cover all 4 inchs of screen in less then 1/2 of a second and have it be noticeable? Are you super human?
So on a 4 inch screen, frames per second can not effect responsiveness in a way that can actually be noticable to the user. It is the increased in hertz of the screen that makes feed back to the controller more responsiveness.
If you increased the controller with out increasing frame rate, you would still see an increase in responsiveness.
Now, except for video games. Normal menus and icons have a frame rate of less then 5 frames per 100 pixels, anything more and it would not be detectable by the human eye as movement at normal viewing distance.
The closer you get to it the more you notice, the futher away you get the less you notice.
So a quick test. Open up a standard word document, make sure the view rate is 100%. Enter 5 periods, that is about 100 pixels. The smaller the screen the smaller the dots.
On a 4 inch screen, with 700 pixels, anything above 35 frames is not displayed by the screen, because you can not draw a frame that is not there.
Since it is unlikely that you would normally cross 4 inches in less then 1 seconds. 30 frames per second would not be noticeable to you when you are moving icons or changing screens.
If you allow more then 5 frames per second, it would press the video card 100% every time you move anything, which is a waste of battery.
If you do not like my answer I am fine with that, but it is the truth, I talked to some people that professionally made operating systems. Everything I am telling you is what they told me.
Which is also why windows xp looks worst then win 7. XP was coded for less then 5 frames per 100 pixels for screens less then 15 inches.
Win 7 was bumped to 10 frames per 100 pixels, for screens smaller then 60 inches.