The other thing is HTC really spends time optimizing their Sense UI for their qualcomm processors. Live wallpapers like the Desire one seems to run better on Sense and on their Qualcomm CPUs in general.
We all know Motorola basically spent zero effort or close to no effort on making the UI of the Droid/Milestone smooth. They can't even fix basic bugs like the Alarm bug, music bug, etc, and yet you expect a polished OS? A Google Experience phone is an excuse for them to slap on RAW ANDROID without any improvements. Even though Moto Blur is probably bloatware compared to Sense, it would be SOME sort of improvement. Moto really dropped the ball here and didn't give a damn about the Droid phones. In some raw tests on graphics the Droid comes out ahead or really close to the N1, so in games we should be fine.... but HTC has really optimized the Android OS for their devices so that it runs nicely. Just look at N1 vs Desire/Droid Incredible. The Sense devices are fast as hell.
Heck even look at the Eris on 2.1 That thing is so freaking fast... makes you wonder why the stock home for the Droid is so slow..