Anand's iPhone 4S comparison...MISLEADING?

[sleewok]

Okay, so anandtech.com had an article comparing the new iPhone to the current top offerings. 2 of the tests were with the browser.

AnandTech - iPhone 4S Preliminary Benchmarks: ~800MHz A5, Slightly Slower GPU than iPad 2, Still Very Fast

I decided to run the benchmarks to see how my device compares. The results shocked me.

I have an HTC Inspire 4g, OCed to 1.5ghz (OnDemand) (I can run stable at 1.7ghz) with CM7 nightly (220), and TalonAce 1.0.6 Kernel.

The top numbers Anand has for Sunspider shows 2130 for the SG Tab.

In Sunspider I get 2300(@1.5ghz) or 2176(@1.7ghz). This has me confused...what's going on? How can the performance of my device beat the SGS2, and be on par with the I4S?

I then decided to run the Rightware Browsermark. I got 77718 (@1.5ghz) and 87179 (@1.7ghz). That's over 20k higher than all the other phones...

So part of the point of this thread is to point out how misleading these benchmarks are, and another part is to figure out if this is just the result of an amazing group of devs that know how to optimize a device to perform better than next generation devices... (Make sense?)

I think it would be great to see how other rooted/customized devices compare to the benchmarks Anand ran. I typically trust articles on Anand, but this has me wondering...

 

[Shocky]

You should not compare an overclocked phone to a stock phone or even to a phone with a custom rom, it's not a fair comparison either.

 

[mikedt]

Oh so we can overclock a phone and get better benchmark numbers. Is this important in the real world? Like does it improve how well it makes phone calls?

 

[B2L]

Oh so we can overclock a phone and get better benchmark numbers. Is this important in the real world? Like does it improve how well it makes phone calls?

Not really, the only difference it really makes is when playing games, or doing something that needs a lot more power. I typically will overclock my Nexus S to 1.44 ghz when I'm running benchmarks. Haven't tried sunspider, but normally OC'd I can get 5100+ on Quadrant.

 

[sleewok]

You should not compare an overclocked phone to a stock phone or even to a phone with a custom rom, it's not a fair comparison either.

Maybe so (this was an article for stock phones), but at the same time my results show what you CAN get for your Android phone. That changes the game for a lot of people. You can't boost the performance of an iPhone. You're stuck with what you got the day you bought it.

It's a similar concept to buying an unlocked CPU that you end up overclocking. Overclocking of CPUs is commonly used in reviews and comparison. I don't see why that shouldn't be the case with phone performance. It's something that many people may want to take into consideration.

 

[Shocky]

It's a similar concept to buying an unlocked CPU that you end up overclocking. Overclocking of CPUs is commonly used in reviews and comparison. I don't see why that shouldn't be the case with phone performance. It's something that many people may want to take into consideration.

Not really, it would be like comparing results with an overclocked and stock CPU and also with different OS. Your results are only comparable to users running the same custom rom.

sleewok, you know exactly why your phone is scoring higher in these benchmarks, there is no mystery here and I'm not going to get into a pointless discussion over the benefits of overclocking and using custom roms.

Your scoring higher because your using a custom rom and you've overclocked, you know this so why are you shocked by the results?

 

[SebastianDongl]

I don't think it is misleading anyhow. These benchmarks have been very reliable and accurate in case of any types of new devices. The latest devices just happen to get higher results due to fantastic work by these brands.

I agree - you definitely should not compare an overclocked phone to a stock phone - if you want to get a fair comparison.

-Seb

 

[southzeztpdot]

Sunspider benchmark is dumb benchmark. You should be concerned with the other benchmarks

 

[Remeniz]

I gotta say those bench results are very impressive...

The SGS2 trails in most of them.

EDIT: Sorry... all of them.