Benchmarks and other comparisons/impressions

[cliche guevara]

Overall sounds very positive, but the benchmark score of 2245 is substantially below the Galaxy S II which has been around the 3K mark...

 

[daivik]

Overall sounds very positive, but the benchmark score of 2245 is substantially below the Galaxy S II which has been around the 3K mark...

That's expected though. (I believe) the Sensation has an A8 (older gen) ARM based Snapdragon processors (which has been heavily modified).

The GSII has a brand new processor (designed for the GSII and the newer/future devices) based on A9-ARM architecture.

Then again, Sensation's main selling points are the aesthetic value (such as Sense, outside looks etc.)

 

[cliche guevara]

Then again, Sensation's main selling points are the aesthetic value (such as Sense, outside looks etc.)

...and the fact that it's not going to break in half if you hold it too tight.

 

[daivik]

...and the fact that it's not going to break in half if you hold it too tight.

Clearly not true though!

I was just sharing info. I wasn't trying to get into an argument and claim that one was better than the other...

 

[cliche guevara]

Clearly not true though!

I was just sharing info. I wasn't trying to get into an argument and claim that one was better than the other...

Woah calm down, I'm not trying to start a fan boy debate either, just that I'm pretty clumsy with my phones and I'd be concerned that I'd break the SGSII due to it's thin build, the Sensation looks sturdier and therefore more suited to me.

 

[chocobosandwich]

The phone screen looks very reflective, especially when compared to the SGS2 on the same site. They also mention the SGS2 having better outdoor visibility. I wonder if that curve on the edges makes it more reflective.

 

[daivik]

The phone screen looks very reflective, especially when compared to the SGS2 on the same site. They also mention the SGS2 having better outdoor visibility. I wonder if that curve on the edges makes it more reflective.

Don't think so. I think the screen is completely flat - it's just a raised bit on the edges.

And the reflectiveness comes from the fact that's its S-LCD which needs "more glass" which adds glare. The SUper AMOLED Plus uses "less glass" or something and hence there's supposed to be less glare.

 

[TheSuper]

NOT A REVIEW - but it's an UNBOXING!

HTC Sensation unboxing & first-impressions [Video] - SlashGear

very thorough

Nice one! Finally, a tech website has an HTC Sensation in their hands!

Also spotted this on SlashGear too: HTC Sensation vs Samsung Galaxy S II [Video] - SlashGear

They say they will have a full review up in a few days time once they've put the phone through its paces. Can't wait!

 

[Shocky]

Huge difference in quadrant, 3500 vs 2250, makes a change for a Samsung phone to well in that benchmark, without tweaks at least.

Quadrant still sucks though.

Samsung didn't need to increase CPU speed to 1.2GHz, but I'm glad they did.

 

[EarlyMon]

Don't think so. I think the screen is completely flat - it's just a raised bit on the edges.

And the reflectiveness comes from the fact that's its S-LCD which needs "more glass" which adds glare. The SUper AMOLED Plus uses "less glass" or something and hence there's supposed to be less glare.

Not sure what you mean by that last part.

FWIW - they went with Sharp as the supplier for the Evo 3D, so maybe this is made by S-LCD or it might be made by Sharp. Will be interesting to see either way.

 

[EarlyMon]

That's expected though. (I believe) the Sensation has an A8 (older gen) ARM based Snapdragon processors (which has been heavily modified).

The GSII has a brand new processor (designed for the GSII and the newer/future devices) based on A9-ARM architecture.

Then again, Sensation's main selling points are the aesthetic value (such as Sense, outside looks etc.)

Another user explained this very well -

Except the Qualcomm 8660 is not A8. It's Qualcomm's own modified ARM architecture, Scorpion. Now if A9 & Scorpion are both running 1.2GHz, the A9 may still be slightly more powerful, but it's not as clearcut as A8 v A9. And processor power aside, it sounds like S II has GPS issues along with other QC problems. Who knows, the 3D/Sensation may have their own QC issues, but unfortunately we won't know until they are released (hopefully soon).

Per Anandtech: "The third contender in 2011 is Qualcomm

 

[jggonzalez]

Yup. Thanks for the info. I'm thinking of getting one of these. Its not the fastest out there, but it seems fast enough for me.

The Quadrant score seems well below the Samsung mark, but how does it compare to a Tegra 2?

 

[EarlyMon]

Several of us in the Evo 3D forum are confused by that display - why did it finish the benchmark so quickly? It seems counter-intuitive -- if a benchmark is to show performance, it seems it ought get the high score on the one that runs it faster.

Benchmarks only tell so much, and it seems if you find a benchmark that favors device A you'll find another that favors B - kinda hard to tell how they relate to real-world use sometimes.

Looking forward to the in-depth review, myself.

 

[EarlyMon]

The dual cores have conflicting benchmark scores and this is a confusing issue. The author of the Smartbench suite commented on this entire matter in the Evo 3D forum.

It is not cut and dry.

Like last year with the initial crop of superphones, confusion seems to happen first - and then eventually, the full analyses come out and folks get an understanding of what numbers mean what.

I caution that it's best to show some respect for your fellow posters, there's no cause for calling people names whatsoever, specific or implied.

 

[Shocky]

The Quadrant score seems well below the Samsung mark, but how does it compare to a Tegra 2?

Tegra 2 scroes higher, they get around 2500 points.

The dual cores have conflicting benchmark scores and this is a confusing issue. The author of the Smartbench suite commented on this entire matter in the Evo 3D forum.

Smartbench sucks, Mali 400MP scores less than SGX540 in that benchmark, CPU score is useful, gaming score is not.

Anyone know what the Sensation scores on GLBenckmark 2.03 or Electopia?

 

[EarlyMon]

Giving equal time -

I agree that no benchmark will exactly duplicate user experiences but as long as you know how to interpret the results benchmark produces, I believe there is a value (otherwise I would not have produced Smartbench in the first place )

What a co-incidence - I also worked on compiler optimization, in my case, for an earlier version of Sun's SuperSparc architecture which at that time, was mainly focused on multi-core and super-scalar architecture. It involved modifying GNU GCC compiler that produces more optimized binaries for this particular architecture.

To give you a general idea, Smartbench 2011 has 5 independent mini-test suits. Pi, Mandelbrot and String tests focus on stressing CPU, while Tunnel and Jellyfish focuses on stressing GPU. CPU tests were re-written for v2011 so that the workload is split equally between 4 threads, hence it will consume up to 4 cores if they are available.

Pi test is just that - it calculates Pi, using 4 separate threads but combines the results. Very tight loops of integer and floating point calculations are used, so there is a good chance that this test will run entirely within L1 cache.

Mandelbrot obviously plots the famous Mandelbrot graphs but I intentionally divided the screen into 4 squares, so that each area can be updated independently by separate threads. This one uses much larger loop, but still focuses on integer and floating point calculations.

String does series of tests on strings - assignments, comparisons, copying and regular expression parsing. Strings used are quite large - my hope here is to completely fill up L1 cache so that it would test CPU to memory interface more thoroughly.

Tunnel test looks simple but I've intentionally increased the load by splitting each wall into many smaller triangles. This was done to make sure no devices hit near 60fps limit. Tegra 2 processors are only doing around 30fps or so in this test. Uses various OpenGL calls that are common in games of today.

Jellyfish, again uses many semi-transparent cubes with textures on it. From what I've seen, some devices do better on Tunnel test, while others do better on the Jellyfish test.

As you can see, what I want to do here is to create more mini-tests that mimic some of the common tasks/operations that phones need to execute. More mini-tests will be developed over time. You will see some new ones in v2012. This is one of the reason why I used the year as a version number - once more mini-tests are added, I need to create a new baseline for the test results, hence all previously captured test results are no longer comparable to the new ones.

Productivity Index is simply an average of 3 CPU benchmark results, normalized to HTC G2 (1000). Games Index is again, an average of 2 GPU benchmark results, again, normalized to HTC G2 (1000). If you get 2000, that means you completed the test twice as fast as 800MHz G2.

Not sure if this answered any of your questions. If you have any more, I will gladly respond.

My apology to everyone, this post is sorta off the track.

So - if you feel that Smartbench is somehow unfair in its gpu assessment, there's the author, you can PM him and ask for clarification.

I've not seen a Sensation benchmark from Anandtech - just the dev phone at 1.5 GHz -

AnandTech - Dual Core Snapdragon GPU Performance Explored - 1.5 GHz MSM8660 and Adreno 220 Benchmarks

Won't be accurate to the 1.2 GHz Sensation.

In my opinion - benchmarking is only fair when the results are clearly understood and correlate to how the numbers may match real-world performance.

Note how the iPhone 3gs beats the iPhone 4 in some benchmarks in that Anandtech link - and how much of that isn't due to the quadrupling of pixels in the latter? Do you believe the iP3gs is superior to the iP4? I don't - and neither do I think that makes the benchmark suck - it simply means that the numbers need some correlation and analysis.

Without that analysis, benchmarks are like little more than revving the car in the driveway - the tach goes way high, but you're not getting anywhere.

So - not sure which part of Smartbench sucks in your estimation, but I'd hope you'd factor in expectations of the entire device, not just the raw processor, before deciding. SGS2 has what? 35% more subpixels than SGS? Just saying.

 

[Shocky]

So - not sure which part of Smartbench sucks in your estimation, but I'd hope you'd factor in expectations of the entire device, not just the raw processor, before deciding. SGS2 has what? 35% more subpixels than SGS? Just saying.

The gaming tests obviously, when every other benchmark contradicts “smartbench” then something is clearly wrong with the way it's testing gaming performance.

We know Mali 400MP is faster than SGX540, other benchmarks show this, it's not that complicated to see which part sucks or is unfair.

I've not seen a Sensation benchmark from Anandtech - just the dev phone at 1.5 GHz -

AnandTech - Dual Core Snapdragon GPU Performance Explored - 1.5 GHz MSM8660 and Adreno 220 Benchmarks

Won't be accurate to the 1.2 GHz Sensation.

Yep, Sensation will likely score less, for comparision Galaxy S II scores 42 fps in the egypt test.

 

[EarlyMon]

The gaming tests obviously, when every other benchmark contradicts

 

[Shocky]

No software dev is immune from defects and bugs - I think it's more productive to alert Acei and ask him to investigate and fix (or justify/explain his ideas) before just saying it sucks or is unfair.

That's my opinion. I think this would do you, him and the entire Android community a world of good through that service.

I'm not a gamer, and as you know, won't even be able to see an SGS2 for some time, so I'm in no way prepared to take this up with him.

But if I were in your shoes and felt that sure, I'd contact him as described - but that's just me and my opinion, you roll the way that's best for you.

Besides - it's not like he's a loser or not a nice guy - to the contrary, he seems like a way decent cat.

I've already e-mailed the developer regarding this, nothing against the guy, he just needs update the benchmark.

It's ment to be Smartbench 2011, it doesnt seem very smart considering other much more demanding graphics benchmarks like GLBenchmark 2.03 and Electopia clearly show Mali 400MP is allot faster than the SGX540.

 

[EarlyMon]

I've already e-mailed the developer regarding this, nothing against the guy, he just needs update the benchmark.

It's ment to be Smartbench 2011, it doesnt seem very smart considering other much more demanding graphics benchmarks like GLBenchmark 2.03 and Electopia clearly show Mali 400MP is allot faster than the SGX540.

Well, let's just wait and see what he says before judging who is smart - no need to get personal. (I couldn't tell if that was personal to him or not - so I'm just covering the bases.)

Maybe you're right - I don't have a dog in this fight other than what's good for the community of users.

But - I sure would be cautious about picking benchmarks because they seem right or satify an expectation. Again - does not GLBenchmark 2 show the iP3 out-performing the iP4?

Not trying to argue - just expressing an honest opinion on my point of view.

 

[mrqs]

here's another way to look at it

so the mali 400mp (hereafter called the "m") is faster than the sgx540 ("s") in most benchmarks, but one benchmark produces an opposite result - why is that?

likely because there are certain operations that the s does faster than the m, and that one benchmark happens to use more of those operations than the other benchmarks - so what does that mean in practice?

benchmarks are generally designed to measure performance in operations commonly in heavy use in certain kinds of software (be it games or whatever). obviously every benchmark puts different weights on different areas and hence the results vary. this one benchmark appears to put so much weight on the areas the s is very good at, that it produces the opposite result from other benchmarks that the s is faster than the m - so is the benchmark wrong?

in a word, no. a benchmark can't be wrong (unless we get anal about it), as it merely measures the performance of running a certain collection of operations.

now whether or not the results of a benchmark correlates with real world applications is another matter, and the question effectively becomes: does the benchmark measure the performance of operations used in real world applications in appropriate ratios?

again, the answer isn't clear cut - we already have categories of "productivity" and "gaming", but there are as many "categories" as there are applications. some apps/games might utilize exactly those operations where s is performing better than m, some might not - who knows what the ratio is.

if we look closer at what each benchmark does, we find that they run a collection of different tests, designed to stress different kinds of operations.
so couldn't we say any single benchmark is a collection of benchmarks?
so by combining all of the different benchmarks, we'd have a single benchmark that was likely more accurate than any of the individual ones - or was it?

what is an accurate benchmark?
you might say, it's the one that tells which chip is fastest, but you'd be wrong, as we've established, different chips are better at different things and worse at others; while different applications utilize different operations at different ratios.

so the only accurate benchmark would appear to be the one that measures performance in the exact application you're using at any given time.


*disclaimer: i rambled on a bit longer than i first intended, and that's because i'm slightly drunk - which may also affect the level of wisdom (and grammar) found in this post.

 

[Shocky]

I agree mrqs it's certainly possible there’s something in the 2d/3d tests that Mali 400 MP just doesn’t handle well.

But the visuals are very basic, I might understand if there were some jaw dropping visuals but they look horrible, I don't see how they could stress Mali 400 MP so much without some very sloppy coding, but then that would usually effect all devices running the benchmark.

Regardless I think it's something the developer should look into.

 

[EarlyMon]

I've worked with Samsung R&D, not the Exynos devs, but enough to know there isn't a dull tool in the shed there. I'd venture to say the same for Qualcomm and nVidia as well.

SoC releases will always be a game of leapfrog, and in my opinion, as they improve, the differences become finer and finer.

Expecting any one SoC to attain perfection is a path to disappointment. Expecting balanced improvements is more reasonable.

 

[Acei]

I've already e-mailed the developer regarding this, nothing against the guy, he just needs update the benchmark.

It's ment to be Smartbench 2011, it doesnt seem very smart considering other much more demanding graphics benchmarks like GLBenchmark 2.03 and Electopia clearly show Mali 400MP is allot faster than the SGX540.

My apology for not responding sooner - I was on vacation so I was mostly ignoring any e-mails.

I suppose there are some truth in what you are saying. When I built the GPU tests for Smartbench, I picked sets of OpenGL functions that I would use if I had to build 3D apps/games. OpenGL is a powerful API so there's always more than 1 way to implement same set of scenes/animations. Some will turn out to be more efficient than others. I'm not sure how mine would stand against others. But I did put a lot of effort though - this wasn't done over few weeks.

One thing I do want to point out though, is that although scenes do look simpler in Smartbench compare to the likes of GLBenchmark's Egypt, etc, it does contain a lot of polygons that are not visible to the naked eyes. Each walls you see are split into many smaller polygons and I've used up the maximum number of light sources Android's version of OpenGL can handle. What I'm trying to say, is that (mainly due to my own lack of graphical skills), I went more brute force way - doesn't look pretty, but I made sure there were enough objects that it will not go close to 60fps on any devices. Does it sound like a techie trying to create a demanding 3D scenes, yes. But I can assure you, they are very taxing to the GPUs, more so than it appears. Perhaps APIs I used aren't as efficient on Mali as other API calls but if that was the case, then the apps I build also would not execute any faster. Having said that, once I get my hands on the SGSII, I will look further into it. I am rather intrigued by "MP" part of Mali.

I have experiences writing codes for server environments, desktop and now phones but I'm no graphics designer, and until my little daughter grows up and start producing nice graphics pieces for the apps I write, I have no choice.

I don't mind taking criticisms though - it is usually the best source to find out best places to improve my apps, so please do continue to point them out.

 

[EarlyMon]

Translation - the graphics benchmark in Smartbench is a stress test.

Yes, devs can optimize apps. I'll bet most of you already own games whose performance is all over the place - because while devs can optimize apps, they'll very often simply use the graphics library functions that look right to them as coders using any number of ideas, and expect the graphics API (application program interface) to handle their needs through hardware.

Takeaway - if you want a benchmark that stresses things to the max before spitting out numbers, looks like Smartbench will do nicely.

Yes - this is a turnaround on my position from before - and it isn't: now we can correlate the results to a meaning, and I've always complained about benchmarks putting out uncorrelated numbers.