1. Download our Official Android App: Forums for Android!

General Benchmarks and other comparisons/impressions

Discussion in 'Android Devices' started by cliche guevara, May 20, 2011.

  1. cliche guevara

    Thread Starter
    Rank:
    None
    Points:
    16
    Posts:
    56
    Joined:
    Oct 7, 2010

    Oct 7, 2010
    56
    4
    16
    Overall sounds very positive, but the benchmark score of 2245 is substantially below the Galaxy S II which has been around the 3K mark...
     

    Advertisement

  2. daivik

    daivik Android Enthusiast
    Rank:
    None
    Points:
    68
    Posts:
    516
    Joined:
    Apr 19, 2010

    Apr 19, 2010
    516
    44
    68
    That's expected though. (I believe) the Sensation has an A8 (older gen) ARM based Snapdragon processors (which has been heavily modified).

    The GSII has a brand new processor (designed for the GSII and the newer/future devices) based on A9-ARM architecture.

    Then again, Sensation's main selling points are the aesthetic value (such as Sense, outside looks etc.)
     
  3. cliche guevara

    Thread Starter
    Rank:
    None
    Points:
    16
    Posts:
    56
    Joined:
    Oct 7, 2010

    Oct 7, 2010
    56
    4
    16
    ...and the fact that it's not going to break in half if you hold it too tight.
     
    maddada likes this.
  4. daivik

    daivik Android Enthusiast
    Rank:
    None
    Points:
    68
    Posts:
    516
    Joined:
    Apr 19, 2010

    Apr 19, 2010
    516
    44
    68
    Clearly not true though!

    I was just sharing info. I wasn't trying to get into an argument and claim that one was better than the other...
     
  5. cliche guevara

    Thread Starter
    Rank:
    None
    Points:
    16
    Posts:
    56
    Joined:
    Oct 7, 2010

    Oct 7, 2010
    56
    4
    16
    Woah calm down, I'm not trying to start a fan boy debate either, just that I'm pretty clumsy with my phones and I'd be concerned that I'd break the SGSII due to it's thin build, the Sensation looks sturdier and therefore more suited to me.
     
  6. chocobosandwich

    Rank:
    None
    Points:
    53
    Posts:
    356
    Joined:
    Jul 6, 2010

    Jul 6, 2010
    356
    57
    53
    The phone screen looks very reflective, especially when compared to the SGS2 on the same site. They also mention the SGS2 having better outdoor visibility. I wonder if that curve on the edges makes it more reflective.
     
  7. daivik

    daivik Android Enthusiast
    Rank:
    None
    Points:
    68
    Posts:
    516
    Joined:
    Apr 19, 2010

    Apr 19, 2010
    516
    44
    68
    Don't think so. I think the screen is completely flat - it's just a raised bit on the edges.

    And the reflectiveness comes from the fact that's its S-LCD which needs "more glass" which adds glare. The SUper AMOLED Plus uses "less glass" or something and hence there's supposed to be less glare.
     
  8. Shocky

    Shocky On Probation
    Rank:
    None
    Points:
    163
    Posts:
    2,872
    Joined:
    Jun 20, 2010

    Jun 20, 2010
    2,872
    319
    163
    Huge difference in quadrant, 3500 vs 2250, makes a change for a Samsung phone to well in that benchmark, without tweaks at least.

    Quadrant still sucks though.

    Samsung didn't need to increase CPU speed to 1.2GHz, but I'm glad they did. :D
     
    EarlyMon likes this.
  9. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Not sure what you mean by that last part. :thinking:

    FWIW - they went with Sharp as the supplier for the Evo 3D, so maybe this is made by S-LCD or it might be made by Sharp. Will be interesting to see either way.
     
  10. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Another user explained this very well -

     
    Jackietreehorn likes this.
  11. jggonzalez

    jggonzalez Member
    Rank:
    None
    Points:
    16
    Posts:
    33
    Joined:
    Mar 1, 2011

    Mar 1, 2011
    33
    2
    16
    Yup. Thanks for the info. I'm thinking of getting one of these. Its not the fastest out there, but it seems fast enough for me.

    The Quadrant score seems well below the Samsung mark, but how does it compare to a Tegra 2?
     
  12. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Several of us in the Evo 3D forum are confused by that display - why did it finish the benchmark so quickly? It seems counter-intuitive -- if a benchmark is to show performance, it seems it ought get the high score on the one that runs it faster.

    Benchmarks only tell so much, and it seems if you find a benchmark that favors device A you'll find another that favors B - kinda hard to tell how they relate to real-world use sometimes.

    Looking forward to the in-depth review, myself.
     
  13. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    The dual cores have conflicting benchmark scores and this is a confusing issue. The author of the Smartbench suite commented on this entire matter in the Evo 3D forum.

    It is not cut and dry.

    Like last year with the initial crop of superphones, confusion seems to happen first - and then eventually, the full analyses come out and folks get an understanding of what numbers mean what.

    I caution that it's best to show some respect for your fellow posters, there's no cause for calling people names whatsoever, specific or implied.
     
  14. Shocky

    Shocky On Probation
    Rank:
    None
    Points:
    163
    Posts:
    2,872
    Joined:
    Jun 20, 2010

    Jun 20, 2010
    2,872
    319
    163
    Tegra 2 scroes higher, they get around 2500 points.

    Smartbench sucks, Mali 400MP scores less than SGX540 in that benchmark, CPU score is useful, gaming score is not.

    Anyone know what the Sensation scores on GLBenckmark 2.03 or Electopia?
     
  15. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Giving equal time -

    So - if you feel that Smartbench is somehow unfair in its gpu assessment, there's the author, you can PM him and ask for clarification.

    I've not seen a Sensation benchmark from Anandtech - just the dev phone at 1.5 GHz -

    AnandTech - Dual Core Snapdragon GPU Performance Explored - 1.5 GHz MSM8660 and Adreno 220 Benchmarks

    Won't be accurate to the 1.2 GHz Sensation.

    In my opinion - benchmarking is only fair when the results are clearly understood and correlate to how the numbers may match real-world performance.

    Note how the iPhone 3gs beats the iPhone 4 in some benchmarks in that Anandtech link - and how much of that isn't due to the quadrupling of pixels in the latter? Do you believe the iP3gs is superior to the iP4? I don't - and neither do I think that makes the benchmark suck - it simply means that the numbers need some correlation and analysis.

    Without that analysis, benchmarks are like little more than revving the car in the driveway - the tach goes way high, but you're not getting anywhere.

    So - not sure which part of Smartbench sucks in your estimation, but I'd hope you'd factor in expectations of the entire device, not just the raw processor, before deciding. SGS2 has what? 35% more subpixels than SGS? Just saying.
     
  16. Shocky

    Shocky On Probation
    Rank:
    None
    Points:
    163
    Posts:
    2,872
    Joined:
    Jun 20, 2010

    Jun 20, 2010
    2,872
    319
    163
    The gaming tests obviously, when every other benchmark contradicts “smartbench” then something is clearly wrong with the way it's testing gaming performance.

    We know Mali 400MP is faster than SGX540, other benchmarks show this, it's not that complicated to see which part sucks or is unfair.

    Yep, Sensation will likely score less, for comparision Galaxy S II scores 42 fps in the egypt test.
     
  17. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
     
  18. Shocky

    Shocky On Probation
    Rank:
    None
    Points:
    163
    Posts:
    2,872
    Joined:
    Jun 20, 2010

    Jun 20, 2010
    2,872
    319
    163
    I've already e-mailed the developer regarding this, nothing against the guy, he just needs update the benchmark.

    It's ment to be Smartbench 2011, it doesnt seem very smart considering other much more demanding graphics benchmarks like GLBenchmark 2.03 and Electopia clearly show Mali 400MP is allot faster than the SGX540.
     
  19. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Well, let's just wait and see what he says before judging who is smart - no need to get personal. (I couldn't tell if that was personal to him or not - so I'm just covering the bases.)

    Maybe you're right - I don't have a dog in this fight other than what's good for the community of users.

    But - I sure would be cautious about picking benchmarks because they seem right or satify an expectation. Again - does not GLBenchmark 2 show the iP3 out-performing the iP4?

    Not trying to argue - just expressing an honest opinion on my point of view.
     
  20. mrqs

    mrqs Android Expert
    Rank:
    None
    Points:
    128
    Posts:
    1,446
    Joined:
    Sep 28, 2009

    here's another way to look at it

    so the mali 400mp (hereafter called the "m") is faster than the sgx540 ("s") in most benchmarks, but one benchmark produces an opposite result - why is that?

    likely because there are certain operations that the s does faster than the m, and that one benchmark happens to use more of those operations than the other benchmarks - so what does that mean in practice?

    benchmarks are generally designed to measure performance in operations commonly in heavy use in certain kinds of software (be it games or whatever). obviously every benchmark puts different weights on different areas and hence the results vary. this one benchmark appears to put so much weight on the areas the s is very good at, that it produces the opposite result from other benchmarks that the s is faster than the m - so is the benchmark wrong?

    in a word, no. a benchmark can't be wrong (unless we get anal about it), as it merely measures the performance of running a certain collection of operations.

    now whether or not the results of a benchmark correlates with real world applications is another matter, and the question effectively becomes: does the benchmark measure the performance of operations used in real world applications in appropriate ratios?

    again, the answer isn't clear cut - we already have categories of "productivity" and "gaming", but there are as many "categories" as there are applications. some apps/games might utilize exactly those operations where s is performing better than m, some might not - who knows what the ratio is.

    if we look closer at what each benchmark does, we find that they run a collection of different tests, designed to stress different kinds of operations.
    so couldn't we say any single benchmark is a collection of benchmarks?
    so by combining all of the different benchmarks, we'd have a single benchmark that was likely more accurate than any of the individual ones - or was it?

    what is an accurate benchmark?
    you might say, it's the one that tells which chip is fastest, but you'd be wrong, as we've established, different chips are better at different things and worse at others; while different applications utilize different operations at different ratios.

    so the only accurate benchmark would appear to be the one that measures performance in the exact application you're using at any given time.


    *disclaimer: i rambled on a bit longer than i first intended, and that's because i'm slightly drunk - which may also affect the level of wisdom (and grammar) found in this post.
     
    EarlyMon likes this.
  21. TheSuper

    TheSuper Android Enthusiast
    Rank:
    None
    Points:
    68
    Posts:
    313
    Joined:
    Oct 6, 2010

    Oct 6, 2010
    313
    112
    68
    Android newbie lol
    Glasgow, Scotland, UK.
  22. Shocky

    Shocky On Probation
    Rank:
    None
    Points:
    163
    Posts:
    2,872
    Joined:
    Jun 20, 2010

    Jun 20, 2010
    2,872
    319
    163
    I agree mrqs it's certainly possible there’s something in the 2d/3d tests that Mali 400 MP just doesn’t handle well.

    But the visuals are very basic, I might understand if there were some jaw dropping visuals but they look horrible, I don't see how they could stress Mali 400 MP so much without some very sloppy coding, but then that would usually effect all devices running the benchmark.

    Regardless I think it's something the developer should look into.
     
  23. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    I've worked with Samsung R&D, not the Exynos devs, but enough to know there isn't a dull tool in the shed there. I'd venture to say the same for Qualcomm and nVidia as well.

    SoC releases will always be a game of leapfrog, and in my opinion, as they improve, the differences become finer and finer.

    Expecting any one SoC to attain perfection is a path to disappointment. Expecting balanced improvements is more reasonable.
     
    jroc likes this.
  24. Acei

    Acei Lurker
    Rank:
    None
    Points:
    6
    Posts:
    6
    Joined:
    Feb 2, 2011

    My apology for not responding sooner - I was on vacation so I was mostly ignoring any e-mails. :D

    I suppose there are some truth in what you are saying. When I built the GPU tests for Smartbench, I picked sets of OpenGL functions that I would use if I had to build 3D apps/games. OpenGL is a powerful API so there's always more than 1 way to implement same set of scenes/animations. Some will turn out to be more efficient than others. I'm not sure how mine would stand against others. But I did put a lot of effort though - this wasn't done over few weeks.

    One thing I do want to point out though, is that although scenes do look simpler in Smartbench compare to the likes of GLBenchmark's Egypt, etc, it does contain a lot of polygons that are not visible to the naked eyes. Each walls you see are split into many smaller polygons and I've used up the maximum number of light sources Android's version of OpenGL can handle. What I'm trying to say, is that (mainly due to my own lack of graphical skills), I went more brute force way - doesn't look pretty, but I made sure there were enough objects that it will not go close to 60fps on any devices. Does it sound like a techie trying to create a demanding 3D scenes, yes. :D But I can assure you, they are very taxing to the GPUs, more so than it appears. Perhaps APIs I used aren't as efficient on Mali as other API calls but if that was the case, then the apps I build also would not execute any faster. Having said that, once I get my hands on the SGSII, I will look further into it. I am rather intrigued by "MP" part of Mali.

    I have experiences writing codes for server environments, desktop and now phones but I'm no graphics designer, and until my little daughter grows up and start producing nice graphics pieces for the apps I write, I have no choice. :D

    I don't mind taking criticisms though - it is usually the best source to find out best places to improve my apps, so please do continue to point them out.
     
    jroc, mrqs, Shocky and 1 other person like this.
  25. EarlyMon

    EarlyMon The PearlyMon
    Rank:
     #1
    Points:
    5,218
    Posts:
    57,632
    Joined:
    Jun 10, 2010

    Jun 10, 2010
    57,632
    70,440
    5,218
    New Mexico, USA
    Translation - the graphics benchmark in Smartbench is a stress test.

    Yes, devs can optimize apps. I'll bet most of you already own games whose performance is all over the place - because while devs can optimize apps, they'll very often simply use the graphics library functions that look right to them as coders using any number of ideas, and expect the graphics API (application program interface) to handle their needs through hardware.

    Takeaway - if you want a benchmark that stresses things to the max before spitting out numbers, looks like Smartbench will do nicely.

    Yes - this is a turnaround on my position from before - and it isn't: now we can correlate the results to a meaning, and I've always complained about benchmarks putting out uncorrelated numbers.
     

Share This Page

Loading...