• After 15+ years, we've made a big change: Android Forums is now Early Bird Club. Learn more here.

Inside the Numbers: Quadrant Advanced & what it means for the SF

saps

Extreme Android User
Sep 12, 2010
5,091
504
WA
So I did some extensive testing with Quadrant today. I do now have the advanced version. Where you get the break down of how and what your score is and I've figured out some of the why's.

First, off our score is actually just the average of the 5 respective tests [CPU, Memory, I/O, 2D and 3D]. Now if you have the free version you cannot see your individual test break downs.

So here are 2 of my top scores which I grabbed a ShootMe of:
snap20101017_010103.png
snap20101017_110318.png

When you add the 5 values and at the bottom and divide by 5 you get the score which is spit out. What is very deceiving about this of course is that each category gets equal weight [1/5 total score] yet because of the numbers I/O is so massively over-rated and something like 2D pales in comparison.

Now here is where I had fun with numbers. I did 25, yes 25 runs thru Quadrant at 1200Mhz [Dirrk's LV DO2 kernel, SetCPU on Performance]. I found some interesting things about each of the 5 individual tests.
1. CPU: In 25 runs I had a low score of 905 and a high of 936. That's right my highest overall score gave me my lowest cpu score. 905-936 is only about a 3% range of variance and it gets even smaller. 20 of my 25 cpu scores were between 924-936. I was going to make a nice chart in excel but quite frankly thats too much work this late on a Sunday night so hopefully I have enough reputation credibility that you will just trust my figures. So with a median/average cpu score in the high 920's lets call it there is about a 1-2% variation on cpu score.
2. Memory: In 25 runs only my highest total score was below 1847; odd huh. Here my range was 1847 to 1920. So again, with a median average in the low 1880's the range of variance here is about 2% from the median point.
3. I/O: Here is where and why we see such large variations in total score average. I had a range of 4048 - 6418 and that is enormous I know. Admittedly, that 4048 was when I received a text during I/O phase of the bench test. So let's even throw that one out. I still had a range of 4977-6418. This score jumped so much and I could not explain the rhyme or reason for it I was just as likely to score 5200 as 6200 and all points in between.
4. 2D: Like cpu score this one barely moved. Again we're talking a 1-2% range of variance. 365-374 was my range here. But out of 25 tests 8 times hit 372, 5 times 373 and 5 times 371. So when I say this one barely moved I mean it.
5. 3D: This one again outside of 2 low ones had a very small variance. Of all 25 tests the range was 942 to 1044. But only twice was my 3D core below 1008. So 23 of 25 were between 1008-1044. So again with a median average in the high 1020's 2% variance.

But I didnt stop there. I wanted to see how linear these figures would be affected by clockspeed. So dropped down to 800Mhz and re tested. I only did 5-6 runs at all subsequent speeds not 25.
So for 800Mhz I found:
1.Cpu: range 641-648; avg. 646
2. Mem: range 1751-1855; avg. 1809
3. I/O: range 3727-4268; this one is hard to avg. I had the two tests in 37xx, the other 4 tests were all 4198-4266
4. 2D: range 263-267
5. 3D: range 1432-1567; this moved a bit more than at 1200Mhz

Think I was done...nope:p
400Mhz cpu clockspeed
1. CPU: range 332-346
2. Mem: range 1145-1221
3. I/O: range 1968-2090
4. 2D: range 143-145
5. 3D: range 814-868

And believe it or not I ran the test 5 times as well at 200Mhz:rolleyes:
1. CPU: range 173-183
2. Mem: range 465-660
3. I/O: range 934-970
4. 2D: range 73-75
5. 3D: range 397-435

Observations:
CPU, 2D and 3D really do have a pretty small degree of variance and as such I was almost able to predict what the scores would be at a given clockspeed. The numbers are not precisely linear but its close and there is a distinct pattern. For every 100% increase in clockspeed there is about a 90% increase in cpu score. So start at 200Mhz, avg cpu score 178, double the speed to 400Mhz and you might think 356 would be the average well not quite its 344 [94% increase]. So 400Mhz is 344, so 800Mhz should be 688, again not quite its 646 [88% increase]. 800 to 1200, so from 646 if it were linear we'd be at 969, not quite its 927-ish [again 88%]. The numbers for 2D were virtually the same in terms of percentages as well. 3D numbers were harder to tie in and if someone wants to crunch the numbers above be my guest.
Memory was/is hard to analyze as a 50% drop in clockspeed did not translate anywhere near a 50% drop, at least not at first. In fact, memory score only dropped about 5% or so going from 1200 to 800Mhz. Now dropping from 800Mhz to 400Mhz there was a much more substantial drop not 50% per se but significant but noticeable like 1/3. Now from 400 to 200 then yes it was almost linear; that is it just about dropped in half just like clockspeed.
I/O. Here's whats interesting while the variation of score is far and away the largest, there is a distinct pattern score-wise and it clearly mirrors cpu clockspeed. So let's start at 200Mhz. I/O averages are about 950-960. Double the clockspeed to 400Mhz, does the I/O double as well... yes it does in fact it more than doubles. 400Mhz average is over 2000. So let's double 400Mhz to 800Mhz, does I/O double? Yup pretty much. Like I said above I had a pair of 37xx scores but also had most of the score in the 4200 range. Finally, 800 to 1200 did we see a 50% increase in I/O scores, not quite, but kinda sorta. I did on numerous occasions crack over 6K but equally true the majority of scores were sub 6K and so here there was not the linear gain we saw at lowers speeds.

Conclusions: We clearly see some established and expected patterns with respect to cpu, 2d and 3d. I/O also has a degree of predictablility based on clockspeed. However, what is underexplained is why there is such a massive variance at the same clock speed. All 4 others test vary only about 2-4% whereas I/O can vary by over 20%. The other obvious conclusion is since Quadrant pumps out the average score of the 5 tests as your score and since Quadrant numbers by themselves exceed the other 4 combined, clearly the key is having a good score there. That's why the previous lag fix was such a booster because it sky-rocketed the I/O score even moreso than voodoo does.

What I'm most curious about is what the guys and gals running the 1250lv are scoring. According to my data and following the fairly linear increases in scores. If I was able to complete the benchmark [which I cannot] I would expect numbers like these:
CPU 960-965+/-
Mem 1900+/-
I/O wow only guessing but 6300-6500 ish [way too much variance to accurately predict]
2D 380-385+/-
3D 1060+/-

So if anyone has Quadrant Advanced and the 1250 kernel I'd love test out my theory versus your actual numbers. Because it seems in reality the 1250 kernel is only scoring the same as the 1200 and I'm trying to search for the why. I wonder if the I/O number is crashing on it, maybe 1250 only gives you 6000 because of stability? Don't know but would love input.

Now how does this extended beyond our SF community? I would say this. If we wanna compare our stuff to other devices lets look at cpu and 3d scores specifically. Let's compare with the same version of Android and then I think we get an idea of where we are at.

Dang Im tried now and I wonder if anything I typed above even makes sense. I hope it does because I've been sitting here an hour typing this shiznit.:rolleyes: If it doesn't I'll edit...tomorrow.
 
LOL... I think I understood 37.9437% of your thread. You sure sounded smart saying all that cool technical stuff! You deserve a hot cookie! With milk!

If I understand your prediction, your thinking that at 1200 you may have just about if not have hit the ceiling in reguards to performance and that 1250 really does not have the infrastructure to support any more performance than what you have already realized with the 1200 clock speed?
 
Upvote 0
I think a good look into the Hummingbird architecture as well as the PowerVR used might help answer some of it. The memory and I/O of modern cpu's in desktops along with the north/south bridges all have different clock speeds but are typically multiples of a core clock speed. Perhaps certain parts of the system do not scale linearly but require speeds to be set to specific multiples. Perhaps the next multiple in say cpu clock speed is 1300 in order for the Memory to hit its next "step".
 
Upvote 0
Thanks guys. I do love to benchmark and do stuff like that.

The things I'd love to see for confirmation and validation are:
Someone else using the same kernel as me on a different phone just to confirm mine is not above or below average; Im assuming the numbers should be the same or within my so called range of variance. Im particularly curious about other peoples I/O numbers as Im wondering with all the flashing and re-flashing and enabling and dis-abling of voodoo I've done has negatively affected me at all. It appears my ability to score highly is intact, as I've not seen a higher Quad score even from the 1250kernel. However, my I/O range is so great, like 15% from one test to the next.

RE: the next speed. james you had a good point there. On a semi-related note, I think we got spoiled by Dirrk releasing new kernels every day for awhile there. The fact that we haven't had a new kernel for like a week now has me going thru withdrawls. I do wish he would come with a 1250SV [another 25-50mV from the LV] and a 1300. Its clear from his posting that 1300Mhz will need at least and likely more than 1300mV, so the term LowVoltage might not be applied anymore. Unless he puts out 2, one with Lower Voltage than the other.
The other sad reality seems to be that we are either unlikely to make it much past 1.3Ghz and/or if we do there wont be any tangible performance benefit to the supposed extra clockspeed. I mean Im pretty content just using what I have here now [1200LV] but knowing there is more left in the tank, rubs me the wrong way
 
Upvote 0
Saps,

talking about the OC speed and voltage: As you do these procedures to your hardware do you feel that the overall life of the product may be shorter due to inherant added heat of the faster processing combined and also the added voltage? Like, what if you find 14 months from now that you just let the majic smoke escape out of your phone and your out of the 1 year warranty.... Just a thought.

I'm really considering the voodoo thing you talk so highly about. I do now see lag in one particular instance. When I wake the phone from sleep and tap the SMS icon, often there is this annoying 5-15 second delay before I get a reaction. I would like instant gratification from the request.

Another laggy issue for me is that if I hold the screen to add a desktop shortcut and select shortcuts and then applications it takes 12-15 seconds to open the application menu for me to scroll through and select the program. Annoying!
 
Upvote 0
Saps,

talking about the OC speed and voltage: As you do these procedures to your hardware do you feel that the overall life of the product may be shorter due to inherant added heat of the faster processing combined and also the added voltage? Like, what if you find 14 months from now that you just let the majic smoke escape out of your phone and your out of the 1 year warranty.... Just a thought.

I'm really considering the voodoo thing you talk so highly about. I do now see lag in one particular instance. When I wake the phone from sleep and tap the SMS icon, often there is this annoying 5-15 second delay before I get a reaction. I would like instant gratification from the request.

Another laggy issue for me is that if I hold the screen to add a desktop shortcut and select shortcuts and then applications it takes 12-15 seconds to open the application menu for me to scroll through and select the program. Annoying!
Great question larry.
Part of the reason the OC/UV kernels are so attractive is that they are undervolted. As such we could potentially be prolonging processor lifespan. Most cpus are built and then configured to run well below their threshold point. In our case Samsung gave us a 1Ghzcpu running on 1300. Now they could have given us a 1.2Ghzcpu as nearly everyone can pull this speed off, its just a matter of voltage. Quite frankly if you follow some of the dev's work they were able to get the cpu running as high as 1.6Ghz.
So, from what I know of cpus its actually voltage moreso than clockspeed which is going to affect lifespan. Yes its true heat is a factor and higher clocked cpus run hotter but so do higher volted ones. At the end of the day my 1.2Ghzcpu running on 1250mV is probably going to at last longer and run cooler than the 1Ghz@1300mV stock. Even if Dirrk or whoever came out with a 1.3Ghz@1300mV kernel or better yet a 1.4Ghz@1350-1400mV I'd still choose to run that. From my experience OCing cpus they live for years and years even OC'd and OV'd [overvolted]. So at the risk of tempting fate I expect this SF to last years and years, even though Im probably upgrading in a year to a 4G/LTE phone, with a Dual Core Tegra2, 1280x720 display, 10MP camera with optical zoom, HDMi out, 128GB mircosdxc card and 2500mah all in a form factor commensurate with the SF...sorry for the daydream :D

To your other point about opening the app draw, yes this can be an issue. Also the more apps you got the longer it takes to open that option. It does get a bit better with voodoo installed but yeah I got 150+ apps installed not to mention the stuff on there already [that I didnt un-bloat] and so to bring up that entire list with icons is going take a few seconds. The other notoriously slow one is Settings>Applications>Manahe Applications. Because again there is just so much in there
 
Upvote 0
So at the risk of tempting fate I expect this SF to last years and years, even though Im probably upgrading in a year to a 4G/LTE phone, with a Dual Core Tegra2, 1280x720 display, 10MP camera with optical zoom, HDMi out, 128GB mircosdxc card and 2500mah all in a form factor commensurate with the SF...sorry for the daydream :D

yeah that sounds like one great device! mine would be quite similar
 
Upvote 0
I don't understand why people put stock in these quadrant scores. Over the last month I've owned an Incredible, Droid X, Droid 2, and now currently own a Fascinate. The Fascinate scores lower in quadrant, but wipes the floor with these other phones when it comes to actual performance. Framerates in games, and live wallpaper are better, and games have better framerates too on it. The browsing experience as far as pinch to zoom, panning around the full webpages and scrolling full webpages is second to none on Android.
 
Upvote 0
I don't understand why people put stock in these quadrant scores. Over the last month I've owned an Incredible, Droid X, Droid 2, and now currently own a Fascinate. The Fascinate scores lower in quadrant, but wipes the floor with these other phones when it comes to actual performance. Framerates in games, and live wallpaper are better, and games have better framerates too on it. The browsing experience as far as pinch to zoom, panning around the full webpages and scrolling full webpages is second to none on Android.
And that was really my point with dissecting this benchmark via each of its 5 individual tests. For instance, most people believe the SF has the best gpu on the market now. So, FWIW lets compare the 3D score of the SF to the DInc, the DX, D2, G2 whatever. Overall score is dependent on the I/O but some of the individual tests I believe can be quite valuable for comparing different devices against each other. Or at least as valuable as any existing other benchmarks out there.
 
Upvote 0
i realize that quadrant can be easily manipulated and isnt the end all/be all when it comes to phone performance but its a benchmark that is useful in comparing set ups and peformance (relatively speaking). I think comparisons between phones ie the fascinate and droid 2 aren't THAT accurate, but its a good comparison in comparing set ups on your own phone.

scores seems kinda in the right ballpark at least though. a fascinate that doesnt have voodoo and scores under 900 generally has some lag to it, and a fascinate with voodoo is a LOT smoother.

at the end of the day some people will hate quadrant and put no stock in it (linpack too) and others like to use it...its your choice
 
Upvote 0
I don't understand why people put stock in these quadrant scores. Over the last month I've owned an Incredible, Droid X, Droid 2, and now currently own a Fascinate. The Fascinate scores lower in quadrant, but wipes the floor with these other phones when it comes to actual performance. Framerates in games, and live wallpaper are better, and games have better framerates too on it. The browsing experience as far as pinch to zoom, panning around the full webpages and scrolling full webpages is second to none on Android.
the browsing comments are quite accurate......on 2.1. however, over at xda, they are complaining that on froyo that the browsing is NOT as smooth anymore. i've also noticed this on my Dx, definitely not as smooth on froyo. changing the plug ins to 'on demand' helps somewhat.....could steve jobs have been right about flash?!?!?!?! :eek:
 
Upvote 0

thats pretty impressive, although the headache of having "temporary" root would make me stay away from the g2 if i was on tmob until that was cracked (if possible).

still like has been posted many times before, quadrant is all about bragging; if i had a g2 that was scoring that id be posting it everywhere possible! haha.

It ll be interesting to see what a froyo-ed, voodoo-ed, oc-ed galaxy s will pull. 2500, maybe higher?

The single best score I ever got on my incredible was on miui oc-ed to 1.19 and it was 1938. I enjoy cracking 2000 on my 2.1 fascinate.

now that the droid x on froyo can overclock up to 1.4 apparently (although no custom kernels) its interesting that they havent touched 2000.
 
Upvote 0
the browsing comments are quite accurate......on 2.1. however, over at xda, they are complaining that on froyo that the browsing is NOT as smooth anymore. i've also noticed this on my Dx, definitely not as smooth on froyo. changing the plug ins to 'on demand' helps somewhat.....could steve jobs have been right about flash?!?!?!?! :eek:


A mentally ******ed deaf mute could come to the same conclusion. Flash on even the more advanced smartphones still is mediocre.
 
Upvote 0

BEST TECH IN 2023

We've been tracking upcoming products and ranking the best tech since 2007. Thanks for trusting our opinion: we get rewarded through affiliate links that earn us a commission and we invite you to learn more about us.

Smartphones