Froyo Quadrant scores, why is Humming bird doing so bad?

[SamsungVibrant]

Honestly, I've seen no need to question the legitimacy of the boosted speeds in Froyo. Yes, having high scores is nice, but why fix what isn't broken? In my experience, Froyo stock is far better than anything on eclair... ever. I was running Eclar + Voodo + OC, and yes, I did get 2000+ on my quadrant score versus the 900 or so on my stock, non-lagfixed Froyo, but Froyo is just as, if not faster than the modified version of eclair I was using.

These are simply numbers, if your phone runs fast, what does the non-reflective "boosted" CPU performance matter? I've seen people freak out, before they even use the phone saying "OH MY GOD I NEED LAGFIX" and can't pinpoint ANY part where there was lag for their uses. I did experience lag, and the modifications fixed it, but I am completely content with how Froyo is running currently. (Cog 2.2 Beta6)

I think the issue is getting the full potential that Froyo offers for CPU gains.

Look the main selling point of 2.2 is the cpu performance increase. Then Flash 10.1

It just doesn't rub me the right way, knowing that for some reason the Hummingbird is unable to use the full potential of Froyo 2.2.
Hummingbird is the fastest cpu on 2.1, but throw in 2.2, and now suddenly the older Snapdragon outperforms our latest and greatest phone. Doesn't that annoy you?

Also it has nothing to do with beta anymore. The Galaxy S Tablets are running official Froyo 2.2, and many sites and youtube videos have shown the CPU scores on the Tablets are seeing no boost on froyo. Again the Galaxy S Tablet is using the same Hummingbird CPU.

Obviously there is some type of issue going on, even Znet commented on the Galaxy S not gaining a huge cpu boost.

Why even upgrade to Froyo if you can't gain the benefits of the main selling point?

It's simple. The Dalvik VM: Just-In-Time (JIT) Compiler is NOT offering Hummingbird the 2-5x faster performance promised.

http://www.youtube.com/watch?v=yAZYSVr2Bhc

Would you buy a Ferrari only to be told "sorry there is a software incompatibility, upgrading to the latest firmware on your cars on board computer will place a limiter on your Ferrari, won't be able to go above 85 mph." You would be pissed right? Same is happening here. The hummingbird has the potential, but it appears the Samsung engineers didn't really design the chip to run anything beyond 2.1. I mean, how the hell do you design a brand new CPU and don't consider the Dalvik VM JIT compiler in 2.2. That means sorry ass old Snapdragons will perform faster on 2.2 than our brand new spanking Galaxy S.

 

[SamsungVibrant]

my cpu scores are almost twice as high as they were on 2.1


cognition 2.2 beta 5, no lag fix

So basically a Nexus One running 2.2, with the old ass janky Snapdragon has better CPU performance than our brand new latest and greatest Hummingbird running 2.2 That is what the results above show.

 

[tylerdurdin]

This is crazy!!!!! The Incredible runs 1150 with froyo, same as or about, as the Droid X. There is no way in hell I can believe that froyo is not going to make the Galaxy crack even a thousand. I am not going to worry though, because our buddy Ryan over at XDA is gonna make a froyo for the galaxy that is going to blow peoples minds. A lag fixed froyo that runs 3000 on the quadrant......Here that Ryan? Please man?........ Please?

 

[CrimsonPride]

people pay wayyy too much attention to quadrant, the scores have been shown to be easily manipulative.

just cause quadrant says something doesnt make it true.
Cyanogen Demonstrates Quadrant’s Flaws &raquo BriefMobile

quadrant can say the captivate runs a 900 but it could still be faster than a n1 running froyo.

 

[Endoran]

Ok, so I thought I'd come in and ask a couple questions; assume that they are true and proceed to make assumptions (haha). First off, from what limited research I've done the Moto droid X does NOT have a snapdragon processor; it was rumored but turned out to be false. Is this correct? The original moto droid does have a snapdragon, correct? Currently droid x, droid, and nexus 1 all are running froyo, correct? Quadrant scores reflects this fact?

If all these are true it would seem that froyo is optimized for only the nexus 1 otherwise the moto droid would be much higher. If those quadrant screen shots are pre-froyo droid then never-mind. What those quadrant scores are telling me is that only the Nexus one has received a dramatic CPU performance boost - as you can see that is the only quadrant result that has changed. Droix X is up so high because its I/O. You can see from the bars that captivate 2.1 actually has a better CPU score (slightly) than Droid X 2.2. Now if some of my assumptions are wrong i.e. those quadrant scores are NOT for 2.2 for any phone except Nexus one then...quadrant needs to get their act together because the droid has had froyo since august and the droid x since last month.

To cap off my post I'll state that I'm in the "benchmarks don't mean shit" category. But this is a curious topic still.

 

[CrimsonPride]

Ok, so I thought I'd come in and ask a couple questions; assume that they are true and proceed to make assumptions (haha). First off, from what limited research I've done the Moto droid X does NOT have a snapdragon processor; it was rumored but turned out to be false. Is this correct? The original moto droid does have a snapdragon, correct? Currently droid x, droid, and nexus 1 all are running froyo, correct? Quadrant scores reflects this fact?

droid x and droid 2 both run TI OMAP processors

 

[s.m.knipe]

droid x and droid 2 both run TI OMAP processors

I'm like 99% certain the original Droid runs a 600mhz TI OMAP too. In fact, I thought (generally) HTC runs snapdragons, Motorola runs TI's, and Samsung runs whatever they can get (in this case their own humming bird).

Also, Quadrant is not a gold standard for comparison, no single app will ever be. Apps have to be written, and are usually based upon a model- in Quadrant's case it is around the snapdragon series and its predecessors since that was what was around. There was a post on XDA about a year ago when the original Droid was all the rage about this, and Quadrant's developer himself said that it has a bias towards Qualcomm's ARM (covering from the G1's clunker, through snapdragon, and on to scorpion) architecture, that the TI's were similar enough for the scores to be relevant but not exact, and that he/she/they were not going to re-write how the app reads proces

​

ses because the whole app would have to be redesigned. If I can find it I will post it through an edit.
Lastly, yes, the JIT compilation will take phone-specific optimization, but again, this may or may not be reflected in benchmarks depending upon their programing and guidelines for scoring. Simply put, the best indicator will be lag (or hopefully lack thereof) when under load, not some voodoo numerical based bragging rights. Take a look at the post above about Cyanogen, and really read many of the posts at XDA (and not just the Captivate ones, for a full background start with the G1 threads, then the MyTouch and/or Legend, then the Droid- possibly on another forum though, they did their own thing for a while, then the N1), they have lots of good information about stuff like this that has already been argued out and explored...

Also, instead of Quadrant, try 0xBenchmark, it lets you store your scores online, gives a really good (although less visual) breakdown, and most importantly allows side-by-side comparisons to your previous runs (really the only thing benchmarks can reliably used for anyways). And I will say this, my stock 2.1 JH7 Captivate blows my 2.2.1 N1 out of the water in terms of mflops and everyday usage...

 

[SamsungVibrant]

This is crazy!!!!! The Incredible runs 1150 with froyo, same as or about, as the Droid X. There is no way in hell I can believe that froyo is not going to make the Galaxy crack even a thousand. I am not going to worry though, because our buddy Ryan over at XDA is gonna make a froyo for the galaxy that is going to blow peoples minds. A lag fixed froyo that runs 3000 on the quadrant......Here that Ryan? Please man?........ Please?

how funny, the video number 2 posted above says ryans lag fix as the title. is this the same ryan you speak of?

 

[Lagunar]

The first time I ran the test I had a bunch of background apps running and the music player was on.

I turned off the extra stuff like fancy widget and batts start etc..
The second time I got 869 and the third time I got 857.

Still pretty bad right? it didn't even hit nine hundred


By the way I love your signature. That's how I am I love animals and I treat them with respect. And I don't judge people I accept people as they are. differences aside Deep down we are all the same.

i trust these guys,

they are saying:

FAQ - Cognition Wiki


Quadrant Standard Linpack Scores Stock Eclair

850-930​

~8 MFLOPS​

Voodoo Eclair

1600-1700​

~8 MFLOPS​

Voodoo + 1.2 GHz Eclair

1700-1800​

~9 MFLOPS​

OCLF Eclair

2000-2300​

~8 MFLOPS​

Stock FroYo

950-1200​

~14 MFLOPS​

OCLF FroYo

???​

~14 MFLOPS​

 

[jflexe99]

To fill in the OCLF Froyo...I only hit about 1800....That was on a fresh install of Cognition Beta 2.2, so not the official Froyo with root and lagfix.

But I am getting solid 14 MFLOPS

 

[SamsungVibrant]

I think International Galaxy S started getting Official Froyo starting today or something. Would be nice to see someone run Quadrant PROFESSIONAL and some other benchmarking apps, to see what the actual CPU score is that shows up only on the pro versin.

 

[pamaguin]

To fill in the OCLF Froyo...I only hit about 1800....That was on a fresh install of Cognition Beta 2.2, so not the official Froyo with root and lagfix.

But I am getting solid 14 MFLOPS

here is my score. COG v2.2 Beta5.5+OCLF

 

[SamsungVibrant]

View attachment 12423 here is my score. COG v2.2 Beta5.5+OCLF

Thats your total score, what are your scores for the individual components. For example, what was the CPU score? You are running the free edition, only the Professional paid edition of Quadrant will break down your score for you.
That is the only real way to see if the CPU is getting improvement or not in the individual cpu score.

 

[jflexe99]

Most people use Free edition. I always have.

My old 2.1 OCLF score was 2200
COG 2.2 beata 2 OCLF is like 1900

....Hopefully official Froyo will push some good numbers. Why haven't their been any testing from Euro? They have it!!!

 

[pamaguin]

Here you go:

 

[Endoran]

Here you go: View attachment 12435

Here we go what? Are those the euro scores from the stock 2.2??

 

[SamsungVibrant]

Here we go what? Are those the euro scores from the stock 2.2??

LOL, those weren't his scores?

 

[mttfrog13]

You are running the free edition, only the Professional paid edition of Quadrant will break down your score for you.
That is the only real way to see if the CPU is getting improvement or not in the individual cpu score.

Exactly, not enough people understand this. As you will see, ALL high scores with galaxy s phones comes from artificially inflated high i/o scores after a lag fix. Even with stock 2.2, the cpu score is only about 1.75x the speed of stock 2.1.

Versus the nexus one who's cpu score x7 when upgraded from 2.1 to 2.2 which is about 3.5x the cpu score of the galaxy s with 2.2

The low i/o scores in stock 2.2 tell me that there is no lag fix in the 2.2 update so the lag problem is hardware related and samsung will never do anything about it.

The low cpu scores in 2.2 tell me that either 1) the cpu test in quadrant is biased towards snapdragon, 2) froyo is biased towards snapdragon, 3) hummingbird processors simply aren't designed to be as fast as snapdragon when it comes to basic cpu tasks but instead hummingbird is only good for 3d.

There is no way in hell I can believe that froyo is not going to make the Galaxy crack even a thousand.

believe it buddy. And any galaxy s that does crack 1000 will do it with a lag fix that falsely inflates the quadrant score. I agree that lag fixes do give a performance increase but I've read that some lag fixes that have quadrant scores of 1000 run just as well as the lag fixes that have scores of 2200+.

 

[tylerdurdin]

Exactly, not enough people understand this. As you will see, ALL high scores with galaxy s phones comes from artificially inflated high i/o scores after a lag fix. Even with stock 2.2, the cpu score is only about 1.75x the speed of stock 2.1.

Versus the nexus one who's cpu score x7 when upgraded from 2.1 to 2.2 which is about 3.5x the cpu score of the galaxy s with 2.2

The low i/o scores in stock 2.2 tell me that there is no lag fix in the 2.2 update so the lag problem is hardware related and samsung will never do anything about it.

The low cpu scores in 2.2 tell me that either 1) the cpu test in quadrant is biased towards snapdragon, 2) froyo is biased towards snapdragon, 3) hummingbird processors simply aren't designed to be as fast as snapdragon when it comes to basic cpu tasks but instead hummingbird is only good for 3d.


believe it buddy. And any galaxy s that does crack 1000 will do it with a lag fix that falsely inflates the quadrant score. I agree that lag fixes do give a performance increase but I've read that some lag fixes that have quadrant scores of 1000 run just as well as the lag fixes that have scores of 2200+.

this may in fact be all true, but that is where quadrant scores do not matter. There is no REAL accuracy, but real world use shows a huge improvement with a lag fix. I have compared my phone to everyone I know with a android, and with the lag fix, app for app, my phone out launches both my brother in-laws Incredible and droid x with no question, to the point of us standing there in disbelief. I would love to see Froyo optimize the CPU but even before I rooted the froyo leak I am running, there was a impressive speed gain over eclair, and was AS fast as the incredible and DroidX with no problem. For a phone that scores so low on quadrant stock it sure can hold its own against the phones with higher scores and in real world show really no disadvantage at all. What kills me is the fact that a simple fix like creating a EXT2 file system can make this phone in real world use fast as hell, and Samsung would not just apply this to their update. Is the lag fix kind of a cheat? Yes, I believe it is. Does it matter? No, My phone is still faster, even on the quadrant, no matter how you slice it. Froyo is disappointing, but I just want the final so I can root it, lag fix it, and stop messing with it.

 

[bdubbs]

Here is what it seems to me in my basic understanding.

There is a single performance test. This test does not give you a higher score with an OS that should give greater performance. Is there another test.

The scores seem to be easily manipulated. What does the lag fix do that would make the hardware 2x as fast?

 

[tylerdurdin]

Well I dont know if this will clarify things but, it is not so much that the test is being manipulated, as it is the test does not follow what the phone is doing after lagfix is applied. The I/O with lag fix is faster because of less disk writes. Flat out, this is genius. but dont take my word for it this is from the developers post:

Q: Why does this lag fix work? Is it slowly destroying my phone?
A: Let's say an application counts from 1 to 10, and writes the value each time to disk.

Stock:
1 -> App tells RFS to write 1 to disk -> RFS writes 1 to disk -> RFS writes journal saying to changed the value on the disk.
2 -> App tells RFS to write 2 to disk -> RFS writes 2 to disk -> RFS writes journal saying to changed the value on the disk.
..
9 -> App tells RFS to write 9 to disk -> RFS writes 9 to disk -> RFS writes journal saying to changed the value on the disk.
10 -> App tells RFS to write 10 to disk -> RFS writes 10 to disk -> RFS writes journal saying it changed the value on the disk.

Total physical disk writes: 20. Speed: SLOW! Wear and tear on disk: HIGH!

Lag Fix:
1 -> App tells EXT2 to write 1 to disk -> EXT2 stores 1 in RAM.
2 -> App tells EXT2 to write 2 to disk -> EXT2 stores 2 in RAM.
..
9 -> App tells EXT2 to write 9 to disk -> EXT2 stores 9 in RAM.
10 -> App tells EXT2 to write 10 to disk -> EXT2 stores 10 in RAM.
..
EXT2 tells RFS to write 10 to disk -> RFS writes 10 to disk -> RFS writes journal saying it changed the value on the disk.

Total physical disk writes: 2. Speed: FAST! Wear and tear on disk: LOW!

This isn't exactly what is happening, but it gives you the general idea.

Q: Can this mod work on other Android devices? Would we see a performance boost on them as well? If not why is it limited to the Galaxy S?
A: SGS has very very good hardware, but it has some parts of it's hardware poorly implemented. The filesystem that samsung chose to use is custom-built using FAT32 as a base, RFS. It has a lot of the problems that FAT32 has, and should have been left back in the 90s, or even the 80s.

One of the big issues with it is how it handles multiple requests - it blocks. It blocks everything. When your mail app wants to read the mail you just tried to view, but your twitter app is busy writing a new tweat it just received, your mail app is forced to wait.

This is bad, but it could be worse! And it is... your twitter app didn't just get one tweat, it got 50 tweaks. It is busy writing the tweats one by one to the filesystem. This would be fine, since all modern filesystems will buffer writes, so instead of writing each one at a time, they will batch them together and write it as a big chunk. Uh oh - RFS does no buffering at all! After each write, it will also write an update to the grafted-on journal system. Guess what happened to your mail you were trying to view while all this happened? It 'lagged' and you got a black screen for half a second, before the mail popped into view.

Luckily the hardware on the device is so good that you usually don't even notice the problem until you have a lot of apps running, all writing their updates when you unlock the phone.

This is mostly speculation based on experiments done on RFS -- RFS is closed source, and we have no idea if the problems are just badly set settings (such as a block size that is too small), coding bugs in the implementation, or if RFS is just really that badly designed.

This fix just grafts a buffer on top of the RFS filesystem, using a very very simple and fast filesystem, EXT2. It fixes most of the issues by writing to RFS as seldom as possible.

So no, this fix won't fix other devices, since they're already running quite close to maximum speed. The SGS at stock is running nowhere near maximum speed, and this lag fix takes it a bit closer. You could probably speed up other devices by tweaking the filesystem settings to give them a big buffer or similar, but it isn't really needed. (I haven't actually tried to put an EXT2 onto any other Android phone, as I don't have any other Android phone, so this is just speculation.)

Q: My phone is fast now!
A: Yeah.

heres the link to the whole post if your interested
[UTIL] One Click Lag Fix 2.0 - xda-developers

 

[viny2cool]

I have voodoo lag fix. But I didnt notice any change in speed or anything. After reading this thread, i used quadrant standard edition and ran the test.

Very disappointed to see the score was 850. But its not convincing enough, as my bar diagram as shown above by some people...shows exactly same for me and galaxy s phones. I guess it just saw that I am using samasung galaxy and gave the results for standard Samsung galaxy phone (based on exact number of 850)

So my question is, based on my finding....all this discussion is worth anything? I mean I get that main point is Froyo and hummingbird...but discussing on the basis of Quadrant score is worth anything?

 

[dell21]

I have voodoo lag fix. But I didnt notice any change in speed or anything. After reading this thread, i used quadrant standard edition and ran the test.

Very disappointed to see the score was 850. But its not convincing enough, as my bar diagram as shown above by some people...shows exactly same for me and galaxy s phones. I guess it just saw that I am using samasung galaxy and gave the results for standard Samsung galaxy phone (based on exact number of 850)

So my question is, based on my finding....all this discussion is worth anything? I mean I get that main point is Froyo and hummingbird...but discussing on the basis of Quadrant score is worth anything?

Please learn to flash ROMs correctly before posting, I flashed cognition 2.2 beta
also, my score was well over 1500. Your phone does not have voodoo on it, please try again.

 

[tylerdurdin]

I have voodoo lag fix. But I didnt notice any change in speed or anything. After reading this thread, i used quadrant standard edition and ran the test.

Very disappointed to see the score was 850. But its not convincing enough, as my bar diagram as shown above by some people...shows exactly same for me and galaxy s phones. I guess it just saw that I am using samasung galaxy and gave the results for standard Samsung galaxy phone (based on exact number of 850)

So my question is, based on my finding....all this discussion is worth anything? I mean I get that main point is Froyo and hummingbird...but discussing on the basis of Quadrant score is worth anything?

You should be getting much higher scores than that. I would feel pretty safe saying that something did not go well while flashing. You can do a quick flash just to see if you like the feel but when I flash I go through a million motions but it has paid off because every rom I have run has been bulletproof and I often do not see the errors and FCs that other people get. I usually:
flash back to stock, factory reset then flash new rom, boot, factory reset, and so on. I also do not do the conventional back up methods though, only due to the fact that my apps I had last installed are always in my downloads on the market so I just take the time to reset up the phone and not reinstall junk I dont need.

Having said all that non topic crap, Pretty much quadrant does not really work on captivate, If it was true every other phone would be kneeling to the King.

 

[tylerdurdin]

Please learn to flash ROMs correctly before posting, I flashed cognition 2.2 beta
also, my score was well over 1500. Your phone does not have voodoo on it, please try again.

Really dude Were you trying to be HELPFUL? ADVICE would be NICE