Benchmarking various ROMs


Last Updated:

  1. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    Hey everyone, I'm new to the forums but have been lurking for about a week now. It's ridiculous the amount of information I've got from here. Well, I got my Hero a little over a week ago and have been in love ever since. I ended up putting Fresh 1.1 and then Pancake 0.1 on. It was blazing fast and loved the look. But it got me wondering if there were any actual benchmarks regarding speed and battery life between the stock ROM and all other variations (Fresh, gumbo, pancake, etc).

    I couldn't really find anything significant, so I decided to revert back to the stock ROM and unroot and run some benchmarks. I had to download a recovery image, recovery-RA-heroc-v1.5.2, so I can't really validate the results compared to a real fresh install. Also, since I was originally rooted and then "unrooted", I don't know if that leaves anything different than the original. But with that said, this is what I decided to do:

    I will then run each benchmark program 3 times to get an average (killing the process once finished). At night, I will charge to 100% and then let it go into sleep mode. 6 hours later, I will check the battery to see how much has drained.

    Here are the results of the first run:

    ==================================================
    FIRST RUN: Android 1.5 w/HTC Sense ("clean" install)
    ==================================================
    BenchmarkPI: 12766, 12467, 12259 (12,497.33 average)
    CaffineMark: 573, 576, 568 (572.33 average)
    Linpack: 2.101, 2.239, 2.362 (2.234 average)
    SpeedTest Download (Cell): 1118, 526, 813 (819 kbps average)
    SpeedTest Upload (Cell): 547, 187, 309 (347.66 kbps average)
    SpeedTest Download (WiFi): 2041, 2616, 2591 (2,416 kbps average)
    SpeedTest Upload (Wifi): 698, 704, 694 (698.66 kbps average)
    Battery life (no use 6 hrs): 6% loss (100%-94%)
    Battery life (intensive use 1 hour): 21% loss (100%-79%)
    (GPS on, WiFi on, Bluetooth on, Pandora playing, screen brightness 100% - never turn off, playing RoboDefense lvl 0)

    ---
    Model number: HTC Hero
    Firmware version: 1.5
    Baseband version: 1.04.01.09.21
    Kernal version: 2.6.27-533ce29d htc-kernal@and18-2
    Build number: 1.56.651.2 CL85027 release keys
    Software version: 1.56.651.2
    Hardware version: 0002
    ==================================================

    Just as a note, I downloaded several games, applications, and utilities that would simulate typical daily use before I ran these benchmarks (Handcent SMS, Facebook, Robo Defense, etc.).

    If all goes according to plan, I will run these benchmarks as follows:
    Monday: benchmark Fresh 1.1
    Tuesday: benchmark Gumbo 1.5c
    Wednesday: benchmark Pancake 0.1
    Thursday: benchmark with JIT Compiler enabled

    The last result will be published Friday morning (if all goes well). Let me know now if there is a different benchmark you'd like me to test. I'll root my phone later this evening, and post the results tomorrow morning.
     

    Advertisement
  2. mysticmead

    mysticmead Well-Known Member

    Joined:
    Nov 29, 2009
    Messages:
    1,348
    Likes Received:
    175
    this needs to be moved to the Rooted section.
     
  3. dan330

    dan330 Well-Known Member

    Joined:
    Jan 22, 2010
    Messages:
    11,596
    Likes Received:
    3,221
    awesome..
    thanks!
    :D:)
     
  4. Dreamliner

    Dreamliner Well-Known Member

    Joined:
    Nov 5, 2009
    Messages:
    691
    Likes Received:
    86
    a w e s o m e . . .
     
  5. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    So I ended up amending my benchmarking plan by removing the day of benchmarking root access only. The speed and battery life was the exact same as without root (which is to be expected since nothing is really changing that would affect performance).

    ==================================================
    SECOND RUN: Android 1.5 w/HTC Sense w/ Fresh 1.1
    ==================================================
    BenchmarkPI: 12821, 12179, 12173 (12,391 average)
    CaffineMark: 568, 578, 579 (575 average)
    Linpack: 2.238, 2.252, 2.261 (2.25 average)
    SpeedTest Download (Cell): 975, 926, 1107 (1,002.66 kbps average)
    SpeedTest Upload (Cell): 509, 223, 347 (359.66 kbps average)
    SpeedTest Download (WiFi): 2223, 1917, 2132 (2,090.66 kbps average)
    SpeedTest Upload (Wifi): 697, 706, 673 (692 kbps average)
    Battery life (no use 6 hrs): 5% loss (100%-95%)
    Battery life (intensive use 1 hour): 18% loss (100%-82%)
    (GPS on, WiFi on, Bluetooth on, Pandora playing, screen brightness 100% - never turn off, playing RoboDefense lvl 0)

    Model number: HTC Hero
    Firmware version: 1.5
    Baseband version: 1.04.01.09.21
    Kernal version: 2.6.27-533ce29d htc-kernal@and18-2 )
    Build number: Fresh ROM 1.1
    Software version: 1.56.651.2
    Hardware version: 0002

    ---------
    Analysis:
    ---------

    Things seemed to operate at the same speed and with slightly better battery use with Fresh ROM 1.1 installed compared to that of the stock ROM. The interface was slicker, and I definitely enjoy the new apps (SpareParts, Wireless Tether, and ES File Explorer), but as far as performance, it benchmarked roughly the same.

    Next stop: Gumbo 1.5c FULL
     
  6. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    ===============================================================
    THIRD RUN: Android 1.5 w/HTC Sense w/ Gumbo ROM 1.5c FULL over Fresh 1.1
    ===============================================================
    BenchmarkPI: 12284, 12058, 11782 (12,041.33 average)
    CaffineMark: 548, 578, 574 (566.66 average)
    Linpack: 2.345, 2.354, 2.36 (2.353 average)
    SpeedTest Download (Cell): 963, 1020, 599 (860.66 kbps average)
    SpeedTest Upload (Cell): 490, 443, 276 (403 kbps average)
    SpeedTest Download (WiFi): (I was having internet issues at home...)
    SpeedTest Upload (Wifi): (I was having internet issues at home...)
    Battery life (no use 6 hrs): 5% loss (100%-95%)
    Battery life (intensive use 1 hour): 20% loss (100%-80%)

    Model number: HTC Hero
    Firmware version: 1.5
    Baseband version: 1.04.01.09.21
    Kernal version: 2.6.27GK-0.32 gbhil@stovepipe )
    Build number: Gumbo ROM 1.5c
    Software version: 1.56.651.2
    Hardware version: 0002

    ---------
    Analysis:
    ---------

    Seemed to run a tad bit faster than Fresh 1.1 alone. The screens seemed a tad bit quicker to flip through, but that could just be me looking too hard into it. In the end, it seems like a good way to improve speed slightly without affecting the battery life.
     
  7. aauussttiinn88

    aauussttiinn88 Well-Known Member

    Joined:
    Jan 31, 2010
    Messages:
    82
    Likes Received:
    4
    what exactly does the JIT do and how exactly do you install it all the guides are breif ie (pull this backup this push that) anyone have any insight>?
     
  8. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    I'm a programmer, but I'm still learning everything as well. Here's what I could find:

    Just-in-time compilation - Wikipedia, the free encyclopedia

    JIT stands for Just-in-time compilation. Essentially, it compiles the code into native machine code before it executes, thus allowing it to execute more quickly than the standard bytecode, which is a little bit slower. Think of bytecode as a middle man between the programming language (in this case, Java) and the machine code.

    A bad analogy could be made that Java is an English speaker, machine code is a Latin speaker, and bytecode is an interpreter between the two. If you could force the English speaker to instantly speak Latin without the interpreter, the conversation will go much faster. JIT does that. It compiles the code into the native language before it runs. Again, horrible analogy, but I tried ;).

    After digging around quite a bit about how to install it, it appears that it only works in 1.6 ROMs and higher. So I may have to modify my future benchmarks to reflect 1.6 and higher ROMs if I want to test with JIT enabled. I'm glad you brought that up. :)
     
  9. aauussttiinn88

    aauussttiinn88 Well-Known Member

    Joined:
    Jan 31, 2010
    Messages:
    82
    Likes Received:
    4
    Appreciate that makes a lot of sense tho...did you come across any good how to's the only one I found was on xda for a G1 would it be the same procedure?
     
  10. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    I plan on installing it tomorrow, so I'll let you know how I did it if I can get it to work.
     
  11. aauussttiinn88

    aauussttiinn88 Well-Known Member

    Joined:
    Jan 31, 2010
    Messages:
    82
    Likes Received:
    4
    Here is what I did to get JIT enabled:


    Downloaded dusted-donuts, took the libdvm.so file out of the zip and put it into my SDK tools directory then I booted into recovery. After my phone was in Recovery I typed the following:

    Code:
    adb shell mount /system
    adb push libdvm.so /system/lib
    adb shell chmod 644 /system/lib/libdvm.so
    adb shell reboot

    This worked perfectly for me.!!:D oh and the dusted donuts link is right here www.androidonroids.com/dusted-donuts just thought i would return the favor ronaldo
     
    bigronaldo likes this.
  12. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    =================================================
    FOURTH RUN: Pancake 0.1 - Rosie-espresso themed SenseUI
    =================================================
    BenchmarkPI: 12037, 11756, 11847 (11,880 average)
    CaffineMark: 583, 582, 575 (580 average)
    Linpack: 2.22, 2.261, 2.283 (2.254 average)
    SpeedTest Download (Cell): (quit testing internet speed...spotty service)
    SpeedTest Upload (Cell): (quit testing internet speed...spotty service)
    SpeedTest Download (WiFi): (quit testing internet speed...spotty service)
    SpeedTest Upload (Wifi): (quit testing internet speed...spotty service)
    Battery life (no use 6 hrs): 3% loss (100%-97%)
    Battery life (intensive use 1 hour): 16% loss (100%-84%)

    ---------
    Analysis:
    ---------
    It felt a little more smooth transitioning between screens and apps, but it could be placebo effect at this point. The benchmarks are rather odd, because Pancake ran BenchmarkPi faster, but the others slower. Simply because the results are inconclusive, I can't really say that this ROM is faster than the others. My gut reaction is that it is though. Battery life performed much better than the others though, which is a huge plus for me. Overall, this is the best performing ROM I've tested yet (speed/battery).

    Next stop: I was thinking of testing DarchDroid w/ JIT enabled :-D
     
  13. bigronaldo

    bigronaldo Member This Topic's Starter

    Joined:
    Feb 28, 2010
    Messages:
    11
    Likes Received:
    0
    Awesome! Thanks a lot man! I'll give it a shot.
     

Share This Page

Loading...