• After 15+ years, we've made a big change: Android Forums is now Early Bird Club. Learn more here.

Benchmarking various ROMs

bigronaldo

Newbie
Feb 28, 2010
11
0
Texas
Hey everyone, I'm new to the forums but have been lurking for about a week now. It's ridiculous the amount of information I've got from here. Well, I got my Hero a little over a week ago and have been in love ever since. I ended up putting Fresh 1.1 and then Pancake 0.1 on. It was blazing fast and loved the look. But it got me wondering if there were any actual benchmarks regarding speed and battery life between the stock ROM and all other variations (Fresh, gumbo, pancake, etc).

I couldn't really find anything significant, so I decided to revert back to the stock ROM and unroot and run some benchmarks. I had to download a recovery image, recovery-RA-heroc-v1.5.2, so I can't really validate the results compared to a real fresh install. Also, since I was originally rooted and then "unrooted", I don't know if that leaves anything different than the original. But with that said, this is what I decided to do:

I will then run each benchmark program 3 times to get an average (killing the process once finished). At night, I will charge to 100% and then let it go into sleep mode. 6 hours later, I will check the battery to see how much has drained.

Here are the results of the first run:

==================================================
FIRST RUN: Android 1.5 w/HTC Sense ("clean" install)
==================================================
BenchmarkPI: 12766, 12467, 12259 (12,497.33 average)
CaffineMark: 573, 576, 568 (572.33 average)
Linpack: 2.101, 2.239, 2.362 (2.234 average)
SpeedTest Download (Cell): 1118, 526, 813 (819 kbps average)
SpeedTest Upload (Cell): 547, 187, 309 (347.66 kbps average)
SpeedTest Download (WiFi): 2041, 2616, 2591 (2,416 kbps average)
SpeedTest Upload (Wifi): 698, 704, 694 (698.66 kbps average)
Battery life (no use 6 hrs): 6% loss (100%-94%)
Battery life (intensive use 1 hour): 21% loss (100%-79%)
(GPS on, WiFi on, Bluetooth on, Pandora playing, screen brightness 100% - never turn off, playing RoboDefense lvl 0)

---
Model number: HTC Hero
Firmware version: 1.5
Baseband version: 1.04.01.09.21
Kernal version: 2.6.27-533ce29d htc-kernal@and18-2
Build number: 1.56.651.2 CL85027 release keys
Software version: 1.56.651.2
Hardware version: 0002
==================================================

Just as a note, I downloaded several games, applications, and utilities that would simulate typical daily use before I ran these benchmarks (Handcent SMS, Facebook, Robo Defense, etc.).

If all goes according to plan, I will run these benchmarks as follows:
Monday: benchmark Fresh 1.1
Tuesday: benchmark Gumbo 1.5c
Wednesday: benchmark Pancake 0.1
Thursday: benchmark with JIT Compiler enabled

The last result will be published Friday morning (if all goes well). Let me know now if there is a different benchmark you'd like me to test. I'll root my phone later this evening, and post the results tomorrow morning.
 
So I ended up amending my benchmarking plan by removing the day of benchmarking root access only. The speed and battery life was the exact same as without root (which is to be expected since nothing is really changing that would affect performance).

==================================================
SECOND RUN: Android 1.5 w/HTC Sense w/ Fresh 1.1
==================================================
BenchmarkPI: 12821, 12179, 12173 (12,391 average)
CaffineMark: 568, 578, 579 (575 average)
Linpack: 2.238, 2.252, 2.261 (2.25 average)
SpeedTest Download (Cell): 975, 926, 1107 (1,002.66 kbps average)
SpeedTest Upload (Cell): 509, 223, 347 (359.66 kbps average)
SpeedTest Download (WiFi): 2223, 1917, 2132 (2,090.66 kbps average)
SpeedTest Upload (Wifi): 697, 706, 673 (692 kbps average)
Battery life (no use 6 hrs): 5% loss (100%-95%)
Battery life (intensive use 1 hour): 18% loss (100%-82%)
(GPS on, WiFi on, Bluetooth on, Pandora playing, screen brightness 100% - never turn off, playing RoboDefense lvl 0)

Model number: HTC Hero
Firmware version: 1.5
Baseband version: 1.04.01.09.21
Kernal version: 2.6.27-533ce29d htc-kernal@and18-2 )
Build number: Fresh ROM 1.1
Software version: 1.56.651.2
Hardware version: 0002

---------
Analysis:
---------

Things seemed to operate at the same speed and with slightly better battery use with Fresh ROM 1.1 installed compared to that of the stock ROM. The interface was slicker, and I definitely enjoy the new apps (SpareParts, Wireless Tether, and ES File Explorer), but as far as performance, it benchmarked roughly the same.

Next stop: Gumbo 1.5c FULL
 
Upvote 0
===============================================================
THIRD RUN: Android 1.5 w/HTC Sense w/ Gumbo ROM 1.5c FULL over Fresh 1.1
===============================================================
BenchmarkPI: 12284, 12058, 11782 (12,041.33 average)
CaffineMark: 548, 578, 574 (566.66 average)
Linpack: 2.345, 2.354, 2.36 (2.353 average)
SpeedTest Download (Cell): 963, 1020, 599 (860.66 kbps average)
SpeedTest Upload (Cell): 490, 443, 276 (403 kbps average)
SpeedTest Download (WiFi): (I was having internet issues at home...)
SpeedTest Upload (Wifi): (I was having internet issues at home...)
Battery life (no use 6 hrs): 5% loss (100%-95%)
Battery life (intensive use 1 hour): 20% loss (100%-80%)

Model number: HTC Hero
Firmware version: 1.5
Baseband version: 1.04.01.09.21
Kernal version: 2.6.27GK-0.32 gbhil@stovepipe )
Build number: Gumbo ROM 1.5c
Software version: 1.56.651.2
Hardware version: 0002

---------
Analysis:
---------

Seemed to run a tad bit faster than Fresh 1.1 alone. The screens seemed a tad bit quicker to flip through, but that could just be me looking too hard into it. In the end, it seems like a good way to improve speed slightly without affecting the battery life.
 
Upvote 0
what exactly does the JIT do and how exactly do you install it all the guides are breif ie (pull this backup this push that) anyone have any insight>?

I'm a programmer, but I'm still learning everything as well. Here's what I could find:

Just-in-time compilation - Wikipedia, the free encyclopedia

JIT stands for Just-in-time compilation. Essentially, it compiles the code into native machine code before it executes, thus allowing it to execute more quickly than the standard bytecode, which is a little bit slower. Think of bytecode as a middle man between the programming language (in this case, Java) and the machine code.

A bad analogy could be made that Java is an English speaker, machine code is a Latin speaker, and bytecode is an interpreter between the two. If you could force the English speaker to instantly speak Latin without the interpreter, the conversation will go much faster. JIT does that. It compiles the code into the native language before it runs. Again, horrible analogy, but I tried ;).

After digging around quite a bit about how to install it, it appears that it only works in 1.6 ROMs and higher. So I may have to modify my future benchmarks to reflect 1.6 and higher ROMs if I want to test with JIT enabled. I'm glad you brought that up. :)
 
Upvote 0
Here is what I did to get JIT enabled:


Downloaded dusted-donuts, took the libdvm.so file out of the zip and put it into my SDK tools directory then I booted into recovery. After my phone was in Recovery I typed the following:

Code:
adb shell mount /system
adb push libdvm.so /system/lib
adb shell chmod 644 /system/lib/libdvm.so
adb shell reboot

This worked perfectly for me.!!:D oh and the dusted donuts link is right here www.androidonroids.com/dusted-donuts just thought i would return the favor ronaldo
 
  • Like
Reactions: bigronaldo
Upvote 0
=================================================
FOURTH RUN: Pancake 0.1 - Rosie-espresso themed SenseUI
=================================================
BenchmarkPI: 12037, 11756, 11847 (11,880 average)
CaffineMark: 583, 582, 575 (580 average)
Linpack: 2.22, 2.261, 2.283 (2.254 average)
SpeedTest Download (Cell): (quit testing internet speed...spotty service)
SpeedTest Upload (Cell): (quit testing internet speed...spotty service)
SpeedTest Download (WiFi): (quit testing internet speed...spotty service)
SpeedTest Upload (Wifi): (quit testing internet speed...spotty service)
Battery life (no use 6 hrs): 3% loss (100%-97%)
Battery life (intensive use 1 hour): 16% loss (100%-84%)

---------
Analysis:
---------
It felt a little more smooth transitioning between screens and apps, but it could be placebo effect at this point. The benchmarks are rather odd, because Pancake ran BenchmarkPi faster, but the others slower. Simply because the results are inconclusive, I can't really say that this ROM is faster than the others. My gut reaction is that it is though. Battery life performed much better than the others though, which is a huge plus for me. Overall, this is the best performing ROM I've tested yet (speed/battery).

Next stop: I was thinking of testing DarchDroid w/ JIT enabled :-D
 
Upvote 0
Here is what I did to get JIT enabled:


Downloaded dusted-donuts, took the libdvm.so file out of the zip and put it into my SDK tools directory then I booted into recovery. After my phone was in Recovery I typed the following:

Code:
adb shell mount /system
adb push libdvm.so /system/lib
adb shell chmod 644 /system/lib/libdvm.so
adb shell reboot

This worked perfectly for me.!!:D oh and the dusted donuts link is right here www.androidonroids.com/dusted-donuts just thought i would return the favor ronaldo

Awesome! Thanks a lot man! I'll give it a shot.
 
Upvote 0

BEST TECH IN 2023

We've been tracking upcoming products and ranking the best tech since 2007. Thanks for trusting our opinion: we get rewarded through affiliate links that earn us a commission and we invite you to learn more about us.

Smartphones