Root ROM Rating system in this subforum?

[inssane]

Now that we have a ROM specific subforum, is there a way to rate these ROMs via the board itself? I'll explain...

-Someone posts a new ROM
(Before you say it, I know we have "stars" but hardly accurate when one person rates 5 stars and that's what shows, that's inefficient in the case of rating ROMs.)
-There is a poll, stars, some other method, in the OP's first post, or attached to the thread.
-5-10 people must rate it so that the rating sticks and averages.
-When the rating is up, **Most Important** it shows externally on the subject in the subforum (before you open the thread)
-And possibly a way to sort high to low.

I am not familiar with the ins and outs of vBulletin, so this may be too time consuming or just not possible.

I think it would be nice, so if someone wants to try three diff. devs, they can choose which one has a better rating and so forth.
Let me know if you are in
Nick

 

[bp328i]

Now that we have a ROM specific subforum, is there a way to rate these ROMs via the board itself? I'll explain...

-Someone posts a new ROM
(Before you say it, I know we have "stars" but hardly accurate when one person rates 5 stars and that's what shows, that's inefficient in the case of rating ROMs.)
-There is a poll, stars, some other method, in the OP's first post, or attached to the thread.
-5-10 people must rate it so that the rating sticks and averages.
-When the rating is up, **Most Important** it shows externally on the subject in the subforum (before you open the thread)
-And possibly a way to sort high to low.

I am not familiar with the ins and outs of vBulletin, so this may be too time consuming or just not possible.

I think it would be nice, so if someone wants to try three diff. devs, they can choose which one has a better rating and so forth.
Let me know if you are in
Nick

Oh Oh Oh it's Magic....You Know....

But really, sounds good.

 

[inssane]

Oh Oh Oh it's Magic....You Know....

But really, sounds good.

Son of a B!tch!!!

 

[breadnatty08]

I think there's perhaps a couple problems with this. First, everybody's Droid is different, so ROMs/kernels/themes sometimes don't jive well, or user error causes problems thus clouding their judgement of the ROM. Second, I wouldn't say it's as big an issue, but a lot of folks are very dedicated to certain devs and would rate their favorite higher regardless. Plus, I guess for me, I'm more concerned about pros/cons of ROMs (be them bugs or certain optimizations) as opposed to just a number rating. My 2 cents.

 

[Fadelight]

I think there's perhaps a couple problems with this. First, everybody's Droid is different, so ROMs/kernels/themes sometimes don't jive well, or user error causes problems thus clouding their judgement of the ROM. Second, I wouldn't say it's as big an issue, but a lot of folks are very dedicated to certain devs and would rate their favorite higher regardless. Plus, I guess for me, I'm more concerned about pros/cons of ROMs (be them bugs or certain optimizations) as opposed to just a number rating. My 2 cents.

This. I'll also add one more flaw... I have seen this many times in Market. People who can't be bothered to read up on how to do it will try to install a ROM and screw something up, then rate it poorly as if it were the ROMs fault.

 

[bp328i]

This. I'll also add one more flaw... I have seen this many times in Market. People who can't be bothered to read up on how to do it will try to install a ROM and screw something up, then rate it poorly as if it were the ROMs fault.

Very good point, I see it all the time where someone does not read what an app does then posts "don't work" 1 star.

 

[inssane]

I agree with the above.
My idea is not formulated per se, but at least you see where I am going.
Yeah, I have a PitBull and I put a couple of videos of him on youtube, ONE PERSON didn't like the vid for some odd reason and he registered 15 different names to give me one star on ALL my videos.
So yeah, I know - but it would be nice if the world was perfect and people were honest.

I have been loyal to BB, but JRummy's ROMs have been great for me. I would vote subjectively, but most wouldn't.

 

[UBRocked]

@ inssane - I didn't want to vote that "You are an idiot" but I don't know that what you are asking for can be done ***IN A HELPFUL AND OBJECTIVE MANNER***. Point proven below...

This. I'll also add one more flaw... I have seen this many times in Market. People who can't be bothered to read up on how to do it will try to install a ROM and screw something up, then rate it poorly as if it were the ROMs fault.

Paraphrasing one of the MANY posts I have read with popular and highly stable ROMs...

"Okay, so this ROM is horrible. My apps would not restore by themselves, I was missing all of the google apps, and after I loaded the 1.2 GHz kernel, I noticed the material on my physical keyboard's space bar started peeling away! I lost root and then a popup appeared on the screen that said my phone is now "bricked"...I'm flashing back to DroidMoid 1.0!"

This is an exaggeration but you get the point. Did the poster wipe data, format their partitions, etc? Did they load a kernel that was listed as "compatible" with the specific ROM?

The ROM ratings would result in people arguing over why they gave too low of a rating or too high of a rating. I like your idea...but I just don't think it is feasible for the entire community to give ratings.

Could their be a Droid ROM rating "panel"? You could get a group of experienced users that have flashed countless numbers of ROMs to provide an "expert" rating. It may not be an exact "popular opinion" rating of the ROMs...but it would be more accurate IMO. This would at least give users a basic guide to how good the ROM is without having to interpret everyone's comments. I'm just spitballin here

 

[JasonRK]

Could their be a Droid ROM rating "panel"? You could get a group of experienced users that have flashed countless numbers of ROMs to provide an "expert" rating. It may not be an exact "popular opinion" rating of the ROMs...but it would be more accurate IMO. This would at least give users a basic guide to how good the ROM is without having to interpret everyone's comments. I'm just spitballin here

UBRocked I like this idea, and hope it's you volunteering to be a ROM reviewer. I like to wait until a ROM stands out or there's a big change before I flash, so I usually find the posters who I know switch ROMs a lot and read their comments and/or signatures.

What if each dev got one sticky post, so readers don't have to dig to read the thread about the most current ROM?

 

[UBRocked]

UBRocked I like this idea, and hope it's you volunteering to be a ROM reviewer. I like to wait until a ROM stands out or there's a big change before I flash, so I usually find the posters who I know switch ROMs a lot and read their comments and/or signatures.

What if each dev got one sticky post, so readers don't have to dig to read the thread about the most current ROM?

Well, I would definitely volunteer to be a "reviewer" as I flash as many ROMs as I can. We would need more than just me though. Reviewers would have to demonstrate a good understanding of the "ROMing" process to ensure that a ROM is not given a bad review based on a novice installation mistake (or taking a shortcut like not wiping/formatting the partitions).

I like the idea of the stickies and the normal "rule of thumb" 5 stickies per sub-forum could overlooked but we would need to be fair and not let it get crazy. Having said that, I could see giving a sticky per ROM (not per Dev) because JRummy for example makes an optimized Motorola ROM, optimized Cyanogen ROM, and a Source Build ROM (and I feel all 3 would be unique). Every time a Dev released a new version, we would unstick the previous release and stick the new one.

 

[gnath9]

you guys are on to something ...

Could their be a Droid ROM rating "panel"? You could get a group of experienced users that have flashed countless numbers of ROMs to provide an "expert" rating. It may not be an exact "popular opinion" rating of the ROMs...but it would be more accurate IMO.

Fantastic Idea !!!

 

[inssane]

Now I just want to know who thinks I am an idiot now.
LOL

 

[UBRocked]

Now I just want to know who thinks I am an idiot now.
LOL

Was at 4 the last time I looked...and now it's at 5

Hey...you made the poll where you gave people the option to call you names

As for the ROM reviews...we will have to see. I am working to see if this would be viable and accepted.

If anyone has any ideas for standard information that you would like to see in reviews please post them here. Any other ideas, comments, etc...please post them as well.

IF YOU'RE AGAINST THIS WHOLE IDEA...Please post why. Are there any different ways to implement this that would change your mind?

Anyone interested in being a reviewer...please PM me. Having said that, we would be wanting users that have flashed many ROMs and are proficient and feel they could give an unbiased and objective opinion

Here are some broad examples of the things that reviewers would address in each review.

- Installation (Recoveries supported, market apps syncing, how long did it take to restore apps, length of first boot, random reboots immediately after flashing and prior to first successful boot, issues encountered, etc.)
- ROM speed and responsiveness with stock kernel
- ROM speed and responsiveness with other kernels (user defined kernel... 1.0GHz, 1.25GHz, etc)
- Feature set (custom scripts, full screen browser, etc.)
- Bugs found...solutions found
- Availability of themes
- Theme review if applicable (not too involved...Did everything work as it should have?...fonts, apps themed correctly, any functionality get broken when theme was applied, etc).
- General observations and liking compared to previous releases, other ROMs, etc.

Looks like a lot but you could summarize most ROMs with basic questions like this fairly quickly and there are many users that provided a lot of this information in discussion. The goal would be to get that information (reliable information) organized and located in the 2nd post of the Thread.

The ROM Thread could potentially have 10-15 stickies where each Dev's current ROM(s) would be posted. As an ROM is updated, that sticky would by unstuck and the new one stuck. This way all

 

[GirLuvsDroid]

Oh Oh Oh it's Magic....You Know....

bp, you suck!!!! I can't believe you did that to us again....aaaaaargh!!! I had a bad enough time getting that stupid tune out of my head a couple of days ago, and you go and start it again. Evil, man, just evil!

 

[GirLuvsDroid]

UBR, I think you're really on to something here that would prove to be extremely helpful to a lot of users.

 

[UBRocked]

UBR, I think you're really on to something here that would prove to be extremely helpful to a lot of users.

Thanks. I'm just trying to find the best, most fair, most helpful way of implementing it....all credit to inssane for putting the idea out there.

 

[Papadroid]

I love the idea but I think the implementation is going to very difficult unless you have quite a few reviewers. At the pace that these ROMs are updating/coming out reviewers would be hard pressed to get to each ROM to give a review, especially since it takes a good two days to really get a feel for a ROM.

 

[inssane]

Was at 4 the last time I looked...and now it's at 5

Hey...you made the poll where you gave people the option to call you names

As for the ROM reviews...we will have to see. I am working to see if this would be viable and accepted.

If anyone has any ideas for standard information that you would like to see in reviews please post them here. Any other ideas, comments, etc...please post them as well.

IF YOU'RE AGAINST THIS WHOLE IDEA...Please post why. Are there any different ways to implement this that would change your mind?

Anyone interested in being a reviewer...please PM me. Having said that, we would be wanting users that have flashed many ROMs and are proficient and feel they could give an unbiased and objective opinion

Here are some broad examples of the things that reviewers would address in each review.

- Installation (Recoveries supported, market apps syncing, how long did it take to restore apps, length of first boot, random reboots immediately after flashing and prior to first successful boot, issues encountered, etc.)
- ROM speed and responsiveness with stock kernel
- ROM speed and responsiveness with other kernels (user defined kernel... 1.0GHz, 1.25GHz, etc)
- Feature set (custom scripts, full screen browser, etc.)
- Bugs found...solutions found
- Availability of themes
- Theme review if applicable (not too involved...Did everything work as it should have?...fonts, apps themed correctly, any functionality get broken when theme was applied, etc).
- General observations and liking compared to previous releases, other ROMs, etc.

Looks like a lot but you could summarize most ROMs with basic questions like this fairly quickly and there are many users that provided a lot of this information in discussion. The goal would be to get that information (reliable information) organized and located in the 2nd post of the Thread.

The ROM Thread could potentially have 10-15 stickies where each Dev's current ROM(s) would be posted. As an ROM is updated, that sticky would by unstuck and the new one stuck. This way all

-I think there should be 20 reviewers. It's a round number - easy to quantify in percentages.
-The ROMs should be judged on a CLEAN WIPE - because that is where the least amount of problems exist and it should be done on new ROMs anyhow.
-Review board should probably test a few standard "ROM apps" - SetCPU, TiBu, etc. That way we avoid giving five stars to a ROM in which SetCPU or the Market, ie, don't work well since those are meat and potatoes to many people.
-Battery life, although subject to kernel, should at least be mentioned if not addressed. Just a 24 hour drain down and useage/hours uptime - this is important to me, maybe others too.

Thanks UBR for the work and the credit - and yes...unbiased

*Please tell us why not, constructive criticism is helpful - and yes, I filled in the options *

 

[Papadroid]

-I think there should be 20 reviewers. It's a round number - easy to quantify in percentages.
-The ROMs should be judged on a CLEAN WIPE - because that is where the least amount of problems exist and it should be done on new ROMs anyhow.
-Review board should probably test a few standard "ROM apps" - SetCPU, TiBu, etc. That way we avoid giving five stars to a ROM in which SetCPU or the Market, ie, don't work well since those are meat and potatoes to many people.
-Battery life, although subject to kernel, should at least be mentioned if not addressed. Just a 24 hour drain down and useage/hours uptime - this is important to me, maybe others too.

Thanks UBR for the work and the credit - and yes...unbiased

*Please tell us why not, constructive criticism is helpful - and yes, I filled in the options *

Now we're getting somewhere, because even if only half of that 20 were able to get a review in a timely manner that would be huge IMO.

 

[UBRocked]

Now we're getting somewhere, because even if only half of that 20 were able to get a review in a timely manner that would be huge IMO.

20 reviewers might be a good number to have, but I don't think that 10 or 20 people need to review each ROM for this to be successful (in fact, I think that would be counter productive as most people may not read more than the first couple of reviews to "get the gist". If we could get at least 2 and no more than 5 reviews per ROM, I think that would be sufficient. This way, people would not feel obligated to flash every ROM that comes out in order to provide a review (just not feasible or fair to ask that of anyone).

 

[johnlgalt]

I *soooooooooooooooooooooooo* wanted to vote "you are an idiot..."

But I actually like this idea. Even better, I like the idea of having separate ROM announcements versus discussion threads....

 

[inssane]

We are being productive - Yay!!
Do it John, vote for idiot, LOL

 

[Papadroid]

20 reviewers might be a good number to have, but I don't think that 10 or 20 people need to review each ROM for this to be successful (in fact, I think that would be counter productive as most people may not read more than the first couple of reviews to "get the gist". If we could get at least 2 and no more than 5 reviews per ROM, I think that would be sufficient. This way, people would not feel obligated to flash every ROM that comes out in order to provide a review (just not feasible or fair to ask that of anyone).

I completely agree. That was my initial concern, feeling obligated to flash each and every ROM. It's just too time consuming and would most likely hurt the quality of the reviews. But like you said if somewhere between 2-5 reviews were done on a ROM and they used that ROM for a couple of days we could quite possibly see great results.

 

[ska.t73]

Love this idea... LOVE IT... the only problem I'm seeing is the timing... As has been said with the rate that these ROMs are released I don't know if the reviews would be able to keep up, also it will take at least a day or two of running the ROM to see how it goes. I also think that the reviews should be writen with as vannila a ROM as possible. I know that some might want to see how apps and themes work with it, but just installing the standard needed apps (i.e. SetCPU, Temp Monitor, etc.) would yeild a better overall review of the ROM itself.

As for the themes I think those should be a whole different animal in and of themselves. Again though I love the idea and am open to helping in anyway I can to see that it can get done.

 

[johnlgalt]

The other thing is going to be getting the reviews to be unbiased and objective.

I have seen millions (yes, literally millions - I am 8 months away from 40) of subjective reviews in my time, and I cannot count (anymore) the number of times I have followed someone's advice, only to find out their either 1) a moron, or 2) very differently aligned than I am, or 3) all of the above.

The ideas that UBR provided sound like a good start - but there is one other *critical* factor to consider.

Let's choose my phone, for example. DROID purchased from VZW over the phone (using my then current Samsung Alias, btw) on 6 Nov and arrived on my doorstep 7 Nov. Now, I didn't root until Jan (back injury less than 4 days after getting the phone, wasn't able to do much other than read a ton of Manga and play some games), but I can tell you this much:

The 2 most stable ROMs on my phone since I rooted have been:

1) Ultimate DROID from 7-9
2) Pete's Bugless Beast back before 2.1 was out and now again with FroYo

(note - is it any surprise that these ROMs, the original and the clone, both are the most stable ones? )

Also, the base FroYo releases have been very kind to me in terms of stability.

Other ROMs that I have tried have sucked on my phone for whatever reason. For example, Cyanogen (and I know I am going to be lambasted for this, but I hope some of you will realize that this is my honest *objective* evaluation of ROMs on my phone) never works right. At some point in time apps start FCing (SportsTap is a major bad one here) and at other times my phone goes unresponsive. This happened with numerous 5.x releases and again last night with the 6.0 RC1.

However, I know plenty of people (including Mel Reyes, of Hacking Firefox fame) who are using and loving Cyanogen Mod.

Same holds true for kernels - Taz's old kernels, then ChevyNo1's kernels, and finally, now, jldfg's kernels, all allow me to run at 1200 MHz without issues.

Most others allow me to run only up to 100 MHz....

So, that all being said, how do you plan to keep objectivity in these reviews?

Actually, I suppose the first question should be: Do you plan to keep the reviews objective or subjective?