Originally Posted by pupkact
The point I was trying to make was that the survey ended on 12/15, the 1st day the GNex was available in the US, so the number of users that could potentially report crashes in ICS was potentially very small prior to that (at least in the US). I am going to over exaggerate to make my point here, and I'm not trying to flame
If there were 1 billion iOS 5 devices in the wild and 0.5% of them had one app error it would still be 5 million app errors reported. If there are 100 Android 4 devices in the wild and 100% reported 10 app errors it would be 1,000 errors. If you put those two statistics in a chart like the one linked below, you could see results like you see between iOS 5 and Android 4.
Yes but the data is normalized so that the total number of apps being launched is not able to skew the results (from Forbes):
But the analysis examined app crashes as a percentage of each app launch, so this data takes out the issue of there being more iOS than Android apps.
The percentages presented would be susceptible to inflation and deflation just based on the sheer amount of reports coming in, and as you point out it would introduce a slight bias for iOS just because of total volume. How they apparently are accounting for this is to create a ratio of launches following a crash to regular launches- we'll call it a "coefficient of failure," and that would make the percentages we are seeing a ratio of a specific coefficient of failure out of all coefficients of failure, not total number of reports. Unless I am missing something about their methodology. In other words, it seems they are taking all data coming in from say iOS 5.0.1, and with each app reporting ((# of launches after a crash) / (# of total launches)) = our coeffecient of failure (across all the volumes using that OS or "normalized"). These ratios are then held as a ratio to themselves (normalized again) yielding the percentages in the chart. That is the only way I can see that they can do this and claim that there will be no volume bias, because percentages derived at the coefficient of failure level would be proportional to the volume of apps used/launched on that OS (as you point out). Which would also mean that the number of cases is normalized, so it does not matter that ICS users had not had a chance to report their crashes. I see your point, but after careful reading the statements made to Forbes I think they have protected against that (or are misleading us about their methodology).
at any rate, I would love to read the actual study just to see how exactly they did normalize their data
I realize that was heavily wordy. They seem to be finding the failure rate within each OS, using some measure of central tendency to describe that OS, then using that MCT value to compare to all others, to find its relative share of the failure. In the process of doing so (specifically at the first or "within the OS" level) they account for over-representation by volume by using that single descriptive statistic.