random high iowaitSupport


Last Updated:

  1. beanfield

    beanfield New Member This Topic's Starter

    Joined:
    Oct 26, 2010
    Messages:
    4
    Likes Received:
    0
    I purchased my evo on the Sprint release date and have had random lagging issues since owning it. Most recently, I've been able to run a terminal emulator and/or adb to get access to 'top'. It seems that the situation most often (but not always) occurs when mediaserver is active (playing mp3's or connected to a2dp Bluetooth). Although sometimes it occurs while my phone is off and I notice it when I power the screen on.

    The UI becomes extremely sluggish. When I run top, a few pids are taking up ~2-4% but most are idle. System and user CPU are low but IOW is 60-80%. It stays like this until I reboot the phone or kill off none system processes with quick sys info. I can't seem to pin it to one or a few processes....killing different pids at different times brings IOW down to 0 eventually....which makes me think it's a transient hardware issue or a kernel/memory management issue.

    I'd normally use pidstat or iotop to get more info...but they're not available on Android.
    I tried echoing 1 into block_dump under /proc to get more info on what pids are doing all the i/o but I don't have permissions. Eclair Cache Problem - Android Linux Kernel Development | Google Groups

    Anyone have any ideas for troubleshooting this? It continues to cripple my phone's resources.


    Edit: possible related thread http://androidforums.com/evo-4g-support-troubleshooting/203542-100-cpu-usage-but-doesnt-add-up.html
     

    Advertisement
  2. beanfield

    beanfield New Member This Topic's Starter

    Joined:
    Oct 26, 2010
    Messages:
    4
    Likes Received:
    0
    This issue cropped up 4 times today. The first time was in the car while connected to a2dp bluetooth. 4g is left on all the time. I was streaming music to a separate bluetooth adapter and it started stuttering on it's own. I pulled over and ran top in a terminal...it again showed iow in the 80-90% range. I had quick system info kill all non android core processes and the I/O wait went back to 0%.

    At the gym it also popped up on it's own. I was listening to music with wired earbuds and the music started sputtering. Again, 4g was left on. I had the exact same symptom of iow going in the 80-90% range. This time I simply turned off 4g and within a ~10 seconds the I/O wait went back to 0%. Dunno if this was just a coincidence or really related to 4g and/or the 4g widget.

    Then I started trying to manually trigger it by flipping on and off 4g using the widget. After a handful of tries, I got it to trigger by...ramping up I/O wait just like before. I immediately flipped off 4g and it went away. Dunno if this was just a coincidence or really related to 4g and/or the 4g widget.

    When I finally got home, I tried flipping 4g on and off about 20-30 times. Eventually it happened again. There was a sudden onset of iow in the 80-90% range. I hooked it up to adb and got the following stats.

    Notice the high IOW yet low cpu usage. No individual process really stand out here. The processes can't get cpu time because they're starved waiting on I/O.
    Notice the "b" column shows 1 process in block state...and the second to last column is the high I/O wait again.
    I let it run for about 5 min while poking around the system...so it ran for longer than the other times. I again turned off 4g with the widget and after about 60 sec iow went back down to 0. If it is related to 4g, it's as if whatever triggers this is some kind of loop that sends i/o requests and the sooner it's caught, the sooner it's able to kill the loop. The longer I wait to turn off 4g (if it truly is some process related to 4g that's causing this), the longer it takes for the system to kill the process while it's queued up waiting for the i/o requests to complete.

    Anyway, the stats after it wen back to normal.

     
  3. beanfield

    beanfield New Member This Topic's Starter

    Joined:
    Oct 26, 2010
    Messages:
    4
    Likes Received:
    0
    I think if I had permissions to enable block_dump, I could see exactly what process is causing all the I/O requests. Unfortunately, Sprint/HTC don't give us rights to write to /proc to help diagnose this.

    Does anyone have any advice for writing to this location? It doesn't look like sudo is a part of android. I'd prefer not to root my device, I don't want to void my warranty if I need to work with sprint/htc support. Or can anyone else who's rooted their phone and having this issue do this above command when the problem is occurring?
     
  4. wlmckee

    wlmckee New Member

    Joined:
    Mar 22, 2010
    Messages:
    1
    Likes Received:
    0
    Thanks for your detailed post beanfield. It's been very helpful for me in tracking down a similar issue on my Samsung Captivate. In my case, my IOW is so high that I have no idle cpu cycles available. It's killing performance. I can reboot and the problem will go away temporarily but then return within an hour.

    Unfortunately, I haven't yet been able to identify the root cause of my high IOWAIT stats but I'm at least on the way to doing so. I have rooted my phone and ran the command to show /proc/kmsg. Here's what the majority of the blocks involve:

    <7>[ 1862.645527] pool-2-thread-1(3488): WRITE block 569352 on mmcblk0p2 (8 sectors)

    The mmcblk0p2 device is my /data directory. It seems that the pool-2-thread-* is a Java-related function. Not sure how to track that back to the app calling the function. I can find plenty of threads about identifying high IO in Linux but many of the commands (like iostat) are not available. Suggestions? Advice?

    UPDATE: I removed Fancy Box (weather widget) and another weather widget which helped briefly. Now I'm seeing another process hog the WRITE blocks:

    <7>[19645.214234] IntentService[A(3046): WRITE block 376 on mmcblk0p2 (8 sectors)

    The WRITE block is different for every event so I don't *think* it's a bad sdcard or RAM. I'm not yet familiar with the Android filesystem so don't know what /data is mapping into (whether it's internal RAM or my add-on sdcard).
     

Share This Page

Loading...