• After 15+ years, we've made a big change: Android Forums is now Early Bird Club. Learn more here.

Rooting, OTAs, and Bootloops (oh my! :)

scary alien

not really so scary
Mar 5, 2010
22,305
23,781
Indy
Note: this thread specifically discusses bootloops caused as result of installing an OTA on a rooted 5.x+ device. We are NOT discussing how to address general bootloops. Also, the information presented here is not intended for the casual or novice user. If you are uncertain about or if your circumstances do not match what is described below, do NOT attempt any of the operations listed herein.

Okay, as many of you already know, I like to root my devices :).

But, I also like to otherwise stay stock (including my recovery partition), but rooted.

When OTAs (over-the-air updates) come along, they're kind of a pain to install without flashing back to stock...

I had held-out installing the somewhat recent (24 Mb) 5.1.1 update on my Nexus 5 since I really, really didn't want to have to deal with resetting things up. Then, the recent stagefright stuff happened and Google subsequently pushed-out yet another (10 Mb) 5.1.1 update.

Gah! I was now two OTAs behind :p.

So, I decided to buckle-down and figure-out how to have my cake (install the OTAs) and eat it, too (not have to flash by device back to stock, etc.).


Why Are OTAs Problematic When Rooted?

1. If you are rooted, then there's a (very) good chance that you've changed something in your /system partition.

If that is indeed the case and an OTA comes around, then the cross-checks that the OTA does may find/see that your device doesn't match the setup that it expects and will refuse to attempt the install--if you're lucky.

OTAs typically patch the files that are on your device vs. out-and-out replacing them. This makes the OTAs smaller and easier to transmit, store, etc. You don't want it to patch a file that doesn't match what it expects, otherwise you'd be left with a garbled file. Not fun.

2. If you happen to also have a custom recovery (TWRP, ClockworkMod, Philz, etc.) installed, it very likely won't be able to install the OTA. The reason for this is that a non-stock recovery's kernel has different system properties (such as ro.build.fingerprint, etc.) that are typically cross-checked at the beginning of the OTA's updater-script file as part of the "is this the right device and environment" type of check done ahead of the myriad file cross-checks previously mentioned.

It is possible to change the updater-script file to remove those checks and allow the OTA to be installed with your custom recovery, but unless you absolutely know why and what you're doing, that's generally a very bad idea.

3. Finally, if you're running a recent (5.x) version of Android and have used (flashed) one of Chainfire's UPDATE-SuperSU-v#.##.zip to root your device and you do end-up getting an OTA installed, you could potentially end-up with a bootlooping device at the end of the OTA installation process.

If you do happen to get your OTA to install while being rooted (I'm guessing you at least figured-out how to restore the stock install-recovery.sh file so that OTA's verification steps would pass), you may very likely end-up with a bootloop after the OTA has finished installing.

Also, bear in mind that there are other /system files that the rooting process changes, so the install-recovery.sh file might not be the only file that is cross-checked. In fact, you likely already had to restore your stock recovery partition--OTAs often do include an update stock recovery which is both verified/cross-checked and updated/replaced.


Why Would A Bootloop Occur?

Great question! Thanks for asking ;) :).

Note/remember: the first reboot after installing an OTA (or a new ROM or factory image) can take a while (sometimes up to 15 or 20 minutes) since there are a lot of files that need to be prepared and/or re-created (i.e., .preparing your app's runtime files). So, you need to be patient and not just assume that your device is necesarily in a boot loop.

It's been a while since rooting your device simply involved getting the su binary installed in the /system partition (LOL, those were the days :p).

With the introduction of Android 5.x ("L" / Lollipop) enabling SE Linux enforcing, Chainfire had to do some pretty innovative stuff to get a stable root for such a wide array of devices. This includes the touching of seemingly non root-related (stock) files like /system/bin/app_process([32][64]) and /system/bin/install-recovery.sh.

The reason for this is that the daemonsu binary (which lives in /system/xbin and is symlinked to by the app_process([32][64]) executable mentioned above) changes the SE Linux context of the app_process([32][64]) files that it needs to operate properly.

For example, the app_process32 on my Nexus 5 changed:

Code:
before OTA:  u:object_r:system_file:s0  app_process32 -> /system/xbin/daemonsu
after  OTA:  u:object_r:zygote_exec:s0  app_process32 -> /system/xbin/daemonsu

Also, the backup ("_original"- suffixed) versions of the app_process32 and install-recovery.sh files that Chainfire's root saved were also changed:

Code:
before OTA:  u:object_r:zygote_exec:s0  app_process32_original
after  OTA:  u:object_r:system_file:s0  app_process32_original

before OTA:  u:object_r:system_file:s0  install-recovery.sh
after  OTA:  u:object_r:install_recovery_exec:s0 install-recovery.sh

We can see that this was done by the OTA's following expressions:

Code:
  set_metadata("/system/bin/app_process32",   "uid", 0, "gid", 2000, "mode", 0755, "capabilities", 0x0, "selabel", "u:object_r:zygote_exec:s0");

  set_metadata("/system/bin/install-recovery.sh", "uid", 0, "gid",   0, "mode", 0750, "capabilities", 0x0, "selabel", "u:object_r:install_recovery_exec:s0");

Changing the SE Linux contexts that the root components require (and that Chainfire set forth) would keep the daemonsu binary (that is pointed-to by app_process32) from functioning properly--i.e., bootlooping.


You Might Get (Sort-of) Lucky...

If it happens that the app_process([32][64]) file(s) are indeed being replaced by the OTA, then the replacement of those files should not cause you to experience a boot loop.

However, the cross-checks done early-on by the OTA would likely have cross-checked the file(s) (via an apply_patch_check expression) and would have kept the OTA from installing and from you encountering the subsequent boot loop.


Okay, So What's The Solution?

The key to avoiding a bootlooping issue is to get the stock app_process([32][64]) file(s) re-installed on your device.

Thankfully, Chainfire's UPDATE-SuperSU-v#.##.zip (update-binary) script makes a backup of your these key files with an "_original" suffix. For a 32-bit CPU, you'll see /system/bin/app_process32_original (stock) file saved for you. A 64-bit system will correspondingly have a /system/bin/app_process64_original file backup.

However, these backup ("_original"-suffixed) files will not necessarily have the proper permissions, ownership, and SE Linux contexts, so we'll also have to make sure those attributes get restored back to their proper stock settings.


An Example Of What I Did

FYI, you could (should) actually do this BEFORE you attempt to install an OTA on your rooted, 5.x device to avoid a boot loop.

1. First, I took a Nandroid backup from custom recovery. Then I copied it off of my device onto my PC just in case. This gave me the ability to restore back to a good, known working state before I monkeyed with anything on my /system partition. These safety steps/features are NOT to be discounted. If you cannot make a Nandroid backup then it's probably not a good idea to attempt anything similar to what I'll be outlining below.

2. Note: these instructions were done on/for my 32-bit Nexus 5. 64-bit devices will want to use "app_process64" instead of "app_process32" in all of the respective references.

3. Reminder: this process WILL break root. It's not technically unrooting your device since there's a ton of root-related files that are left intact and untouched. You should theoretically be able to re-root after this process without issue.

4. You took a Nandroid backup, right? :)

5. I've done this process from both custom recovery (TWRP) and while booted-up in "normal" Android. I would normally advise doing this while your device is booted into custom recovery since that's a quiesced state.

I've also used this process to fix a bootlooping device (specifically caused/due to installing an OTA while I had a v2.46 root installed) and used it ahead of time to prevent said bootloop.

If you're already experiencing a bootloop and you don't have access to a custom recovery (and adb access to it), then these instructions will NOT be of use to you :(.

6. The outline below shows the Linux shell commands that I used from an adb shell initiated from my Windows PC.

You should also be able to do these from an Android Terminal Emulator app's session on your device. Additionally, I believe you could also use Root Explorer to replicated these functions, but that will not be documented here (since I haven't tested that :p).

The Android Terminal Emulator app (and other terminal emulator apps) should allow you to copy/paste the commands. For Android Terminal Emulator you long-press the screen and select the proper Edit text option (Select text, Copy all, Paste, Send control key, Send fn key), etc.

7. First, boot-up to normal Android, and fire-up an adb shell (since we're rooted, we can do stuff :))

Code:
c:\android-sdk-windows> adb shell
shell@hammerhead:/ $

If you're using the Android Terminal Emulator app, you'll just launch it and you'll already be at a shell prompt.

8. Next, we need a root shell to be able to mess with our /system partition (if you're doing this while booted into custom recovery, you'll already have a "#" root shell prompt):

Code:
shell@hammerhead:/ $ su
root@hammerhead:/ #

9. Now, we need to remount /system in read-write mode, then we'll rename the relevant app_process([32][64]) files:

Code:
root@hammerhead:/ # mount -r -w -o remount -t ext4 /system

Note 1: the above mount command may be different for your device

Note 2: if you're using a custom recovery, you should be able to mount /system in read-write mode from there, too

10. Let's take a cue from Chainfire's update-binary script and use the toolbox utility to help us with some of our commands:

Code:
shell@hammerhead:/ # SYSTEMLIB=/system/lib
root@hammerhead:/ # cat /system/bin/toolbox > /system/toolbox
root@hammerhead:/ # chmod 0755 /system/toolbox
root@hammerhead:/ # LD_LIBRARY_PATH=$SYSTEMLIB /system/toolbox chcon u:object_r:system_file:s0 /system/toolbox

11. Next, let's verify that you do indeed have the /system/bin/app_process32_original (or app_process64_original) backup files that Chainfire's root created for you:

Code:
root@hammerhead:/ # ls -l /system/bin/app_process*_original
-rwxr-xr-x root     shell       13588 2015-02-19 05:16 app_process32_original

If you do not see that you do indeed have an "app_process32_original" (or "app_process64_original") file, then DO NOT PROCEED any further with these instructions.

12. Next, if you list all of the app_process([32][64]) files in your /system/bin folder, you should see something similar to this:

Code:
root@hammerhead:/ # ls -l /system/bin/app_process*
lrwxrwxrwx root     root              1970-01-18 00:43 app_process -> /system/xbin/daemonsu
lrwxrwxrwx root     root              1970-01-18 00:43 app_process32 -> /system/xbin/daemonsu
-rwxr-xr-x root     shell       13588 2015-02-19 05:16 app_process32_original
-rwxr-xr-x root     shell       13588 1970-01-18 00:43 app_process_init

This shows that this device was/is indeed rooted by Chainfire's recent root process (v.2.46 in my case) and the app_process and the app_process32 files are symlinked to the daemonsu binary. There's also the aforementioned app_process32_original file (good!) and the app_process_init file that Chainfire's root creates (notice it's identical to the app_process32_original file).

Again, if you do not see something similar to the above, then DO NOT PROCEED any further with these instructions.

13. Okay, stuff happens for real from here on...again, if you're already experiencing a bootloop and you don't have access to a custom recovery (and adb access to it), then these instructions will NOT be of use to you :(.

14. Remove the old app_process([32][64]) files touched by the rooting process:

Code:
root@hammerhead:/ # rm /system/bin/app_process
root@hammerhead:/ # rm /system/bin/app_process32

15. Next, rename our old (stock) files that Chainfire's root saved for us:

Code:
root@hammerhead:/ # mv /system/bin/app_process32_original /system/bin/app_process32

16. We also have to recreate a symbolic link between app_process and app_process32:

Code:
root@hammerhead:/ # /system/toolbox ln -s /system/bin/app_process32 /system/bin/app_process

17. While we're at it, we need to restore the install-recovery.sh file back to stock, otherwise, any future OTA will likely complain / fail the cross-check.

Note: it's possible (likely) that you've already restored the install-recovery.sh file back to stock (since you've apparently successfully installed an OTA), so this step may be skipped if you have done so. Never mind that you would have also had to have restored your recovery back to stock.

First, we'll list the files that do exist (your output should look very similar):

Code:
root@hammerhead:/ # ls -l /system/bin/install-recovery*
lrwxrwxrwx root     root              1970-01-18 00:43 install-recovery.sh -> /system/etc/install-recovery.sh
-rwxr-x--- root     root          646 2015-02-19 05:16 install-recovery_original.sh

Note: do NOT do the following two commands if you don't still have the install-recovery_original.sh file on your device!

Next, we'll remove the Chainfire root-specific one and rename the backup to the normal one:

Code:
root@hammerhead:/ # rm /system/bin/install-recovery.sh
/system/bin # mv /system/bin/install-recovery_original.sh /system/bin/install-recovery.sh

18. So now, we need to reset the permission, ownership, and SE Linux context settings back to stock for each of these files:

Code:
root@hammerhead:/ # chown  0:2000 /system/bin/app_process32
root@hammerhead:/ # chown -h 0:2000 /system/bin/app_process
root@hammerhead:/ # chown  0:0   /system/bin/install-recovery.sh

root@hammerhead:/ # chmod 755  /system/bin/app_process
root@hammerhead:/ # chmod 755  /system/bin/app_process32
root@hammerhead:/ # chmod 750  /system/bin/install-recovery.sh

root@hammerhead:/ # LD_LIBRARY_PATH=$SYSTEMLIB /system/toolbox chcon u:object_r:system_file:s0  /system/bin/app_process
root@hammerhead:/ # LD_LIBRARY_PATH=$SYSTEMLIB /system/toolbox chcon u:object_r:zygote_exec:s0  /system/bin/app_process32
root@hammerhead:/ # LD_LIBRARY_PATH=$SYSTEMLIB /system/toolbox chcon u:object_r:install_recovery_exec:s0 /system/bin/install-recovery.sh

All of this should now leave you with these three files looking like this:

Code:
root@hammerhead:/ # ls -Z /system/bin/app_process* /system/bin/install-recovery.sh

lrwxr-xr-x root  shell  u:object_r:system_file:s0 app_process -> app_process32
-rwxr-xr-x root  shell  u:object_r:zygote_exec:s0 app_process32
-rwxr-x--- root  root   u:object_r:install_recovery_exec:s0 install-recovery.sh

19. If you intend on re-rooting, it's advised that you remove the /system/bin/app_process_init file, too, that the rooting process creates:

Code:
root@hammerhead:/ # rm -f /system/bin/app_process_init

20. Some final cleanup:

Code:
root@hammerhead:/ # rm -f /system/toolbox

21. At this point, you should be able to reboot your device with the app_process([32][64]) (and install-recovery.sh) files back to their stock state.

From here, you should not have or a get a bootloop that's caused as a result of having a rooted device that took an OTA update.


Some Caveats / Notes / Warnings

1. The above process will break root. You should be able to re-root afterwards, though.

2. If you don't have a way to make a Nandroid backup before making changes similar to the above and the ability to restore a backup if you encounter any problems, then you really shouldn't attempt any of the these changes.

3. If you don't understand or are not comfortable with what's being discussed / addressed here in this thread, then again, you should not attempt any of these changes on your device.

4. Accepting / installing an OTA while rooted is a potentially soft-bricking (bootlooping) operation, so do not attempt this if you are not 100% comfortable with the consequences and/or recovery methods.

5. The information above was based on my experiences with an Android 5.x Nexus 5 device that was rooted using Chainfire's version of his UPDATE-SuperSU-v2.46.zip flashable. Versions of Android prior to this that did not have SE Linux enforcing enabled and didn't use a version of the flashable that used app_process([32][64]) to start-up the daemonsu at boot time will not benefit from these instructions. Additionally, future versions of the UPDATE-SuperSU-#.##.zip will likely change the process/requirements for doing the above, so caveat emptor.

Hope this was helpful to you :).
 
Last edited:
Thanks, CS! :)

I'm guessing this is a very small population of folks that might encounter this, but I've had a couple of folks ask me about related things before and figured I'd research, debug, test, and document it for others.

Bummed that I can't color-code the text in the "[ code ]" boxes...hopefully, folks will not have a problem following the specific shell commands.

Cheers!
 
Upvote 0
By the way, I tried using the "Full unroot" option in SuperSU (Settings -> Full unroot) to remove root and restore the app_process([32][64]) and install-recovery.sh files back to their normal, stock state (using the "_original"-suffixed files as their source).

This worked splendidly for the app_process([32][64]) files and a subsequent reboot did not boot loop and I verified that the file permissions, ownership, and SE Linux contexts were all correct.

This was, however, NOT the case for the install-recovery.sh file :(. I had been hoping that the full unroot option would also have restored (and reset) this file from the install-recovery_original.sh file, but instead all copies of the install-recovery* had been deleted.

This means that if you attempt a subsequent OTA install, you will likely find that the OTA install will fail because the install-recovery.sh file is missing (and you'd have to find the stock version of that file and somehow get that file re-installed in your /system/bin partition without being rooted :( -- only real way to do that would be to boot into custom recovery and install it from that rooted adb shell).
 
Last edited:
Upvote 0
Paste works with some roms with Terminal Emulator and Hackers Keyboard - control, shift, v.

It also seems to work with Shell Terminal Emulator - control, v.

Another commonly accepted system read-write command from root for a number of devices is -

mount -o rw,remount /system

Or using *'s instead of spaces to clarify for readability, and don't try this -

mount*-o*rw,remount*/system

Do NOT try the version with the *'s, ever, or everyone will laugh at you. The *'s are to show where spaces go.
 
Upvote 0
^^^ ;) :) :D, thanks, EM! I'll update the OP to reflect the paste info you kindly provided :).

Yeah, I'm hoping that folks will use the first post as a guide and not necessarily verbatim for every command...I was/am hesitant to post a flashable .zip file for this since there are a couple of scenarios here (i.e., preventing the bootloop and after you've already run into it).

Also, for everyone's benefit, if you examine Chainfire's update-binary script in the UPDATE-SuperSU-v#.##.zip file, you'll see how he sometimes issues several commands to accomplish the intended task (such as remounting /system in read-write mode) since different devices and kernels perform/support said commands in sometimes very different ways.

For example, here's a snippet from the script that shows various commands he uses for remounting /system:

mount /system
mount -o rw,remount /system

mount -o rw,remount /system /system

I'm also able to use "mount -o remount /system" to remount /system in read-write mode on my Nexus 5, so that's yet another version of the syntax for that particular command.

And, LOL, thanks for pointing-out the asterisks usage :p -- I'll update the references above to be a little more explicit, just to help reduce confusion.

Thanks!
 
Upvote 0

BEST TECH IN 2023

We've been tracking upcoming products and ranking the best tech since 2007. Thanks for trusting our opinion: we get rewarded through affiliate links that earn us a commission and we invite you to learn more about us.

Smartphones