Go Back   Android Forums > Android Phones > LG Optimus F6

Like Tree102Likes

test: Reply
LinkBack Thread Tools
Old January 17th, 2014, 03:37 AM   #1 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default Proposal/Theory for External SD storage solution on F6

In know the audience will range from the inexperienced to experts, and while this discussion is intended to include all, be warned:

Nothing here is a downloadable solution for novices or casual users, but is a discussion for those of us who know how to unbrick our phones when we experiment with dangerous ideas, understand the Linux command line, the development tools (specifically adb), and know how to partition an SD card through ADB from the linux command line served from the phone without some downloaded app.

If that's not you, you may still find this discussion interesting (and hopeful regarding the storage problem), but only try any of this if you know what you're doing.

Read this like the notes from an engineer's desk after running some experiments.

What I'm posting here is a working theory tested in my F6 (and genymotion) for a grand solution to the typical storage problem. We're all running out of room when there's an empty 32 GByte SD card installed. We all know that the various "link" and swap solutions aren't really doing the trick. You may have even gotten your phone's storage page to claim there's 29Gbytes of free space, but realize after trying to use it that it's not entirely true.

Well...this may do the trick, but it's not without a compromise, but one chosen to be outside the area of compromising storage.

If we could edit init.rc we could simply alter the boot sequence so the main directory of interest, /data, would no longer mount from mmcblk0p15 (maybe it's 14, I'm not looking at it right now)..but from some other source, like the external SD card (typically mmcblk1p1). mmcblk0 is the base node of the 4 Gbyte internal SD card, and mmcblk0p15 is a 1.2 Gbyte partition on that card, holding the content of the /data directory.

Substituting mmcblk0p15 for mmcblk1p1 (or p2, if the external card has two partitions) would provide a genuine primary source from the SD card for /data, providing THE solution to the storage problem.

However, the F6's boot loader is locked, so every time the phone boots, init.rc, much of the root directory structure, and anything we might custom configure is restored to "default" conditions.

If not for that lock, this could be as simple as doing something like this just one time:

cp -rp /data /storage/ext2filesys/data

which duplicates /data and all it's contents (pretty much all the applications and their data), including permissions and links.

and then during boot:

mount -o rw,bind /storage/ext2filesys/data /data

Assuming that ext2filesys represents a partition from the external SD card formatted in ext2 filesystem. It would be more like:

mount -o rw -t ext2 /dev/block/mmcblk1p2

If we were making a custom ROM and installation on the external card for this sort of thing, but we're not. We're using the bind option to mount a directory over another directory instead, but it's similar.

....and, of course, a few other machinations in detail.

However, the fact that we can't edit init.rc (or it's cousins) without cracking the boot loader lock means that we can't perform this mount before zygote is launched. Zygote is name of the process running the Dalvik runtime, it's basically what everyone see's as Android, running as a process inside Linux.

Most here are familiar with smanager and/or universal init.d, or something similar. They run scripts "during boot"....but, they run them AFTER zygote is running, because they're Android applications, not Linux scripts. At that point zygote (another name for Android) is already dependent on the content of /data, and in particular some of it's major and minor subdirectories. Google's playstore is key among them, because it's a wimpy, whiny app that will repeatedly complain once /data is swapped out underneath it, making the phone all but unusable.

So....the basic novel idea I'm proposing is to kill the zygote.

At this point the astute readers probably have an expression of utter disgust.

It's a FUGLY proposition, but one that happens to work so far as I can see.

The idea is to view Android in two parts. The first part is Linux. It boots up and operates like a stripped down but otherwise standard Linux build. Once it's going, which happens rather quickly, it launches zygote. That's when you see the boot logo.

At that point it's already too late to REALLY substitute storage, unless you're willing to view zygote (and thus the Android PORTION of this two piece structure) as individually rebootable - by killing zygote.

Init, the grandparent process of the entire Linux implementation here, will restart zygote, probably within 1ms. When it does, it happens AFTER the substitution mount command example above, making /data serve from the external SD card.

You can try this without script editing yourself, if you're a bold experimenter. Simply arrange for a duplicate of /data on an ext2 filesystem, mount it (driving Google's apps nuts), then ps | grep zygote, note the pid and then kill zygote. It LOOKS like the phone reboots, but it's only the zygote rebooting (and thus Android) - NOT the entire Linux operating system underneath Android.

And zygote will boot as if for the first time, after having reconfigured Linux the way we want.

So far I'm observing zero complications and 100% satisfaction on the storage problem with this approach.

No app by app fiddling with links, or move to phone - just space. Actual, real space.

If you're familiar with fdisk, mkfs.ext2, mkfs.vfat (for a fat32 partition of you want one), the mount command and such, you can probably put the pieces together yourself.

Now, making a script that can grep the zygote process from the output of ps, then sed and/or awk that to get the pid to fashion a kill command is an ugly looking bit of code which causes me to reach back 30+ years into my early Unix work, which I've happily let slip from memory, though Google provides.

Frankly I'm so much more comfortable with the NDK, and the stand alone toolchains that I'm motivated to write a C/C++ command line utility to do it instead, but maybe I'll try my hand at the script for a bit.

The basic plan is this:

Upon power up, Android (as zygote) initializes from the default, internal version of /data as mounted from mmcblk0p15 - the stock version of /data, after rooting, equipped with smanager or universal init.d to initiate a script in /system/etc/init.d.

That goes through machinations to mount the ext2 partition (they don't automatically mount easily on the F6) - then, calls another script on the ext2 partition to continue. I suggest this so that simply pulling the SD card boots a "standard" configuration based on the stock version of /data.

The second script, probably launched as a new thread (using the & at the end of the command, for example), would perform the substitution for /data served out of mmcblk1p1 or p2, then kill zygote to reboot it.

An interesting consequence is that, from the Android perspective, you have two operating systems. One on the internal card at mmcblk0p15, and the other on the external card at mmcblk1p2 (or p1 if you choose a single ext2 partition).

This means that all applications you install after such a modification will end up in the external partition. If you pull the card and reboot, with the design plan mentioned above, the "stock" or internal version of /data remains...without those applications installed.

Or any photos you've taken, or downloads...etc.

This essentially abandons that admittedly puny 1.2Gbyte partition that came with phone, relegating it to a boot and self destroy function, simply to launch the version from the external card.

Until someone cracks the boot loader lock (before the phone is outdated and replaced, that is), this may be the only way to gain REAL open storage on the external SD card.

Anyone up for a ride?
aaanadie, irha73, bidzer and 1 others like this.

Advertisements
JVene is offline  
Reply With Quote
The Following 8 Users Say Thank You to JVene For This Useful Post:
aaanadie (January 27th, 2014), infowarz (January 30th, 2014), jr223 (February 19th, 2014), katone (August 15th, 2014), Lordvincent 90 (February 22nd, 2014), morgankslade (January 20th, 2014), o4tuna (April 14th, 2014), WarrantyVoider (August 23rd, 2014)
sponsored links
Old January 17th, 2014, 03:41 PM   #2 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default On second thought....

On second thought....

That really is a FUGLY proposed theory.

And, it does have pitfalls - under certain conditions rebooting zygote that way doesn't quite do so well.

I just found another way I'm going to tinker with today.

I didn't realize that the entire /system subdirectory survives reboots.

That's key.

Here's why.

As I pointed out above, what we don't have, and would prefer, is a way to run script code before zygote launches. Changes to init.rc are out.

But, taking a cue from busybox and toolbox, and newly recognizing that changes made to content in /system/bin survives reboots, I realize now that at a key point in init.rc, they call mount.

mount is in /system/bin, with another link in /system/xbin (when busybox is installed).

They way toolbox and busybox work is that they read the command line and perform functions as aliases to real commands, like this:

busybox chmod a+rw filename

The real Linux command is

chmod a+rw filename

Now, for toolbox, chmod is a link to toolbox, and in /system/xbin, when busybox is installed, chmod is a link to busybox.

We may type

chmod 777 filename

But chmod links to toolbox, which calls toolbox instead of some chmod code.

On standard Linux, chmod is a c program. On Android implementations it's not. It's a link to toolbox, which then reads the command which launched it. In the arguments it receives it can tell the command line reached the toolbox executable via chmod.

This is opposed, to say, the same thing happening to reach toolbox (the exact same executable) via mount, or some other command.

Here's what I've been looking for.

I can substitute the mount command. I can change the link from toolbox to, say, customtoolbox. I intend to make customtoolbox.

It will be a simple c program, compiled with the standalone toolchain under NDK9b, probably clang3.3, targeting ARM and x86 for Android's implementation of Linux.

This is the way something like busybox is written.

The objective will be, in default behavior, to forward the command to toolbox so operating is exactly as before.

However, I should be able to determine, within customtoolbox, if the OS is booting. I've not yet worked out the details on that point, dozens come to mind immediately and I'll have to search for the best fit.

The point is that when init.rc calls mount, it will be before zygote launches. Factually, it may be before much of anything is mounted - it should end up calling mount very early on in the boot process (and several times subsequently).

Any command could be intercepted in this way....cp, ls, mount....just about any of them.

At the time customtoolbox is about to forward a command to toolbox, it can check to see if this is the first occasion that's happened since boot, and to subsequently check to see if sufficient storage resources have already been mount - such that customtoolbox can call scripts in, say, /system/etc/custominit, in a style similar to /system/etc/init.d, but long before zygote is involved.

This would be a great moment to look for /data, unmount it (if required), and subsequently mound a new version of content for /data from the external SD card.

I'm going to work on it later tonight.

Anyone else interested in experimenting too?
Trevor Philips likes this.
JVene is offline  
Reply With Quote
The Following 4 Users Say Thank You to JVene For This Useful Post:
Lordvincent 90 (February 22nd, 2014), morgankslade (January 20th, 2014), o4tuna (April 14th, 2014), Trevor Philips (January 17th, 2014)
Old January 17th, 2014, 09:21 PM   #3 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Big shout out to Trevor Philips , and thanks.

I've had a Eureka moment on the theme above.

/system/bin can indeed retain modifications made to content, and there's an even better approach to substitution that just hit me like a ton of bricks.

The app_process binary.

When init.rc executes, there comes a point when the service zygote is to be started. The code from init.rc on the F6 is:


service zygote /system/bin/app_process -Xzygote /system/bin --zygote --start-system-server
class main
socket zygote stream 660 root system
onrestart write /sys/android_power/request_state wake
onrestart write /sys/power/state on
onrestart restart media
onrestart restart netd
onrestart restart surfaceflinger

What's key here is /system/bin/app_process.

The source code for app_process (like all the source of Android) is online.

I can rebuild the app_process binary with a slightly modified custom version, which will survive reboots.

It is ideal given the circumstances (of a locked bootloader).

I'm triple checking I have the same version of source as that of the 4.1.2 on the F6.

The code is extremely simple. app_process is used to launch various android applications, and intercepting here is perfect:

Code:
    if (zygote) {
        runtime.start("com.android.internal.os.ZygoteInit",
                startSystemServer ? "start-system-server" : "");
    } else if (className) {
        // Remainder of args get passed to startup class main()
        runtime.mClassName = className;
        runtime.mArgC = argc - i;
        runtime.mArgV = argv + i;
        runtime.start("com.android.internal.os.RuntimeInit",
                application ? "application" : "tool");
    } else {
        fprintf(stderr, "Error: no class name or --zygote supplied.\n");
        app_usage();
        LOG_ALWAYS_FATAL("app_process: no class name or --zygote supplied.");
        return 10;
    }

Where the "if (zygote)" code initializes the main instance of the zygote engine, and the "else if (className)" block launches an app in an existing zygote process.

This moment is an excellent opportunity to synchronize things. At the zygote initialization I can block while running scripts, certain that zygote is not yet launched - then let it continue as if nothing was interrupted.

I'm going to prepare for editing / rebuilding this on the F6 tonight.
irha73 likes this.
JVene is offline  
Reply With Quote
The Following 4 Users Say Thank You to JVene For This Useful Post:
Lordvincent 90 (February 22nd, 2014), morgankslade (January 20th, 2014), o4tuna (April 14th, 2014), WarrantyVoider (August 23rd, 2014)
Old January 17th, 2014, 11:30 PM   #4 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Update:

The download of the source for Android OS 4.1.2 is a little over 8 Gbytes, which will take a while. That means my experiment will likely not commence until late Saturday. I could attempt just to compile the app_process binary by itself, but it is dependent on OS shared libraries targeted at 4.1.2, and I'd prefer to install the build into an AVD and experiment with it as a proxy for the F6, so I'm loading the whole thing. Building it will take some time, too.

In the meantime, while studying the potential hinted in my previous post, I have some observations to add.

This can do more than merely remount /data on the external SD card. It can also customize the OS considerably.

As you probably realize, when Linux boots the stock software is restored, destroying any interesting customizations one might hope for in a custom ROM. Until there's a sure way of unlocking the boot loader, customization could still be realizable if we accept a few inconveniences in the mix.

First, this is still going to be 4.1.2. That won't change. This technique can't easily substitute a new kernel. However, 4.1.2 could be considerably customized, simply because app_process will have control in Linux before zygote is launched, and will block the initiation of zygote until all adjustments are completed.

This means, for example, there could be a custom boot logo sitting on the SD card, waiting to be copied into position. The original is obviously going to be in position at first during boot (every time), but once it's placed, just prior to zygote's initiation, the new app_process could substitute a replacement. The zygote process is what shows the boot logo. You see nothing on the display until zygote shows it to you during a normal boot.

It should be possible to ensure that those pesky, bloated applications we can never delete are removed, too - and it should be possible to replace some of them with better ones.

Put another way, outside the realm of updating the kernel, a great many customizations should be possible simply because the boot cycle has a new point of custom control, via a new version of app_process.

My first experiments will be in an AVD, hopefully during the weekend. When I think it's ready, I'll post it with usage hints (no, not instructions - if you're not already familiar with the adb shell, root, commands like mount, cp, mv, fdisk, mkfs.ext2 and others, this isn't ready for you). Experience users would then be able to experiment until I can fashion something more suitable for a wider audience.
jokerpks and lucasmun09 like this.
JVene is offline  
Reply With Quote
The Following 4 Users Say Thank You to JVene For This Useful Post:
Lordvincent 90 (February 22nd, 2014), lucasmun09 (January 20th, 2014), morgankslade (January 20th, 2014), o4tuna (April 14th, 2014)
Old January 20th, 2014, 01:37 PM   #5 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

Nice job and really nice documentations you got there! But I advise you to pause what you are doing for a moment... Optimus F6 may be receiving the 4.4 kitkat from the lg soon. LG to skip Android 4.3 and update many of its devices to Android 4.4

What you are doing is really great but I am afraid that certain things that you are trying to do may not be compatible with the new KitKat's structure.... Not that I'm going against what you are doing lol.... but I can't just see someone who is working very hard to solve a problem ending up becoming obsolete because of the incompatibility...

Quote:
Originally Posted by JVene View Post
Update:

The download of the source for Android OS 4.1.2 is a little over 8 Gbytes, which will take a while. That means my experiment will likely not commence until late Saturday. I could attempt just to compile the app_process binary by itself, but it is dependent on OS shared libraries targeted at 4.1.2, and I'd prefer to install the build into an AVD and experiment with it as a proxy for the F6, so I'm loading the whole thing. Building it will take some time, too.

In the meantime, while studying the potential hinted in my previous post, I have some observations to add.

This can do more than merely remount /data on the external SD card. It can also customize the OS considerably.

As you probably realize, when Linux boots the stock software is restored, destroying any interesting customizations one might hope for in a custom ROM. Until there's a sure way of unlocking the boot loader, customization could still be realizable if we accept a few inconveniences in the mix.

First, this is still going to be 4.1.2. That won't change. This technique can't easily substitute a new kernel. However, 4.1.2 could be considerably customized, simply because app_process will have control in Linux before zygote is launched, and will block the initiation of zygote until all adjustments are completed.

This means, for example, there could be a custom boot logo sitting on the SD card, waiting to be copied into position. The original is obviously going to be in position at first during boot (every time), but once it's placed, just prior to zygote's initiation, the new app_process could substitute a replacement. The zygote process is what shows the boot logo. You see nothing on the display until zygote shows it to you during a normal boot.

It should be possible to ensure that those pesky, bloated applications we can never delete are removed, too - and it should be possible to replace some of them with better ones.

Put another way, outside the realm of updating the kernel, a great many customizations should be possible simply because the boot cycle has a new point of custom control, via a new version of app_process.

My first experiments will be in an AVD, hopefully during the weekend. When I think it's ready, I'll post it with usage hints (no, not instructions - if you're not already familiar with the adb shell, root, commands like mount, cp, mv, fdisk, mkfs.ext2 and others, this isn't ready for you). Experience users would then be able to experiment until I can fashion something more suitable for a wider audience.
lucasmun09 is offline  
Reply With Quote
The Following User Says Thank You to lucasmun09 For This Useful Post:
morgankslade (January 20th, 2014)
Old January 20th, 2014, 03:23 PM   #6 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by lucasmun09 View Post
Nice job and really nice documentations you got there! But I advise you to pause what you are doing for a moment... Optimus F6 may be receiving the 4.4 kitkat from the lg soon. LG to skip Android 4.3 and update many of its devices to Android 4.4

What you are doing is really great but I am afraid that certain things that you are trying to do may not be compatible with the new KitKat's structure.... Not that I'm going against what you are doing lol.... but I can't just see someone who is working very hard to solve a problem ending up becoming obsolete because of the incompatibility...

Thanks for the heads up (I didn't know they were planning 4.4 soon) - and let me address the point and how this relates to the Xpose framework.

I'm new here, so a touch more self introduction may be in order. I have over 30 years of experience as a software engineer, with targets ranging from business applications to device drivers, with stopovers in video post production products, 3D graphics engines, robotic manufacturing and machine control...a great deal of the entire map. Unix has been at the heart of my work (Linux, too), but of course Windows and Mac have taken my attention at key points over the years. My point is, this is so far from my 10th rodeo that it's familiar territory. Particularly regarding your point of the platform changing out from underneath my work.

Also, it's not really hard work here. I have to share the Internet with a family of users, so occasionally my huge downloads cause family .... uh...discussion For that, I accept the occasional delay. I'm simultaneously dealing with everything from homework help to laundry to a flat tire to 3 simultaneous development projects (somewhat related to each other) targeting iOS, Android, Tizen and Blackberry - all temporarily interrupted by a frantic call by an OLD friend (colleagues about 10 years ago) sweating bullets because he accidentally deleted an entire corporate database on AIX based RS-6000 servers at his job, at about 2AM - for which there's no "undelete" feature - so I had to reconstruct the directories for him before business opened at 6am.

The project discussed here is just a public sharing of work I'm doing for my son's phone, a Christmas gift. He's 12 - wants a few games in the 300 to 900 Mbyte download range, and I don't want to have to hear about it ongoing . I assume the result will be of interest, because my own experiments thus far demonstrate it's a complete solution (caveat on your very valid point included, because that too affects my son's phone).

I found that this approach I'm settling on happens to be related to the Xpose framework. Xpose also modifies app_process as the primary hook for performing it's feature set. The purpose is entirely different, but the key point of contact is the same.

I pointed this out to Xpose primary developer, rovo89 on xda-developers, because conceptually there are only about 2 or 3 lines of code to be added to app_process in order to provide the needed feature, and since BOTH ideas replace app_process, they would be mutually exclusive without either some cooperation on the point, or a fork of the Xpose source only to provide an alternative app_process binary.

To that end, I've taken careful review of the 4.4 Android source. For app_process there's only one source file, app_main. It's really tiny, extremely simple. This code hasn't changed significantly since about 1.6 or earlier. There are minor differences between froyo and ICS (enough that versions of app_process must be built for either), and I have interest in producing a froyo version as well.

There's no difference between that source in ICS, JB...basically 4.0 through the current master branch of 4.4+.

The code itself produce a tiny binary, which does most of it's work by dynamically loading a shared library, to inherit the runtime object of the VM (that way, it adapts to the local version of Android nicely - app_process built for 4.04 works fine in 4.1.2, for example).

Another build may be required for 4.3 and yet another version for 4.4, primarily due to differences in the shared library it references - but so far I've found that in the emulators it makes very little difference. Providing 3 or 4 versions of app_process is a simple matter. It takes less than 4 seconds to build app_process - less if my machine weren't also busy transcoding video in the background (long story).

Key to the design plan is the fact that new code in app_process doesn't have to do much. It simply runs a script - a hook into making everything else happen. If the script isn't there, the code just continues doing what it's always done - launching zygote. If the script is there, that's where all the work required is done - not in app_process.

When an update occurs OTA or by recovery, the /system/bin directory would undoubtedly be updated, and the stock app_process replaced. The planned approach to the storage solution would simply boot the phone in a "stock " arrangement when this happens. A simple subsequent replacement of app_process targeting 4.4 would restore the feature completely, with one caveat. The /data directory on the SD card would likely be the older version (the OTA update from the service provider would probably have ignored it, and certainly a recovery flash would behave as though it were not there). This means a simple script which compares the content of the /data form the internal card to the /data version on the external card would suffice to align the external correctly (or the reverse if an OTA update happens to route to the external SD card's version of the content).

Like I posted early on, this is a thread documenting a little research and experiment on an F6 in my possession, and soon I'll post an app_process build (or 3) which can be used by knowledgeable members here who may wish to experiment, fully prepared to recover their devices.

After a little more effort on the project it might be more suited for lesser experienced individuals, or for someone else to take the concept further, along with work done with the Xpose framework itself.
irha73 and jokerpks like this.
JVene is offline  
Reply With Quote
The Following 3 Users Say Thank You to JVene For This Useful Post:
Lordvincent 90 (February 22nd, 2014), morgankslade (January 20th, 2014), o4tuna (April 14th, 2014)
Old January 20th, 2014, 04:16 PM   #7 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Oh, nearly forgot to add this interesting possibility:

During this critical moment in the boot cycle, with zygote not yet initialized, it may be possible to substitute the Dalvik VM. This is one reason I'm loading the entire source for several versions of Android. My hope is that, if desired, a build of the Dalvik runtime generated with the Linaro toolchain could be substituted. If the original ROM was not built with Linaro, this could offer a considerable performance boost. I'll get to that tinkering after storage is solved, but like the Xpose framework, this hook (app_process) offers a wide range of possible enhancements to devices with locked bootloaders typically associated with custom ROMs.
JVene is offline  
Reply With Quote
The Following User Says Thank You to JVene For This Useful Post:
morgankslade (January 20th, 2014)
Old January 20th, 2014, 04:19 PM   #8 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

Just wondering with all the respect, have you tried this? [MOD]App2External_SD (Fixed Duplicate files)
I see that your problem is with your son having a small amount of storage and is not able to install apps larger than 600mb and so on. This swaps the sd and internal partitions and enables the user to download these such apps. It is working fine for me. If I missed something I am very sorry but that is what I have taken in from the post above.

Also, your goal is to swap the partitions not just the data files but the actual mounting point I presume so when I install the app, it installs the app installment and the data directly to the sd card without emulating it to be one.

Quote:
Originally Posted by JVene View Post
Thanks for the heads up (I didn't know they were planning 4.4 soon) - and let me address the point and how this relates to the Xpose framework.

I'm new here, so a touch more self introduction may be in order. I have over 30 years of experience as a software engineer, with targets ranging from business applications to device drivers, with stopovers in video post production products, 3D graphics engines, robotic manufacturing and machine control...a great deal of the entire map. Unix has been at the heart of my work (Linux, too), but of course Windows and Mac have taken my attention at key points over the years. My point is, this is so far from my 10th rodeo that it's familiar territory. Particularly regarding your point of the platform changing out from underneath my work.

Also, it's not really hard work here. I have to share the Internet with a family of users, so occasionally my huge downloads cause family .... uh...discussion For that, I accept the occasional delay. I'm simultaneously dealing with everything from homework help to laundry to a flat tire to 3 simultaneous development projects (somewhat related to each other) targeting iOS, Android, Tizen and Blackberry - all temporarily interrupted by a frantic call by an OLD friend (colleagues about 10 years ago) sweating bullets because he accidentally deleted an entire corporate database on AIX based RS-6000 servers at his job, at about 2AM - for which there's no "undelete" feature - so I had to reconstruct the directories for him before business opened at 6am.

The project discussed here is just a public sharing of work I'm doing for my son's phone, a Christmas gift. He's 12 - wants a few games in the 300 to 900 Mbyte download range, and I don't want to have to hear about it ongoing . I assume the result will be of interest, because my own experiments thus far demonstrate it's a complete solution (caveat on your very valid point included, because that too affects my son's phone).

I found that this approach I'm settling on happens to be related to the Xpose framework. Xpose also modifies app_process as the primary hook for performing it's feature set. The purpose is entirely different, but the key point of contact is the same.

I pointed this out to Xpose primary developer, rovo89 on xda-developers, because conceptually there are only about 2 or 3 lines of code to be added to app_process in order to provide the needed feature, and since BOTH ideas replace app_process, they would be mutually exclusive without either some cooperation on the point, or a fork of the Xpose source only to provide an alternative app_process binary.

To that end, I've taken careful review of the 4.4 Android source. For app_process there's only one source file, app_main. It's really tiny, extremely simple. This code hasn't changed significantly since about 1.6 or earlier. There are minor differences between froyo and ICS (enough that versions of app_process must be built for either), and I have interest in producing a froyo version as well.

There's no difference between that source in ICS, JB...basically 4.0 through the current master branch of 4.4+.

The code itself produce a tiny binary, which does most of it's work by dynamically loading a shared library, to inherit the runtime object of the VM (that way, it adapts to the local version of Android nicely - app_process built for 4.04 works fine in 4.1.2, for example).

Another build may be required for 4.3 and yet another version for 4.4, primarily due to differences in the shared library it references - but so far I've found that in the emulators it makes very little difference. Providing 3 or 4 versions of app_process is a simple matter. It takes less than 4 seconds to build app_process - less if my machine weren't also busy transcoding video in the background (long story).

Key to the design plan is the fact that new code in app_process doesn't have to do much. It simply runs a script - a hook into making everything else happen. If the script isn't there, the code just continues doing what it's always done - launching zygote. If the script is there, that's where all the work required is done - not in app_process.

When an update occurs OTA or by recovery, the /system/bin directory would undoubtedly be updated, and the stock app_process replaced. The planned approach to the storage solution would simply boot the phone in a "stock " arrangement when this happens. A simple subsequent replacement of app_process targeting 4.4 would restore the feature completely, with one caveat. The /data directory on the SD card would likely be the older version (the OTA update from the service provider would probably have ignored it, and certainly a recovery flash would behave as though it were not there). This means a simple script which compares the content of the /data form the internal card to the /data version on the external card would suffice to align the external correctly (or the reverse if an OTA update happens to route to the external SD card's version of the content).

Like I posted early on, this is a thread documenting a little research and experiment on an F6 in my possession, and soon I'll post an app_process build (or 3) which can be used by knowledgeable members here who may wish to experiment, fully prepared to recover their devices.

After a little more effort on the project it might be more suited for lesser experienced individuals, or for someone else to take the concept further, along with work done with the Xpose framework itself.
lucasmun09 is offline  
Reply With Quote
The Following User Says Thank You to lucasmun09 For This Useful Post:
morgankslade (January 20th, 2014)
Old January 20th, 2014, 05:42 PM   #9 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Yes, I tried several approaches along the same basic lines.

If you examine the script for app2external_SD, you'll notice it mounts /storage/sdcard0 over a directory created /data/internal_sd. If you examine the stock device, you'll realize /storage/sdcard0 doesn't actually represent a card, or a partition of a card, it represents a mount of the directory /data/media. It's just a directory inside /data. It's also where many of the downloads are directed.

/data, however, is a partition of the internal card (it's mounted from an entry in /dev/block, not another directory in a filesystem previously mounted). The /storage/sdcard0 intent of the stock phone is simply to put stuff in /data/media, alongside everything in /data/app, /data/data and other material under /data. /data is the mount of a 1.2Gbyte partition on the internal SD card. What is being swapped is NOT the internal_SD card, but a representation of a subdirectory masquerading as an internal sd card.

Most downloads route to a location in /data/media via /storage/sdcard0 or some other approach. However, data downloaded by other applications may end up in some other directory underneath /data/data, which app2external_SD hasn't re-routed, and is sharing that 1.2Gbyte storage space.

In other words, while the internal storage size now appears to be large (and therefore DOWNLOADS go to a larger area), the location for actual applications and their data is still limited - it's still in the 1.2 Gbyte region. You can download a large file, but if the phone is stuffed (like my son's gets) with applications already, that's not going to solve the larger problem - the storage of the application and it's data after it's downloaded.

Some have reported combining app2external_SD with link2sd. That application works with a different approach. Instead of remapping drives, it does a grander version of what "move to phone" does in the stock usage.

Move to phone creates a virtual filesystem, mounts it, then points Android to that location for the APK which launches the application (removing it from /data/app, freeing space). Unfortunately, in most cases, it doesn't move material in /data/data. The virtual filesystems created are fixed sizes, created to match the storage requirements of the APK and it's cousins. It won't expand over time, and since application data DOES expand, move to phone doesn't help all that much for material in /data/data.

Not only does "move to phone" leave /data/data material in that 1.2Gbyte partition, it also doesn't relocate the dalvik-cache directory under /data.

link2sd helps move more of the material out of the 1.2Gbyte card than move to phone can do, using symbolic links.

Unfortunately, this still means we're functioning out of a 1.2Gbyte partition of a 4Gbyte sdcard for everything else.

Put another way, the page in settings which shows storage is wide open can lie to you. It's easily fooled because it doesn't represent the entire map of storage as does the df command in Linux, which you can see through a terminal app or the shell.

It does help, at first, but you'll run into yet another wall while plenty of room is still open on the external card but none left in that 1.2Gbyte partition on the internal card.

Most of the other solutions are done application by application (tedious) - and while app2external_SD helps, it too will yield to the reality that only PART of the storage has been remapped.

At one point I remounted /data, the root of all that. It drove Android nuts until I executed a soft reboot (killing zygote). That works, and soft reboots are well known on many devices, but on the F6 soft reboots are spuriously prone to errors. This phone doesn't really like soft boots. Plus, a soft boot shows the boot logo - and in my first attempt at a solution, that meant two boot cycles (or, when it went wrong, a boot loop lock).

The engineer in me wanted to open up the phone, desolder the internal SD chip, and put a 32Gbyte chip in it's place.

Both my son and my wife had objections, so I didn't

I'm determined, therefore, to do exactly the same thing without the hot iron!

Grab a free terminal app if you don't have ADB and get to the command line, the type df.

You'll get a report something like this (though this one is mine, mid experiment)

Code:
Filesystem                 Size         Used         Free    Blksize
/dev                    402.14M       64.00K      402.08M       4096
/system                   1.19G        1.06G      132.54M       4096
/mnt/asec               402.14M        0.00K      402.14M       4096
/mnt/obb                402.14M        0.00K      402.14M       4096
/data                    29.04G        5.63G       23.41G       4096
/sns                      7.91M        4.11M        3.80M       4096
/mpt                     31.54M       12.50M       19.03M       4096
/persist-lg               7.91M        4.14M        3.77M       4096
/cache                  788.39M       14.38M      774.01M       4096
/persist                  7.91M        4.10M        3.81M       4096
/firmware                63.95M       41.80M       22.16M      16384
/storage/external_SD     29.04G        5.63G       23.41G       4096
This tells you the real story of storage.

Notice there's no /storage/sdcard0 ?

That's because it's not really a device.

Did you notice /data above? You can df /data to see. Compare it to yours.

Now try this, the df command directed to interrogate a path:

df /storage/sdcard0

Code:
Filesystem                 Size         Used         Free    Blksize
/storage/sdcard0         29.04G        5.63G       23.41G       4096
Yours MIGHT look like mine, because they're routed to the external card. The stock phone would show only a 1.27Gbyte partition here.

In my device, under experiment, /data is not mounted from /dev/block/mmcblk0p15, as it is in the stock device. It's mounted from /dev/block/mmcblk1p1 (soon to be changed to p2) - the external card.

Now compare a little deeper:

Examine with these commands in a shell

df /storage/sdcard0
df /data/app
df /data/data
df /data/dalvik-cache
df /data/media

What space is it showing you? The space of the external card for all these directories?

On a stock phone, /data/app would look like this:

Code:
/data/app                 1.27G      584.55M      716.93M       4096
Representing a 1.27 Gbyte partition, with 584 Mbytes consumed, 717 Mbytes available.

On my device all of these directories show the external storage space, just like /storage/external_SD on my device. Real, genuine open space.

Code:
/data/app                29.04G        5.63G       23.41G       4096
Here it shows I have room for another 23Gbytes of apps.

If any of yours do not show the external storage space, then the smaller ones are on the internal card and are going to fill up at some point, with plenty of room open on the external card and no room to install applications left.

The Android storage app can lie to us when tools remount storage partially.
JVene is offline  
Last edited by JVene; January 20th, 2014 at 05:51 PM.
Reply With Quote
The Following 2 Users Say Thank You to JVene For This Useful Post:
aaanadie (January 26th, 2014), morgankslade (January 20th, 2014)
Old January 20th, 2014, 06:11 PM   #10 (permalink)
Senior Member
 
morgankslade's Avatar
 
Join Date: Aug 2011
Location: Southeast, Michigan
Posts: 786
 
Device(s): Nexus 5 v.4.4.4 Purity ROM/CM11, Moto G Stock 4.4.4, LG F6 Stock
Carrier: MPCS

Thanks: 661
Thanked 112 Times in 89 Posts
Default

I am really glad you guys are putting your heads together to help F6 thread out.
I don't know how you all are doing this, hats off to you.

Will all this work if we get rumored kitkat update?
__________________
Who!
morgankslade is online now  
Reply With Quote
sponsored links
Old January 20th, 2014, 06:53 PM   #11 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by morgankslade View Post
I am really glad you guys are putting your heads together to help F6 thread out.
I don't know how you all are doing this, hats off to you.

Will all this work if we get rumored kitkat update?

At the risk of invoking Senator Palpatine's voicever likeness:

I will make it work, because I have the F6.

o4tuna and 13onk like this.
JVene is offline  
Reply With Quote
The Following 2 Users Say Thank You to JVene For This Useful Post:
morgankslade (January 20th, 2014), SUPPORTLOCAL (April 19th, 2014)
Old January 20th, 2014, 07:30 PM   #12 (permalink)
Senior Member
 
morgankslade's Avatar
 
Join Date: Aug 2011
Location: Southeast, Michigan
Posts: 786
 
Device(s): Nexus 5 v.4.4.4 Purity ROM/CM11, Moto G Stock 4.4.4, LG F6 Stock
Carrier: MPCS

Thanks: 661
Thanked 112 Times in 89 Posts
Default

In Fat Albert's voiceover likeness:

Hey, Hey, Hey.
morgankslade is online now  
Reply With Quote
Old January 21st, 2014, 02:43 AM   #13 (permalink)
Member
 
pressy4pie's Avatar
 
Join Date: Feb 2012
Location: /media/connor
Gender: Male
Posts: 358
 
Device(s):
Carrier: MetroPOS

Thanks: 136
Thanked 154 Times in 76 Posts
Default

Quote:
Originally Posted by JVene View Post
In know the audience will range from the inexperienced to experts, and while this discussion is intended to include all, be warned:

Nothing here is a downloadable solution for novices or casual users, but is a discussion for those of us who know how to unbrick our phones when we experiment with dangerous ideas, understand the Linux command line, the development tools (specifically adb), and know how to partition an SD card through ADB from the linux command line served from the phone without some downloaded app.

If that's not you, you may still find this discussion interesting (and hopeful regarding the storage problem), but only try any of this if you know what you're doing.

Read this like the notes from an engineer's desk after running some experiments.

What I'm posting here is a working theory tested in my F6 (and genymotion) for a grand solution to the typical storage problem. We're all running out of room when there's an empty 32 GByte SD card installed. We all know that the various "link" and swap solutions aren't really doing the trick. You may have even gotten your phone's storage page to claim there's 29Gbytes of free space, but realize after trying to use it that it's not entirely true.

Well...this may do the trick, but it's not without a compromise, but one chosen to be outside the area of compromising storage.

If we could edit init.rc we could simply alter the boot sequence so the main directory of interest, /data, would no longer mount from mmcblk0p15 (maybe it's 14, I'm not looking at it right now)..but from some other source, like the external SD card (typically mmcblk1p1). mmcblk0 is the base node of the 4 Gbyte internal SD card, and mmcblk0p15 is a 1.2 Gbyte partition on that card, holding the content of the /data directory.

Substituting mmcblk0p15 for mmcblk1p1 (or p2, if the external card has two partitions) would provide a genuine primary source from the SD card for /data, providing THE solution to the storage problem.

However, the F6's boot loader is locked, so every time the phone boots, init.rc, much of the root directory structure, and anything we might custom configure is restored to "default" conditions.

If not for that lock, this could be as simple as doing something like this just one time:

cp -rp /data /storage/ext2filesys/data

which duplicates /data and all it's contents (pretty much all the applications and their data), including permissions and links.

and then during boot:

mount -o rw,bind /storage/ext2filesys/data /data

Assuming that ext2filesys represents a partition from the external SD card formatted in ext2 filesystem. It would be more like:

mount -o rw -t ext2 /dev/block/mmcblk1p2

If we were making a custom ROM and installation on the external card for this sort of thing, but we're not. We're using the bind option to mount a directory over another directory instead, but it's similar.

....and, of course, a few other machinations in detail.

However, the fact that we can't edit init.rc (or it's cousins) without cracking the boot loader lock means that we can't perform this mount before zygote is launched. Zygote is name of the process running the Dalvik runtime, it's basically what everyone see's as Android, running as a process inside Linux.

Most here are familiar with smanager and/or universal init.d, or something similar. They run scripts "during boot"....but, they run them AFTER zygote is running, because they're Android applications, not Linux scripts. At that point zygote (another name for Android) is already dependent on the content of /data, and in particular some of it's major and minor subdirectories. Google's playstore is key among them, because it's a wimpy, whiny app that will repeatedly complain once /data is swapped out underneath it, making the phone all but unusable.

So....the basic novel idea I'm proposing is to kill the zygote.

At this point the astute readers probably have an expression of utter disgust.

It's a FUGLY proposition, but one that happens to work so far as I can see.

The idea is to view Android in two parts. The first part is Linux. It boots up and operates like a stripped down but otherwise standard Linux build. Once it's going, which happens rather quickly, it launches zygote. That's when you see the boot logo.

At that point it's already too late to REALLY substitute storage, unless you're willing to view zygote (and thus the Android PORTION of this two piece structure) as individually rebootable - by killing zygote.

Init, the grandparent process of the entire Linux implementation here, will restart zygote, probably within 1ms. When it does, it happens AFTER the substitution mount command example above, making /data serve from the external SD card.

You can try this without script editing yourself, if you're a bold experimenter. Simply arrange for a duplicate of /data on an ext2 filesystem, mount it (driving Google's apps nuts), then ps | grep zygote, note the pid and then kill zygote. It LOOKS like the phone reboots, but it's only the zygote rebooting (and thus Android) - NOT the entire Linux operating system underneath Android.

And zygote will boot as if for the first time, after having reconfigured Linux the way we want.

So far I'm observing zero complications and 100% satisfaction on the storage problem with this approach.

No app by app fiddling with links, or move to phone - just space. Actual, real space.

If you're familiar with fdisk, mkfs.ext2, mkfs.vfat (for a fat32 partition of you want one), the mount command and such, you can probably put the pieces together yourself.

Now, making a script that can grep the zygote process from the output of ps, then sed and/or awk that to get the pid to fashion a kill command is an ugly looking bit of code which causes me to reach back 30+ years into my early Unix work, which I've happily let slip from memory, though Google provides.

Frankly I'm so much more comfortable with the NDK, and the stand alone toolchains that I'm motivated to write a C/C++ command line utility to do it instead, but maybe I'll try my hand at the script for a bit.

The basic plan is this:

Upon power up, Android (as zygote) initializes from the default, internal version of /data as mounted from mmcblk0p15 - the stock version of /data, after rooting, equipped with smanager or universal init.d to initiate a script in /system/etc/init.d.

That goes through machinations to mount the ext2 partition (they don't automatically mount easily on the F6) - then, calls another script on the ext2 partition to continue. I suggest this so that simply pulling the SD card boots a "standard" configuration based on the stock version of /data.

The second script, probably launched as a new thread (using the & at the end of the command, for example), would perform the substitution for /data served out of mmcblk1p1 or p2, then kill zygote to reboot it.

An interesting consequence is that, from the Android perspective, you have two operating systems. One on the internal card at mmcblk0p15, and the other on the external card at mmcblk1p2 (or p1 if you choose a single ext2 partition).

This means that all applications you install after such a modification will end up in the external partition. If you pull the card and reboot, with the design plan mentioned above, the "stock" or internal version of /data remains...without those applications installed.

Or any photos you've taken, or downloads...etc.

This essentially abandons that admittedly puny 1.2Gbyte partition that came with phone, relegating it to a boot and self destroy function, simply to launch the version from the external card.

Until someone cracks the boot loader lock (before the phone is outdated and replaced, that is), this may be the only way to gain REAL open storage on the external SD card.

Anyone up for a ride?
very interesting, but over at the metro side of the woods we have loki ( not sure if tmo does or not) and im making a boot.img for stock roms, with said data tweak in the init.rc. anyone in tmo-land wanting to test 4 me?
__________________
Sammyz is a babe
pressy4pie is online now  
Reply With Quote
Old January 21st, 2014, 09:42 AM   #14 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

I would be glad to but im not exactly sure what you are trying to do....
Quote:
Originally Posted by pressy4pie View Post
very interesting, but over at the metro side of the woods we have loki ( not sure if tmo does or not) and im making a boot.img for stock roms, with said data tweak in the init.rc. anyone in tmo-land wanting to test 4 me?
lucasmun09 is offline  
Reply With Quote
Old January 21st, 2014, 10:41 PM   #15 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by lucasmun09 View Post
I would be glad to but im not exactly sure what you are trying to do....
loki is an exploit that creates a new boot image bypassing the security method used to lock the ROM, so custom ROM's can be installed.

It was originally written for some Samsung devices, but two examples of the LG Optimus F6 have appeared.

The problem, however, as I see it, is that there's a real possibility (and a fairly high one) that it will seriously brick the phone in a way not familiar to even many experienced users.

Loki happily recognized my "h" metro ROM as an "e" version - and verified that it created a new, unlocked version I could modify.

Unfortunately the resulting image wasn't valid. The source reveals there are tests used which are too simple, such that it's easy for it to misidentify what it's editing.

If the match is exact, loki has worked. If not, you can very easily wreak havoc. If you flash backwards to "e" it should work, but that would also be an issue if one wanted to avail themselves of the 4.4 update pending. It would be a wait to get a copy of the 4.4 which could be fashioned by loki to work.

I like the potential, but I'll personally wait on implementing loki on the F6.
JVene is offline  
Reply With Quote
Old January 24th, 2014, 10:29 AM   #16 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Ok, here's something you can try if you qualify yourself as reasonably experienced on tinkering with Android devices.

If you're completely satisfied with solutions like app2external_sd and/or link2sd, or similar applications, you should probably keep using them. If you discover, however, like I did, that they're not complete solutions and instead want the primary storage of the entire /data directory to mount from the external SD card, this may be what you're looking for.

As of this post, this is an ALPHA version, not packaged for end users. If you're experience with Android, are familiar with ADB, have rooted the phone, familiar with Linux command line usage and are completely prepared to restore the phone with either LG's tools or a kdz of the stock operating system as a precaution for your own experiments, then you're qualified to try this. If you're not entirely familiar with these concepts then you should wait for this project to advance, but you may find it interesting to follow.

Read what follows as if you found the notes from an engineers desk detailing an experiment. This explains how it works and how to try it.

NOTE: I said TRY IT. Some are frustrated with the LG devices, F6 and F3 among them, because of the storage problem, and are eager to try solutions. The link(s) at the bottom are EXPERIMENTAL, and the reason they're posted is for experienced users to assist in PROVING them to be viable. If you're more of an end user looking for a quick solution you can install, this is not yet ready for that.

If you are knowledgeable and confident to experiment, then consider this a sincere request to try it and post back with problems, observations and suggestions. The intent is to provide something more suitable for the typical user capable of rooting their phone, but little else.

I have this working on my son's F6 - it can work on yours, or your F3, L9 or most any device with Android 4.0.3 through about 4.2.2, with a really simple and quick move all the way to 4.4.2.

I type fast and, like Pascal once noted, I don't have time to write a short post, so I wrote a long one.


This project was started around Jan 1, 2014, prompted by a Christmas gift of an LG Optimus F6 for my son. While the phone is wonderful, the /data directory is mounted from a 1.27 GByte partition on the internal SD card, which easily fills up (that took all of 2 hours to discover). I tried various solutions, from app2external_sd combined with link2sd, as well as manual experimentation using a shell through adb on the phone. Nothing really works. They help, there's no doubt. You can manage to relocate applications to the point where you free up another 300 to 500 Mbytes, maybe more, but that 1.27 Gbyte partition, mmcblk0p15, still fills up eventually. My son is 12. He's already fetched a 600Mbyte GTA 3 and a 900 MByte GTA San Andreas. It's possible to finagle these into the phone, but that's only a temporary solution. Between my son and my wife there will be over two hundred apps installed, I see it coming. /data/dalvik-cache may need 1 Gbyte just for the dex files.

Well, I'm a software engineer of 30+ years experience, with a long history of Unix and Linux work, so naturally I decided to cobble together a serious solution.

While there is information that "loki" is available to unlock the boot loader of the F6, there are valid reasons for concern using it. Loki exploits a minor flaw in the security method of the boot cycle, allowing a modified image to be booted instead of the approved factory image. The problem with using it, as I discovered, is that at this point support for the F6 is limited to specific carriers and versions of the OS. Loki happily generated an invalid image for my F6 without warning, and the potential for serious bricking of the phone is fairly high. Rumor has it that Android 4.4 will be coming to the F6, and if you rely on loki to unlock the boot loader as a means of solving the storage problem, you may be out of luck for a long while waiting for loki to support version 4.4 on your combination of carrier and version. I support what loki is doing, but I can't see myself using it for this problem.

The solution presented here will still work on the 4.4.x version, without waiting. The technical modification is actually simple. It does rely on the fact that LG leaves the content of /system intact between boots. I suspect they do this because they link pre-installed applications from /data/app to /system subdirectories, and want to be able to update them without updating the system.img stored in ROM. Factory resets, as a result, will restore the content of /system, and therefore remove the modifications proposed here. They are, however, easily re-installed.

I have this working on my son's F6, and the experiments have been repeated on emulators and other devices, including a Kindle Fire first generation on Android 2.3.3 (but I'm not bothering with posts on supporting earlier than 4.0.3 here). It works in all of those situations. Your mileage may vary.

The goal is to provide a source for the /data directory from the external sd card - all of /data, not just links or subdirectory mounts. We are prevented from doing this because the boot loader is locked, and therefore all changes to the important file init.rc and it's cousins will evaporate at each boot. This would be trivial to solve if we could edit the ramdisk.img, or files in other images, but we can't (reliably, IMO).

/data is mounted from a partition of 1.27 Gbytes, /dev/mmcblk0p15 (on the F6). The content of other important directories like /data/dalvik-cache, /data/data (and all it's children) and /data/media are all served from that 1.27 Gbyte partition. It isn't practical to mount other sources over these directories because in the stock configuration (which we're bound to by a locked boot loader) we have no way of performing those mounts until after the Dalvik VM has launched, and critical Android applications are already dependent on the content of these directories.

Early on I tried to mount /data from the external card, which irritates Google's Gapps and it's cousins to the point of repeated pop-up complaints, making the phone unusuable until they're disabled. A soft reboot fixes this, but the F6 is very touchy about soft reboots for some reason. I don't like a soft boot approach either - it's very messy, leaving material in various databases unkempt, sometimes confusing applications.

What is required is a way to fire a shell script before the Dalvik VM launches. Smanager, Universal init.d and similar tools can't do that. They begin execution from within the Dalvik VM, after it's already running.

Outside of altering init.rc, the solution is to replace a single small binary file called app_process found in /system/bin. Links at the bottom for alternate version(s) of this binary were built from AOSP source version 4.1.2_r1, which are indentical from 4.0.3 through the current master branch, 4.4.2 - though I'm going to append one built from the 4.4.2_r1 AOSP source to be certain we're set for the 4.4 update. This modified version of app_process blocks the initializatin of zygote, the name given to the Dalvik VM's process, until a shell script at /system/etc/init.d/zinit.sh is finished executing, if and only if the script exists (otherwise, zygote is launched as it always was).

The solution coincides with another product called Xpose. The key to Xpose is also a modified version of the app_process binary, which provides additional "hooks" for customizing Android when the boot loader is locked, or a custom ROM is undesirable or unavailable.

I noticed the key importance of the app_process binary just before I discovered Xpose, while perusing both init.rc and the AOSP source of Android 4.1.2_r1. This is a tiny file, and a deceptively simple source code. Xpose does not take advantage of the timing of the initialization of the Dalvik VM.

I point all of that out because if you use Xpose you must choose a version of the app_process which is actually a fork of Xpose - so that you get BOTH of these features combined. Installing this solution would otherwise replace Xpose, disabling it. I've posted this with only app_process without Xpose as of this writing, but an Xpose version will appear later.

The original version of the app_process source found at https://github.com/android/platform_frameworks_base/blob/master/cmds/app_process/app_main.cpp has this clause:

Code:
    if (zygote) {
        runtime.start("com.android.internal.os.ZygoteInit",
                startSystemServer ? "start-system-server" : "");
    }
This is what starts the Dalvik engine. This code hasn't changed since 4.0.3 or before, and is very similar to code going back to 1.6 and earlier. This clause is run only once at power on, or on a soft boot if zygote dies. Xpose does not alter this clause. However, I alter it something like this:

Code:
    if (zygote) {

        if ( access( "/system/etc/init.d/zinit.h", X_OK ) == 0 ) system( "/system/etc/init.d/zinit.h" ); 

        runtime.start("com.android.internal.os.ZygoteInit",
                startSystemServer ? "start-system-server" : "");
    }
This simple system call, executed on the condition that the script zinit.sh exists and has execute permission, is the entire modification made to app_process under this proposal (Xpose does other modifications). It is therefore conditional - that is, app_process works exactly as always if the script doesn't exist. Readers familiar with C may recognize known objections to the use of access, favoring stat - but they are concerns based on irrelevant security issues in this context. The validity of the objection is the fact that access uses a view of the file's permission in a way that differs from functions which actually open the file. It's possible these two permission profiles differ. However, in this use, the code is running as super user (even if the phone were not rooted, obviously after install) - so the notion is entirely irrelevant. stat would report the same view here, while access is smaller.

What's important is that zygote will NOT be initialized until after the script completes (if it's executed at all), giving us the opportunity to reconfigure a great deal of the operating system without disturbing Dalvik, or Google's Gapps, or anything else known to us as Android.

Installation is merely:

Code:
mount -o remount,rw /system
cp /storage/external_SD/app_process_non_xpose_4.1.2 /system/bin/app_process
busybox chmod a+x /system/bin/app_process
mount -o remount,ro /system
Where /storage/external_SD/app_process_non_xpose_4.1.2 is the modified binary provided for non Xpose compatability (or one of the other versions I'll be appending via links at the tail of this post). Any means of copying app_process into /system/bin is fine.

The remount is required when /system is mounted read only (the default after boot).

Obviously, busybox was assumed in this example...the default version of chmod doesn't support a+x, it takes only octal parameters.

Now, astute readers immediately realize this has nothing to do with storage.

That's correct, this only provides a hook by which we now have the ability to execute script(s) before Dalvik starts using /data. Consider this the installation of part 1 of the solution, the first pre-requisite to making this possible. The key is the ability to use mount while Dalvik is not initialized, which is a condition entirely out of reach unless we can unlock the boot loader reliably.

Now we have to have something to mount. The typical external SD card is formatted in the vfat filesystem, otherwise known as FAT32. Unfortunately vfat doesn't support all of the identities and security data required by Android. For that we must use a linux compatible file system. You could choose ext2, ext4 or whatever you like which is linux compatible, and is supported by Android.

Even many regarding themselves as expert may need reminding that the examples of commands which follow purposely erase the entire SD card. The word "backup" does not suffice to explain what is required if there is so much as 1 byte of material you must preserve. Transfer is, perhaps, a more suitable word. The recipient which stores the content of the SD card you may be re-partitioning will be, for a while, the primary source for that material. If there's anything important, it should be duplicated in a physically separate device. Many may not recognize what hidden files/directories are on the external card. These are commonly given names preceded with a ".", which are only visible when specifically directed for display (that is, ls -la instead of ls -l, or directing Linux GUI file browsers to show all files, including hidden files).

Of distinct importance, however, are the hidden virtual filesystem which may appear on the SD card. These are uniquely invisible. Typically these appear as the result of using "move to phone" or other applications which relocate applications to the SD card. These are difficult to backup correctly. The unfamiliar should not apply. Personally, I just reserve the apk's of applications so that re-installation is trivial.

This is exacty why I warned earlier that this is not a tutorial or instruction, but an engineering discussion on how this can be done. If this content is familiar to you (and most Android developers and Linux/Unix users are), this isn't a problem.


At this point it is assumed that the SD card has no content of importance, as it will be completely eradicated.

You can choose whatever method you prefer to partition the SD card, but from my notes this is a general outline of doing this at the command prompt in a shell, accessed via adb with root access:

On the stock device, observe the mount command (type mount with no parameters).

You'll notice that the external SD card is mounted as a vold device with major/minor numbers 7, 33.

Code:
ls -l /dev/block
Reveals that /dev/block/mmcblk1 also is 7,33. This is the BASE to the external SD card on the F6 (other devices may differ). When partitioned, the first partition will be mmcblk1p1, partition two (fashioned automatically when present) will be mmcblk1p2, and so on. They appear automatically when the card is partitioned, or are entirely absent if the card is not partitioned.

The external card must be unmounted

Code:
unmount /storage/external_SD
At one point this was refused, so I had to

Code:
mount -o remount,rw /
rm /dev/block/fuse
unmount /storage/external_SD
Fuse will be re-created on reboot, so that's actually harmless.


I used

Code:
fdisk /dev/block/mmcblk1
The SD card can't be partitioned if it's mounted - the block device is busy when it's mounted, Linux will stop you.

I'll leave operation of fdisk to your own Google research. You might prefer an app like Aparted or something. If you use fdisk, be certain you use the "v" command to verify the partition configuration you choose, and use the "w" command to write that to the drive. If you don't "w" (write), it doesn't change the partitions, and if you don't "v" (verify) you can generate an invalid configuration.

If you're disappointed that I don't detail the use of fdisk, then these notes and this project is not yet ready for you to experiment. There will likely be a version, later, more suitable where scripts may do this work for you.

I personally decided upon a 4Gbyte partition and a 26Gbyte parition as partitions 1 and 2 respectively. fdisk automatically creates linux filesystem partitions (type 83). One most manually chose to edit the partition type and choose Win95 FAT32 to create a vfat partition.

This configuration gave me the feature of having an automatically mounted external SD partition of 4Gbytes, while providing a 26 GByte partition for use as a source for the new /data directory.

When all is finished I have 24+ Gbytes available after the basic applications plus about 6 games and few others installed, without any need for app by app fidling with storage configuration or concern about where downloads are going.

In earlier experiments I chose one 32Gbyte partition instead. That works, but the phone behaves as though there is no external SD card (because that requires vfat to work), and occasionally a message appeared indicating the card seemed damaged (because it was expecting vfat).

Creating the partitions doesn't complete the task, though.

You have to create the filesystems, analogous to formatting a disk.

On my 2 partition arrangment I have 3 devices routing to 7, 33:

/dev/block/mmcblk1
/dev/block/mmcblk1p1
/dev/block/mmcblk1p2

The first is the base....you leave that alone after fdisk has done it's work.

The second, mmcblk1p1, should be the vfat partition (it's where the LG expects to find it). So, make the filesystem for it with

Code:
mkfs.vfat /dev/block/mmcblk1p1
Then, depending on the filesystem you prefer, start with
Code:
mkfs.ext2 -m 1 -LExtData /dev/blockmmcblk1p2
the -m 1 is "reserve 1% for superuser". You might have reason to use 2 or 3...Google for research if you need.

The -L creates the label, in this case ExtData, for the filesystem. It's not an important point, your choice prevails.

Now, if you want to use ext4 you can...but there is no mkfs.ext4. For ext4 you must turn the ext2 filesystem into an ext4 filesystem. I left ext2 on mine, it's fully compliant with Android's requirements, but to make this ext4 you use tune2fs. Google will provide the details, but an example command is:

Code:
tune2fs -O extents,uninit_bg,dir_index,has_journal /dev/block/mmcblk1p2
Now, at this point the partitions are ready for mounting. In my own F6, where mmcblk1p1 is the vfat I mount thus:

Code:
mount -o rw -t vfat /dev/block/mmcblk1p1 /storage/external_SD
This is the vfat partition mounted over the same position as the stock external SD card in a stock device. This mount is performed manually during the work, but upon reboot this partition is mounted automatically by the OS, because it is partition 1 and it is in vfat format. If either of these two points are not true, it will NOT automatically mount during boot.

The second partition of my F6 is mounted thus:

Code:
mount -o remount,rw /
mkdir /storage/sdcard_partition2
mount -o remount,ro /
mount -o rw -t ext2 /dev/block/mmcblk1p2 /storage/sdcard_parition2
A little explanation:

Every boot, the /storage directory is RESTORED to contain only two directories, sdcard0 and external_SD card.

Every boot, therefore, a directory for the second partition must be created (it will not automatically mount). /storage is "owned" by the root directory, which is mounted read only. It must be remounted read write in order to create the directory. That's what the "remount,rX" commands are about.

On my card I chose the ext2 filesystem. If you used tune2fs to create an ext4 system, or you want to use yaffs2, your parameter following -t will differ. Google provides, but again I remind readers if this is Greek, you're still just touring the experiment.

Some may inquire, why not just wait and mount /dev/block/mmcblk1p2 over /data when the time comes? That works, and it is more "natural". However, in my own intent, I want access to the storage of the larger ext2 partition for other purposes (whatever that may be in the future), which means I intend to use a bind mount from a directory on the ext2 partition to serve /data.

Ok, the stage is set...we're almost ready for the play.

In manual experiment I created:

Code:
mkdir /storage/sdcard_partition2/.remounts
mkdir /storage/sdcard_partition2/.remounts/data
This creates a directory which will hold the original contents of /data, and eventually serve (ongoing) all purposes of the original /data, but with larger storage (26Gbytes in this example).

I'll leave the appropriate chmod for you to choose, or to Google if you must...but if chmod is foreign to you, wait for a more public release.

The original /data may be of use, for several reasons, but it will eventually be left alone ongoing until some reason comes that I should boot the phone without the external SD card. It can't be unmounted while the phone is running, and mounting over it with empty contents is the same result.

Copying /data to the external card is as simple as:

Code:
cd /storage/sdcard_partition2/.remounts/data
cp -rp /data/* ./
Which takes a minute or so, depending on the size occupied, but it's not a good idea while in a shell. It will work, and maybe suffice, but Android is running. Google has it's databases open, tinkering around. Whatever we copy is going to be read while in use, possible creating SOME databases with invalid content. Most (maybe 98% or more) will copy just fine, but in my experiments Google was left puzzled when booted with a copy made this way. It eventually settles, but to me it's best to make this copy during the newly created opportunity of running scripts in Linux while Dalvik is suspended from initiating.

The idea is to drop some scripts into position, intiated from /system/etc/init.d/zinit.sh. That script, whatever is in it, will be executed by the new app_process binary just before zygote (the Dalvik engine process) is launched.

So, a one time installation script is in order, followed by a script left in place which operates this configuration from then on.

Further, it is important that we should be able to boot the phone in a "stock" fashion at will...that is, with the original 1.27Gbyte partition servicing /data...from mmcblk0p15.

Also, we may have good reason to review the content of the old friend now and then.

So, here's one of the reasons I wanted to mount a directory in the second external partition rather than mount the partition over /data...zinit.sh will call a script from /storage/external_SD/.bootscripts/external_boot.sh, or something to that effect.

The idea is that when zinit.sh fires on boot, if the external card is present, the script umounts /data from the interal source, mmcblk0p15, then mounts /data from the external card at /storage/external_sd/.remounts/data over /data. If the external card is NOT present, or a different card without a script at /storage/external_SD/.bootscripts/external_boot.sh is inserted, then the original /data is left mounted from mmcblk0p15 and the phone operates as a stock device.

The minor catch being that the script in /system/etc/init.d/zinit.sh must check to see if there's a partition at /dev/block/mmcblk1p2 (in my example configuration), and that it mounts correctly as an ext2 filesystem (or whatever fileysystem you prefer, like ext4 or yaffs2), and then check to see if a script is found at /storage/external_SD/.booscripts/external_boot.sh before attempting to execute it.

The script in /system/etc/init.d must, therefore, be fashioned according to your chosen configuration of the external SD card.

While I suppose some mastery of sed and awk could fashion this script from source text (for a proper "consumer" level installer), I'd personally rather use C++ - but that's for later in the project, not this post.

So, imagining for a moment that we're able to fashion a script that runs before Dalvik is active, I explore a few commands which would be of use in such a script.

The first being:

Code:
unmount /data

mount -o remount,rw /
mkdir /storage/original_data
mount /dev/block/mmcblk0p15 /storage/original_data
mount -o remount,ro /
This block is likely to be a part of that which performs the creation and mount of /storage/sdcard_partition2 mentioned above.

At which point the original content of /data, as viewed on the stock running phone, is visible at /storage/original_data.

The new proposed source for /data will be visible at /storage/sdcard_partition2/.remounts/data, while /data is actually empty at the moment.

As such, for a one time configuration script during boot:

Code:
cp -rp /storage/original_data/* /storage/sdcard_parition2/.remounts/data
Would create a duplicate of the original data, with ownership and permissions duplicated, and links duplicated.

This means that LG and the service provider's links into /system/.... for the pre-installed applications would already be correctly set.

This leaves two interesting potential issues. There may be links created by link2sd or applications like it. They may have pointed to targets in /storage/external_SD/... or to hidden virtual filesystems the way "move to phone" does, depending on what application was used. If the 4Gbyte partition is restored with the old content of the external_SD, links would be restored, but the virtual filesystems are more difficult. I avoided the virtual filesystem problem by simply re-installing the applications, while the links I manually established earlier were served by restoring content to /storage/external_SD.

When the copy is complete

Code:
mount -o rw,bind -t ext2 /storage/sdcard_partition2/.remounts/data /data
supplies the needed /data directory from the external SD card's second partition, an ext2 filesystem in my example (adjust to suit your choice).

The script which performed the copy above would then be removed, replaced with a script that just performs the mount over /data for subsequent boots.

Additional scripts or an app may be required for special situations. The following describes what that's about.

Configuring and using the device this way presents us with the following attributes:

The original /data is known as /dev/block/mmcblk0p15, mounted as /storage/original_data. It will retain older content, but will not be updated when applications update.

The "new" /data is served from /storage/sdcard_partition2/.remounts/data, is therefore visible from two perspectives (though .remounts is technically a hidden directory, it's very easy to get to it), and will receive application updates.

This means there is an opportunity, on occasion, to review /data (as mounted from the external card), compare that to /storage/original_data, and copy updated content to it, including refashioning links that may have changed (should they even exist).

If the phone is booted without the external card, the older versions of the applications would be used...then perhaps updated, if given the opportunity.

If the script in /storage/external_data/.bootscripts/external_bootscript.sh is renamed, a reboot would leave the original /data (from /dev/block/mmcblk0p15) in place, operating on older content, but leaving the external card available - /storage/original_data would not exist, but /storage/external_SD and /storage/sdcard_partition2 would - or could - be mounted and useable, just not used by Dalvik (or Android).

Now to some more experimental notes. I've not written the scripts described above, entirely (especially the copy of the /data partition). Instead, I did this:

Code:
sleep 7200
In place of the external_boostscript.sh

The reason was simple: I was (and still am) experimenting, and if you're considering it, you'll want this feature too.

This sleep is in the external script, which means /system/etc/zinit.sh fired, mounted the external partition 2, but then slept for 2 hours.

Zygote, and therefor the Dalvik runtime...everything we think of as Android...is suspended from initializing. The phone may be showing a black screen, or a boot logo...but otherwise seems stuck.

Through adb, a shell allows me to manually arrange what I want for experiment.

For example, I manually copied the original /data directory to the new partition on the SD card.

I then manually unmounted /data (from /dev/block/mmcblk0p15, the original) and then mounted that duplicate from the external card over /data.

When I'm ready to try it out, I then examine the ps report (for those not familiar, this is a list of all processes). From that I find the pid of the sleep which is hanging there, then kill it. The suspense is over. Android to boots (because zygote is no longer blocked by the sleep), but now in my new, manually arranged configuration (a one time experiment). A reboot returns things to "normal".

Well.....normal after a 2 hour boot sleep. You'll want to use the shell to remove that script (this caught me twice).

Further, it is not absolutely required that you format the SD card to experiment.

If you've read this far you're probably going to try this at some point, even if you have to study a bit to figure out what each of these command lines are.

So...how, if the SD card's stock vfat is incompatible with the concept of servicing /data?

A virtual filesystem.

Here's how.

Make a directory on the external card....in the vfat system.

cd to it.

Now consider these notes.

Code:
dd if=/dev/zero of=filesys_data bs=32768 count=65536
This creates a 2 Gbyte file on the sdcard called filesys_data, filling it with zeros. Alter count such that 32768 * count = size you want.

This will be your virtual filesystem. It can evaporate at the mere "rm filesys_data".

I discovered that on the F6 the maximum is 2Gbytes. VFAT is supposed to be able to handle 4Gbyte files, but on the F6, they evaporate on reboot. It works as long as you don't reboot, but there's some kind of bug involved. 2Gbytes is the maximum that actually worked for me. At least that's a little larger than the 1.27 Gbyte source, the size of /data.

To use it, you need a node.

Code:
mknod /dev/block/filesys_data b 7 101
This creates a node for a block device named filesys_data (you may substitute at will). Google for more info if you need. Think of it as mounting a socket on a wall to plug things into.

Wire up the socket with losetup:

Code:
losetup /dev/block/filesys_data /storage/external_SD/whereveryouputit/filesys_data
Adjust to your choices...whatever directory you create to store filesys_data (or whatever name you preferred). Losetup wires the socket to the source, in this case the blank file you created.

Now, the socket refers to a blank fake disk, represented by the file filesys_data, in whereeveryouputit.

We essentially plug in a piece of tech gear to prepare it with:

Code:
mkfs.ext2 -LExtData -m 1 /dev/block/filesys_data
mksf.ext2 creates an ext2 filesystem (fully Linux compatible) using the socket (well...it's a node) /dev/block/filesys_data, which is wired to the blank file create whereeveryouputit/filesys_data.

You can use tune2fs to make it ext4 if you prefer.

It will be as large as the file you created ( bs * count ).

It can now be mounted as if it were a separate SD card, formatted as ext2 (or 4 if you chose).


Code:
mount -o remount,rw /
mkdir /storage/fake_data
mount -o rw -t ext2 /dev/block/filesys_data /storage/fake_data
mount -o remount,rw /
We must have a directory to mount over, and we have to make it since /storage is always "restored" on boot, and since /storage is on a read only filesystem from root, we have to remount /.

At this point /storage/fake_data is an ext2 (or ext4 if you made that) filesystem, hosted as a virtual disk created as a blank file on the SD card.

Note, if you used tunefs to make this ext4, your -t parameter will be ext4 not ext2.

This allows you to experiment WITHOUT having to reformat the SD card.

You can now make directories inside /storage/fake_data....like .remounts/data...with which to experiment. It's a Linux filesystem, so it understands identities and permissions that vfat does not.

Yes, you can copy all of /data into it and use THAT as the primary source for /data, as in the previous discussion. It will only be 2Gbytes, but it's larger than 1.2Gbytes and lets you prove this works before you commit to partitioning and re-formatting your SD card.

This is closely related to what happens when you use the "move to phone" feature.

Now for the rest of the story

I have this working, but there's a complaint....perhaps the last piece of the puzzle to solve.

The device boots fine, runs fine...but it complains that identies on the /data directory are not "synchronized"....I believe this is simply an unattended detail as of this post. I'll continue studying tonight. It's a minor flaw, but has to be fixed (figured out) before it can be fashioned into a solution for a wider public. I just ignore the warning - the phone and all software, new installs - etc - work fine.

This represents the state of the experiment as of Jan 24, 2014...it's a part time adventure.


The binary for app_process version 4.1.2_r1, like the phone, which works from 4.0.3 through 4.2.2 (tested)...should work on 4.4.2, but I'll verify later, is at this link:

http://www.ih-palladian.com/app_process_4_1_2_r1.gz

BTW, if you didn't recognize that you have to gunzip that before installing it, you may not be ready to try this.

If you read the above, you'll know what to do with it.

Please post with experimental results and observations.
JVene is offline  
Last edited by JVene; January 24th, 2014 at 11:15 AM.
Reply With Quote
The Following 5 Users Say Thank You to JVene For This Useful Post:
aaanadie (January 26th, 2014), iWindoze (February 26th, 2014), jjfad (January 24th, 2014), o4tuna (April 14th, 2014), WarrantyVoider (August 23rd, 2014)
Old January 26th, 2014, 10:08 AM   #17 (permalink)
Junior Member
 
Join Date: Jan 2011
Posts: 58
 
Device(s):
Carrier: Not Provided

Thanks: 56
Thanked 14 Times in 10 Posts
Default

This looks really promising. Thanks for keeping the thread updated. I look forward to testing a beta version.
Trevor Philips likes this.
ZipLipZ is offline  
Reply With Quote
Old January 26th, 2014, 06:45 PM   #18 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by ZipLipZ View Post
This looks really promising. Thanks for keeping the thread updated. I look forward to testing a beta version.
Thanks.

Sunday I manage to complete the study (fixed that last, one unattended detail). I now have all the materials in place to make a clean, fully functional install of this technique. Aside from customizing the ROM, this is as close to an ideal result as can be managed.

Now I'm going to prepare the "beta"....it may be an EARLY beta, but something most anyone with root access and ADB can handle.

That should be in place by tomorrow.

After I finally got this completely done, I did a hard reset on my phone, installed the solution (took about 10 minutes), then rebooted and proceeded to install all of the games and materials my son and wife wanted.

At this point I have about 9 Gbytes of software installed...around 40 applications.

Not once did I have to consider the storage issue.

My 1.27 Gbyte partition, which is the one that fills up for everyone, isn't even mounted right now.

The entire "Android" storage, the /data directory, is served from the external SD card from a 28 Gbyte partiion (I left 4 GBytes in vFat).

This suddenly got real.

Now I'll package for others to try
Trevor Philips likes this.
JVene is offline  
Reply With Quote
The Following 4 Users Say Thank You to JVene For This Useful Post:
infowarz (January 30th, 2014), jjfad (January 27th, 2014), o4tuna (April 14th, 2014), Trevor Philips (January 26th, 2014)
Old January 27th, 2014, 05:02 AM   #19 (permalink)
Junior Member
 
Join Date: Aug 2010
Posts: 88
 
Device(s):
Carrier: Not Provided

Thanks: 51
Thanked 9 Times in 7 Posts
Default

Quote:
Originally Posted by JVene View Post
Thanks.

Sunday I manage to complete the study (fixed that last, one unattended detail). I now have all the materials in place to make a clean, fully functional install of this technique. Aside from customizing the ROM, this is as close to an ideal result as can be managed.

Now I'm going to prepare the "beta"....it may be an EARLY beta, but something most anyone with root access and ADB can handle.

That should be in place by tomorrow.

After I finally got this completely done, I did a hard reset on my phone, installed the solution (took about 10 minutes), then rebooted and proceeded to install all of the games and materials my son and wife wanted.

At this point I have about 9 Gbytes of software installed...around 40 applications.

Not once did I have to consider the storage issue.

My 1.27 Gbyte partition, which is the one that fills up for everyone, isn't even mounted right now.

The entire "Android" storage, the /data directory, is served from the external SD card from a 28 Gbyte partiion (I left 4 GBytes in vFat).

This suddenly got real.

Now I'll package for others to try
SWEET...
jjfad is offline  
Reply With Quote
Old January 27th, 2014, 11:40 AM   #20 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

YESSS!!! You finally did it! Congrats!!!!!!! But then what happens to that 1.2gb of partition..... maybe can't you combine it with other internal partitions that you discovered?
Quote:
Originally Posted by jjfad View Post
SWEET...
lucasmun09 is offline  
Reply With Quote
sponsored links
Old January 27th, 2014, 03:27 PM   #21 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by lucasmun09 View Post
YESSS!!! You finally did it! Congrats!!!!!!! But then what happens to that 1.2gb of partition..... maybe can't you combine it with other internal partitions that you discovered?

The 1.2Gbyte partition, mmcblk0p15, must stay as a "stock" configuration. The phone must be able to boot normally if the external card is removed. Factory resets with data wipes will direct their operation to that partition.

It should be mountable as, say, /storage/internal_data, for review and even usage if there were really a reason, but I'll have to triple check it's availability for that.

Easily one could erase the content of /storage/internal_data/dalvik-cache (the original, not the new, /storage/ext_part2/external_data. Doing that is of zero consequence if booting to stock is desired, because the cache is rebuilt automatically when required.

The problem I see is that this partition must have enough space for the phone to operate if booted without the external card (or any other means of intentionally booting the stock configuration). Using it is hardly much benefit, but in my current state the partition has nearly 1 Gbyte available, because I did a hard reset before the "final" configuration of the phone. Put another way, if a hard reset sets the partition at 1 Gbyte free, like mine, the stock requirement is about 250 Mbytes. As long as that 250 Mbytes is intact, give or take the dalvik-cache content of perhaps 80 Mbytes, then that 1Gbyte should be available for user access.

I don't think I'd touch the other partitions, because they're devoted to boot images, system images and other materials - including the /system directory which is still linked from the external_data source exactly as the stock configuration.

I'm still dotting the "i"s and crossing the "t"s for a beta level installer. My current configuration was manually arranged according to the notes above with one simple final detail added.

I had to be certain the ownership of /storage/ext_part/external_data was system:system. When it was manually created it was root:root, and that caused Android to complain about that. Attending that detail caused the phone to boot as if nothing had changed.

It operates as if my son and wife had NOT objected to me soldering in a 32Gbyte replacement chip for the 4Gbyte internal one. Android has zero objection with the configuration.

Depending on the day's events, there's a good chance I'll have something posted tonight, but it could slip a day.

I'll need volunteers to test what looks like a 3 step process.
Trevor Philips and lucasmun09 like this.
JVene is offline  
Reply With Quote
The Following User Says Thank You to JVene For This Useful Post:
o4tuna (April 14th, 2014)
Old January 28th, 2014, 02:15 PM   #22 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

Hello! I don't know if you seen this thread before but I thought this may be very useful for future use. Its in the XDA Forums though..... Official bootloader unlock procedure from LG AND the one that works for everyone - xda-developers

The only usages that I know for unlocking bootloader are for custom roms and etc...... Maybe you can find more significant usage

Quote:
Originally Posted by JVene View Post
The 1.2Gbyte partition, mmcblk0p15, must stay as a "stock" configuration. The phone must be able to boot normally if the external card is removed. Factory resets with data wipes will direct their operation to that partition.

It should be mountable as, say, /storage/internal_data, for review and even usage if there were really a reason, but I'll have to triple check it's availability for that.

Easily one could erase the content of /storage/internal_data/dalvik-cache (the original, not the new, /storage/ext_part2/external_data. Doing that is of zero consequence if booting to stock is desired, because the cache is rebuilt automatically when required.

The problem I see is that this partition must have enough space for the phone to operate if booted without the external card (or any other means of intentionally booting the stock configuration). Using it is hardly much benefit, but in my current state the partition has nearly 1 Gbyte available, because I did a hard reset before the "final" configuration of the phone. Put another way, if a hard reset sets the partition at 1 Gbyte free, like mine, the stock requirement is about 250 Mbytes. As long as that 250 Mbytes is intact, give or take the dalvik-cache content of perhaps 80 Mbytes, then that 1Gbyte should be available for user access.

I don't think I'd touch the other partitions, because they're devoted to boot images, system images and other materials - including the /system directory which is still linked from the external_data source exactly as the stock configuration.

I'm still dotting the "i"s and crossing the "t"s for a beta level installer. My current configuration was manually arranged according to the notes above with one simple final detail added.

I had to be certain the ownership of /storage/ext_part/external_data was system:system. When it was manually created it was root:root, and that caused Android to complain about that. Attending that detail caused the phone to boot as if nothing had changed.

It operates as if my son and wife had NOT objected to me soldering in a 32Gbyte replacement chip for the 4Gbyte internal one. Android has zero objection with the configuration.

Depending on the day's events, there's a good chance I'll have something posted tonight, but it could slip a day.

I'll need volunteers to test what looks like a 3 step process.
lucasmun09 is offline  
Reply With Quote
Old January 28th, 2014, 02:56 PM   #23 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by lucasmun09 View Post
Hello! I don't know if you seen this thread before but I thought this may be very useful for future use. Its in the XDA Forums though..... Official bootloader unlock procedure from LG AND the one that works for everyone - xda-developers

The only usages that I know for unlocking bootloader are for custom roms and etc...... Maybe you can find more significant usage
Thanks for the link. I've found dozens of references for different approaches, and I had run across that one, too.

Unlocking the bootloader would be a wonderful alternative to the modified app_process approach I use here, but unfortunately the F6 was introduced after that thread was initiated, so it doesn't universally apply.

Loki can allow for booting a custom ROM, too, but like all such approaches it is dependent on specific versions and carriers. If the details are just right it does work. It exploits a vulnerability in the process for checking the signature during boot, interrupts it and loads any suitable ROM whether it passes the signature check or not. The problem is producing the image correctly. Loki happily generated an invalid image for my MetroPCS "i" ROM. Loki was expecting "e", but didn't complain (acted as though I had an "e").

There are ways to finagle Loki into working on most F6 devices. My concern is 4.4.

Loki has zero support for 4.4, and it could be a long while before it does, there's no way to know. Owners would have to be sure they block an automatic update, staying with 4.1.2 until Loki adds support, and then again it would be carrier/version specific ongoing.

The modified app_process approach used here doesn't have such issues. It will work on 4.4.2, and can easily apply to any version, any carrier and any brand Android device. This applies only to the technique of modifying the configuration during boot during that critical (and very short) time period where Android is not yet active, but Linux is up and running.

I would very much enjoy having an unlocked boot loader, but I would insist on that being a well supported, widely applicable solution independent of carrier and version. While the initial posts in that thread (and many others) promise a "universal" solution, that claim applied in early 2013. It doesn't anymore.

From a more general standpoint, implementing a lock defeat approach is a LOT more work than creating and installing the alternative app_process binary, both for developers and owners.

In all cases, once the ability to configure the operating system at the critical time synchronized point is established, the rest of what should be done is generally identical.

That particular detail is already proven and solid here. My current time for this project is devoted to packaging the installation process for the average but slightly technically aware owner of the device, with a design that allows for expansion to other models and potentially any brand and OS version from 4.0.3 forward, even if the bootloader remains locked or the owner wishes to avoid using such exploits.
JVene is offline  
Reply With Quote
The Following 3 Users Say Thank You to JVene For This Useful Post:
jjfad (January 28th, 2014), o4tuna (April 14th, 2014), Trevor Philips (January 31st, 2014)
Old February 1st, 2014, 11:55 AM   #24 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Well, i reached all my internal storage today... sign me up for testing
13onk, JVene and lucasmun09 like this.
Trevor Philips is online now  
Reply With Quote
Old February 3rd, 2014, 11:40 AM   #25 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by Trevor Philips View Post
Well, i reached all my internal storage today... sign me up for testing

Ok, let's deal with the pre-requisites, and I'll package some materials for you to automate the setup.

Let me know if all of this is already true for your device:

- rooted
- adb and USB drivers installed and working
- USB cable
- full backup of data on SD card, including any hidden directories/files
- full backup of all APK's (just in case).

The last two are related. If you've used "move to phone", the APK's are stored in hidden filesystems on the SD card. In the event you don't have those backed up, you may have no other recourse after this but to install those again (but, from ADB, that's trivial).

There is one thing to ponder. I've not experimented with doing this on a phone which has app2external_sd and/or link2sd (and it's cousins). I don't see this would be much of a problem, but here's what I do expect:

The "swap" of internal / external storage will no longer be important, and may even be "in the way" after this change. I chose a configuration with 4Gbytes for the vfat partition (which still looks like an external card to Android, just 4 Gbytes in size)...if your adjustments with these two approaches put more than, say, 8Gbytes of material on the external card that could create a kind of space squeeze. In my view, while any space increase over the 1.27G is an improvement, I can see that we all really need at least 16Gbytes on Android devices, save for the very few.

If you have a card reader on a PC, I do suggest you consider putting your external SD card on the PC for an image style backup using something like Acronis, EaseUS (which is free) or other software that can backup images without regard to disc layout, permissions or hidden directories - assuming you have material which may be valuable to you.

I think you can use AParted (free) from the app store to partition the SD card. I've not used it, but if you're not familiar with the command line fdisk in Linux, that may be a good approach. You MIGHT be able to find something that could shrink your existing vfat partition, leaving space for a new partition sized to become your primary storage, in a Linux filesystem.

If you find you can get all the way but not past the point of making a Linux filesystem on the second partition, that's fine.

I'll prepare some scripts to automate the setup and post no later than Wednesday (likely Tuesday night).

The scripts will assume there are two partitions on the external card. If you need help to get the partitions created let me know.

Here's what you can look forward to based on my few weeks on the F6 with this working:

My son wanted Need for Speed MW, Hot Pursuit AND Shift (he also wanted GTA and others, but they don't actually work well) - AND he's requested to try out Gangstar in 3 flavors. The three NFS have around 2 GBytes of data each when uncompressed. The download and install was effortless. Zero concern for where to put things. Android sees all of it, software and the (combined) 6 GBytes of data as residing on the primary storage (which it THINKS is the internal card - an internal card with 25+ Gbytes available).

It REALLY works as though I've replaced the Internal card with a much larger one.

By the end of the week I think we'll have around 16 Gbytes of application / game data on the device. That will leave me with only 10Gbytes available ( only ), but eventually I'm sure one of those games will be exhausted and removed.

I may look into rumors that 64 GByte cards work.
o4tuna and Trevor Philips like this.
JVene is offline  
Reply With Quote
The Following 3 Users Say Thank You to JVene For This Useful Post:
jjfad (February 3rd, 2014), o4tuna (April 14th, 2014), Trevor Philips (February 3rd, 2014)
Old February 4th, 2014, 06:37 PM   #26 (permalink)
Junior Member
 
Join Date: Jan 2014
Posts: 56
 
Device(s):
Carrier: Not Provided

Thanks: 2
Thanked 13 Times in 7 Posts
Default

I might sound a bit impatient and I know you are also a very busy person but I really do want to try out this mod.

When you said the gta would not work very well... How so? I know all of android's GTA apk's are not able to be moved to sdcard via app2sd function built in the F6.

Also lets say I have did this mod spectacularly but I "accidentally" pull out the SD card... What will happen then? Will the phone crash? Or will it just remount the old internal storage back to what it was mounted before?
Trevor Philips and JVene like this.
lucasmun09 is offline  
Reply With Quote
Old February 4th, 2014, 09:41 PM   #27 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

i'm just gonna reset my phone
Trevor Philips is online now  
Last edited by Trevor Philips; February 6th, 2014 at 06:10 PM.
Reply With Quote
Old February 4th, 2014, 11:33 PM   #28 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

I've had to reset about a dozen times along the way, but then I had REALLY ticked off Google's apps, so I really had to.

lucasman09 - I hear you. I had a math tutoring session tonight (my son is 12, probability isn't his thing yet), and I'm pushing to get a release build on a product I plan on putting on the app stores soon (it's still a few months).

Anyway, this is SUCH a significant improvement that I know it should be packaged and made publicly available. To that end I started putting together a Windows application for managing this on most devices (not just LG brand) - but, I'll pause and just package something simple that works on the F6.

GTA Vice city has a graphics problem. The F6 may throw it for a loop because of the GPU, the OS support for extensions or any number of other minor things GTA didn't account for. I think San Andreas can run on the F6 (I've heard that it does), so I'm going to try that one.

The NFS series had no problem, and I THOUGHT I had GTA Vice City running at one point, but I don't actually play it - my son informed me otherwise.

It wasn't a space issue in the least.

Now, when this storage solution is implemented the way I have it on my son's F6, the script which controls the mount of /data from the external card is actually run from the external card. Let me call that "external /data" - where primary storage for all of Android is from the external card. "Internal /data" would be the stock arrangement, where we have that 1.27 Gbyte partition serving all of Android.

There is a script on the INTERNAL card that starts things off, checks to see if it can mount partition 2 from the external, and if so, runs the second phase from the external card which mounts "external /data".

If the card isn't there, or another card is put in it's place, the first script will "realize" this and leave the older /data in place. That means the phone would boot with "internal /data".

There is an interesting consequence. Let's say you run the phone for weeks or months with the "external /data" mounted. During that time you load 10 apps, take 100 pictures...etc.

If you pull the external card and boot...none of those 10 apps will be there. They're on the "external /data"...only the apps installed at the time we apply this alteration will be on the "internal /data". Further, if you removed apps while the "external /data" was in place, which were also on the "internal /data"....they'll be back if you boot without the external card.

Put another way, it is like having two phones. When you boot with the "internal /data" it's like phone A..the original. When you boot with the "external /data" it's like phone B. Whatever you do to phone B doesn't affect phone A. Nor does what you do to phone A affect phone B.

You could ALSO clone the external card (using a PC and a card reader) using standard partition based backup or cloning software.

At which point you would have phones A, B and C - entirely separate from each other.

Meaning, also, you could clone an F6, and all it's installed applications (and it's contacts, email cache...everything)...to another phone.

I haven't tried it, but conceivably you'd HAVE to check that the personal info....like your own phone number...isn't cloned in the process (just fix it by editing).

I THINK (but not confirmed) that configuration information MAY be stored in a /system directory, not the "Android" /data directory. If that's true, then the configuration information would NOT be cloned by that process...I'd need to look, but it's minor.
Trevor Philips likes this.
JVene is offline  
Last edited by JVene; February 4th, 2014 at 11:35 PM.
Reply With Quote
The Following 2 Users Say Thank You to JVene For This Useful Post:
o4tuna (April 14th, 2014), Trevor Philips (February 5th, 2014)
Old February 5th, 2014, 10:25 PM   #29 (permalink)
Member
 
jokerpks's Avatar
 
Join Date: Apr 2013
Location: New York
Gender: Male
Posts: 331
 
Device(s): LG Optimus F6
Carrier: MetroPCS

Thanks: 149
Thanked 105 Times in 66 Posts
Default

Quote:
Originally Posted by JVene View Post
I may look into rumors that 64 GByte cards work.

Yup 64 GB cards work on this phone.

o4tuna and Trevor Philips like this.
jokerpks is offline  
Last edited by jokerpks; February 5th, 2014 at 10:32 PM.
Reply With Quote
Old February 5th, 2014, 11:37 PM   #30 (permalink)
Member
 
pressy4pie's Avatar
 
Join Date: Feb 2012
Location: /media/connor
Gender: Male
Posts: 358
 
Device(s):
Carrier: MetroPOS

Thanks: 136
Thanked 154 Times in 76 Posts
Default

https://github.com/konnorrigby/Xposed

this is the xposed version of app_process. should still work the same but enables xposed modules also.
JVene likes this.
pressy4pie is online now  
Last edited by pressy4pie; February 6th, 2014 at 12:17 AM.
Reply With Quote
sponsored links
Old February 5th, 2014, 11:45 PM   #31 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by JVene View Post
If you pull the external card and boot...none of those 10 apps will be there. They're on the "external /data"...only the apps installed at the time we apply this alteration will be on the "internal /data". Further, if you removed apps while the "external /data" was in place, which were also on the "internal /data"....they'll be back if you boot without the external card.
I personally find that pretty cool
o4tuna and JVene like this.
Trevor Philips is online now  
Reply With Quote
Old February 6th, 2014, 08:08 AM   #32 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by pressy4pie View Post
https://github.com/konnorrigby/Xposed

this is the xposed version of app_process. should still work the same but enables xposed modules also.
I had contacted the author of Xpose to persuade him to include that. He replied with interest in the subject but didn't appear prepared to include this.

I'm delighted to see that added.

I'll pull this in and append a link to the app_process binary built with this as an option for the installer scripts.

I'm working on the installer scripts - got quite a distance last night but alas it's not quite ready. Script debugging / verification is actually tougher than working in C++ because the environment isn't as conducive.

I'm getting there.
13onk and Trevor Philips like this.
JVene is offline  
Reply With Quote
The Following 4 Users Say Thank You to JVene For This Useful Post:
jjfad (February 13th, 2014), jokerpks (February 8th, 2014), o4tuna (April 14th, 2014), Trevor Philips (February 6th, 2014)
Old February 6th, 2014, 06:02 PM   #33 (permalink)
Member
 
Join Date: Feb 2012
Posts: 108
 
Device(s):
Carrier: Not Provided

Thanks: 9
Thanked 19 Times in 14 Posts
Default

I'd like to test this i meet the following requirements:
- rooted
- adb and USB drivers installed and working
- USB cable
- full backup of data on SD card, including any hidden directories/files
- full backup of all APK's (just in case).
please pm me with links/instructions
JVene likes this.
tony_duane is offline  
Reply With Quote
Old February 8th, 2014, 02:57 PM   #34 (permalink)
Member
 
jokerpks's Avatar
 
Join Date: Apr 2013
Location: New York
Gender: Male
Posts: 331
 
Device(s): LG Optimus F6
Carrier: MetroPCS

Thanks: 149
Thanked 105 Times in 66 Posts
Default

Quote:
Originally Posted by JVene View Post
I had contacted the author of Xpose to persuade him to include that. He replied with interest in the subject but didn't appear prepared to include this.

I'm delighted to see that added.

I'll pull this in and append a link to the app_process binary built with this as an option for the installer scripts.

I'm working on the installer scripts - got quite a distance last night but alas it's not quite ready. Script debugging / verification is actually tougher than working in C++ because the environment isn't as conducive.

I'm getting there.

When are you going to release the script? I don't mean to rush you or anything just curious. Thanks for your work I really appreciate it.
Trevor Philips and JVene like this.
jokerpks is offline  
Reply With Quote
Old February 8th, 2014, 03:17 PM   #35 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by jokerpks View Post
When are you going to release the script? I don't mean to rush you or anything just curious. Thanks for your work I really appreciate it.
Patience is a virtue
o4tuna, Kaat72, jokerpks and 1 others like this.
Trevor Philips is online now  
Reply With Quote
Old February 11th, 2014, 01:29 PM   #36 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by duckiebear View Post
I'd like to test this i meet the following requirements:
- rooted
- adb and USB drivers installed and working
- USB cable
- full backup of data on SD card, including any hidden directories/files
- full backup of all APK's (just in case).
please pm me with links/instructions
Quote:
Originally Posted by jokerpks View Post
When are you going to release the script? I don't mean to rush you or anything just curious. Thanks for your work I really appreciate it.
Quote:
Originally Posted by Trevor Philips View Post
Patience is a virtue
Thanks for the interest everyone.

I'm getting there.

Just a little more debug on the refinements.

For now I'm calling this thing "zinit" - that became the script name, a contraction of "zygote initialization", zygote being the name Android gives to the process of the Dalvik runtime, the Java engine of Android.

Bash (like) script programming isn't my cup of tea, but a necessary requirement of the technique. What started as a 5 line script in the experiments has become a 1500+ line script capable of dealing with a wide range of unexpected scenarios and configurations, opening the application of zinit to most Android devices.

I've banged up the F6 quite badly in the process (restored it several times). At one point I had a typical Android problem on the external installation, where FB was arguing with AVG over control of the contact database, soft booting and then soft bricking Android. I had to be sure if zinit was in any way responsible for the soft brick or the problems, and fortunately it wasn't.

However, this left the second partition on the F6 full of difficult filesystem errors which e2fsck (the ext2/ext4 file system check/repair tool) could not fix automatically. It required (as it stated to me) that I run the tool manually, to decide what to do with some files it had to fix.

In the process of this I found that zinit became a tool for recovery from the soft brick, and set about the task of implementing features which aid in the repair.

It appears many of us can't know when soft bricking the phone is because of an application gone awry, or from soft-reboots causing filesystem corruption which grows to the point that Android gets locked up trying to boot. We're locked out of running e2fsck manually during boot on a stock device with a locked bootloader. zinit now opens that ability up.

I was able to repair the soft-bricked installation of Android without resorting to a restore or hard reset.

An interesting side point is that zinit can operate without root installed. You still have to be rooted to install zinit, but then the su application could be removed and zinit will still operate. This is because after installation, the boot scripts executed by this approach are granted root privilege by init when it launches zygote by the very nature of Android's design, even if a root application is not present. This does not grant root privilege to anything else (it can't, because the root privilege is only operational during the boot), but it means those things zinit must be 'root' to do can be done even without root still installed. I've not verified that busybox dependency is entirely cleaned away from the script...but I'll let that slip for the moment to get this out the door.

The exception being manual file system checks - for that one must have root privilege, which could be a temporary installation, unless the owner put the external SD card into a PC with either Unix or Linux (even in a VM) performing the file system check.

There is also a feature to configure the boot datasource. Datasource is defined as the source of the /data directory. By setting a value, one can choose to boot the factory (default) way, or boot to any suitable datasource from the external card. I used this feature in a primitive form in my recovery and formalized it as a result. At one point I had 4 different "personalities" on the external card, one of them a copy of the factory version of /data, another a backup of my "son's" installation and backup of my "development" installation. Think of each as a separate installation of Android.

When my development installation was soft-bricked, I simply set the option to boot the factory backup, rebooted and operated normally. I then switched to my development backup installation, rebooted and operated normally. Then I booted the factory (default) way, performed file system check maintenance on the external card, restored only the missing files in the development installation from backup, set the configuration to boot the development system (previously soft-bricked), and upon reboot - everything was normal.

The entire series of events from the start of fixing the soft-brick to the final reboot into the previously soft-bricked development installation (through two reboots of two different personalities and the factory boot) took under 12 minutes.

That did require that I adjust the boot configuration from an ADB shell.

I have a plan to implement a feature in zinit that monitors boot cycles, to determin if Android is soft-bricked, so as to automatically fall back to other boot datasources in order to get the phone working again (it would take a few reboot attempts to trigger) - but that can wait a bit.

Ok, back to debugging....hitting the final stretch to a beta now.
aaanadie, bidzer, FireBean and 2 others like this.
JVene is offline  
Reply With Quote
The Following 2 Users Say Thank You to JVene For This Useful Post:
jjfad (February 11th, 2014), o4tuna (April 14th, 2014)
Old February 11th, 2014, 11:08 PM   #37 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by JVene View Post
Thanks for the interest everyone.

I'm getting there.

Just a little more debug on the refinements.

For now I'm calling this thing "zinit" - that became the script name, a contraction of "zygote initialization", zygote being the name Android gives to the process of the Dalvik runtime, the Java engine of Android.

Bash (like) script programming isn't my cup of tea, but a necessary requirement of the technique. What started as a 5 line script in the experiments has become a 1500+ line script capable of dealing with a wide range of unexpected scenarios and configurations, opening the application of zinit to most Android devices.

I've banged up the F6 quite badly in the process (restored it several times). At one point I had a typical Android problem on the external installation, where FB was arguing with AVG over control of the contact database, soft booting and then soft bricking Android. I had to be sure if zinit was in any way responsible for the soft brick or the problems, and fortunately it wasn't.

However, this left the second partition on the F6 full of difficult filesystem errors which e2fsck (the ext2/ext4 file system check/repair tool) could not fix automatically. It required (as it stated to me) that I run the tool manually, to decide what to do with some files it had to fix.

In the process of this I found that zinit became a tool for recovery from the soft brick, and set about the task of implementing features which aid in the repair.

It appears many of us can't know when soft bricking the phone is because of an application gone awry, or from soft-reboots causing filesystem corruption which grows to the point that Android gets locked up trying to boot. We're locked out of running e2fsck manually during boot on a stock device with a locked bootloader. zinit now opens that ability up.

I was able to repair the soft-bricked installation of Android without resorting to a restore or hard reset.

An interesting side point is that zinit can operate without root installed. You still have to be rooted to install zinit, but then the su application could be removed and zinit will still operate. This is because after installation, the boot scripts executed by this approach are granted root privilege by init when it launches zygote by the very nature of Android's design, even if a root application is not present. This does not grant root privilege to anything else (it can't, because the root privilege is only operational during the boot), but it means those things zinit must be 'root' to do can be done even without root still installed. I've not verified that busybox dependency is entirely cleaned away from the script...but I'll let that slip for the moment to get this out the door.

The exception being manual file system checks - for that one must have root privilege, which could be a temporary installation, unless the owner put the external SD card into a PC with either Unix or Linux (even in a VM) performing the file system check.

There is also a feature to configure the boot datasource. Datasource is defined as the source of the /data directory. By setting a value, one can choose to boot the factory (default) way, or boot to any suitable datasource from the external card. I used this feature in a primitive form in my recovery and formalized it as a result. At one point I had 4 different "personalities" on the external card, one of them a copy of the factory version of /data, another a backup of my "son's" installation and backup of my "development" installation. Think of each as a separate installation of Android.

When my development installation was soft-bricked, I simply set the option to boot the factory backup, rebooted and operated normally. I then switched to my development backup installation, rebooted and operated normally. Then I booted the factory (default) way, performed file system check maintenance on the external card, restored only the missing files in the development installation from backup, set the configuration to boot the development system (previously soft-bricked), and upon reboot - everything was normal.

The entire series of events from the start of fixing the soft-brick to the final reboot into the previously soft-bricked development installation (through two reboots of two different personalities and the factory boot) took under 12 minutes.

That did require that I adjust the boot configuration from an ADB shell.

I have a plan to implement a feature in zinit that monitors boot cycles, to determin if Android is soft-bricked, so as to automatically fall back to other boot datasources in order to get the phone working again (it would take a few reboot attempts to trigger) - but that can wait a bit.

Ok, back to debugging....hitting the final stretch to a beta now.
Thanks for taking your time on this JVene! Once you're done, This + overclocking = Badass F6
aaanadie, o4tuna and JVene like this.
Trevor Philips is online now  
Reply With Quote
Old February 12th, 2014, 06:53 AM   #38 (permalink)
Junior Member
Thread Author (OP)
 
Join Date: Jan 2014
Posts: 83
 
Device(s):
Carrier: Not Provided

Thanks: 3
Thanked 83 Times in 32 Posts
Default

Quote:
Originally Posted by Trevor Philips View Post
Thanks for taking your time on this JVene! Once you're done, This + overclocking = Badass F6
Overclocking!?

Hmmm....I shall have to give that some attention next!
o4tuna and Trevor Philips like this.
JVene is offline  
Reply With Quote
Old February 12th, 2014, 05:57 PM   #39 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by JVene View Post
Overclocking!?

Hmmm....I shall have to give that some attention next!
http://androidforums.com/metropcs-optimus-f6-all-things-root/827737-kernel-ms500-freedom-kernel-overclocked.html
Trevor Philips is online now  
Reply With Quote
Old February 13th, 2014, 03:12 PM   #40 (permalink)
Member
 
pressy4pie's Avatar
 
Join Date: Feb 2012
Location: /media/connor
Gender: Male
Posts: 358
 
Device(s):
Carrier: MetroPOS

Thanks: 136
Thanked 154 Times in 76 Posts
Default

Quote:
Originally Posted by JVene View Post
Overclocking!?

Hmmm....I shall have to give that some attention next!
GameTheory and I already have an overclock kernel out, but it is a boot.img so it needs loki. and some users have a difficult time using that because of the tmo/metro versions..
Trevor Philips likes this.
pressy4pie is online now  
Reply With Quote
sponsored links
Old February 19th, 2014, 11:05 PM   #41 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Hey Jvene, stopping by to ask how this is going
Trevor Philips is online now  
Reply With Quote
Old February 22nd, 2014, 09:30 PM   #42 (permalink)
Member
 
Join Date: Feb 2012
Posts: 108
 
Device(s):
Carrier: Not Provided

Thanks: 9
Thanked 19 Times in 14 Posts
Default

I can't wait till this is packaged and completed the anticipation is agony...
legacybeats likes this.
tony_duane is offline  
Reply With Quote
Old February 22nd, 2014, 10:03 PM   #43 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by duckiebear View Post
I can't wait till this is packaged and completed the anticipation is agony...
i feel you lol, i dont even feel like i can use the f6 without this
legacybeats likes this.
Trevor Philips is online now  
Reply With Quote
Old February 22nd, 2014, 10:31 PM   #44 (permalink)
New Member
 
Join Date: Feb 2014
Posts: 6
 
Device(s):
Carrier: Not Provided

Thanks: 0
Thanked 1 Time in 1 Post
Default ^ i agree ^

After researching, and much trial and error, it's apparent that all mods are essentially temporary to say the least...
legacybeats is offline  
Reply With Quote
Old February 22nd, 2014, 10:33 PM   #45 (permalink)
New Member
 
Join Date: Feb 2014
Posts: 6
 
Device(s):
Carrier: Not Provided

Thanks: 0
Thanked 1 Time in 1 Post
Default

Btw I'm new here, you guys have truly helped out allot, couldn't help but become a part of the community
legacybeats is offline  
Reply With Quote
The Following User Says Thank You to legacybeats For This Useful Post:
SUPPORTLOCAL (April 19th, 2014)
Old February 22nd, 2014, 11:03 PM   #46 (permalink)
Member
 
hubsabubs's Avatar
 
Join Date: Jan 2012
Location: Dallas, GA
Gender: Male
Posts: 255
 
Device(s): Huawei Ascend II, LG Optimus M, Coolpad Quattro, LG Optimus F6
Carrier: MetroPCS

Thanks: 28
Thanked 61 Times in 49 Posts
Default

Ummm, after the Kit Kat update, I assume this phone will then be back shelved and not slated for anymore updates, as that is a major update for us.

This is my solution, and I explained how I do this on a thread in the Coolpad Quattro 4G section because that phone had terrible internal memory. Also for some reason these phones being put out today are locked down and they are getting good at doing it. If you root the phone, good luck getting an OTA update to work properly after that. My Quattro is still OTA Update stuck and I still can't get the damn thing to go away.

My plan, wait patiently for the Kit Kat update, then backup my phone. Root my phone, then use Link2SD, and Mini Partition Wizard Tool to format a 32gb SD Card with a 16gb EXT2 partition and install Link2SD on my phone. It attaches itself to the 16gb partition and then to the 1.1gb internal memory and pairs them leaving me with 17gbs of internal storage and 16gb of external memory. Set a few settings and everything then after pairs with Link2SD automatically. In my opinion all these "mods" and "Band-Aid" fixes are unnecessary. To much work for such a simple solution.

Sidenote on something mentioned above. I also read that you can partition a 64Gb SDCard with a 32Gb Ext2 partition and a 32Gb Fat32 Partition and utilize both, the phone does support a 32Gb Card and being its the only Fat32 partition the phone thinks its the only memory available and leaves you with 33Gb internal memory via Link2SD as well.
hubsabubs is offline  
Reply With Quote
Old February 22nd, 2014, 11:04 PM   #47 (permalink)
Senior Member
 
Trevor Philips's Avatar
 
Join Date: Nov 2013
Location: Sandy Shores
Posts: 523
 
Device(s):
Carrier: Not Provided

Thanks: 138
Thanked 163 Times in 87 Posts
Default

Quote:
Originally Posted by legacybeats View Post
Btw I'm new here, you guys have truly helped out allot, couldn't help but become a part of the community
Welcome
legacybeats likes this.
Trevor Philips is online now  
Reply With Quote
The Following User Says Thank You to Trevor Philips For This Useful Post:
scary alien (February 23rd, 2014)
Old February 24th, 2014, 08:35 PM   #48 (permalink)
Junior Member
 
Join Date: Feb 2014
Location: United States
Gender: Male
Posts: 26
 
Device(s): LG Optimus F6
Carrier: MetroPCS

Thanks: 6
Thanked 1 Time in 1 Post
Default

Anything new on a solution or anything?
TehFlash is offline  
Reply With Quote
Old February 25th, 2014, 09:16 AM   #49 (permalink)
Member
 
hubsabubs's Avatar
 
Join Date: Jan 2012
Location: Dallas, GA
Gender: Male
Posts: 255
 
Device(s): Huawei Ascend II, LG Optimus M, Coolpad Quattro, LG Optimus F6
Carrier: MetroPCS

Thanks: 28
Thanked 61 Times in 49 Posts
Default

Any possibility on being able to partition a 16gb sdcard into 2 8gb fat32 partitions and use the internal/external swap method? Swap the internal with 1 of the 8gb partitions and still use the other 8gb partition as a external storage location?
hubsabubs is offline  
Reply With Quote
Old February 25th, 2014, 10:25 AM   #50 (permalink)
Member
 
Join Date: Feb 2012
Posts: 108
 
Device(s):
Carrier: Not Provided

Thanks: 9
Thanked 19 Times in 14 Posts
Default

I've got dual partitions on my sd card, I have links2sd and I have the swap installed. I still have an issue, because despite all the room /data gets full rather quickly even with plenty of space elsewhere. It is an incomplete solution
tony_duane is offline  
Reply With Quote
Reply

LG Optimus F6
Current Rating:
Rate this Phone:


Tags
external storage space app2sd link2sd sd card


Go Back   Android Forums > Android Phones > LG Optimus F6

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On



All times are GMT -5. The time now is 02:05 PM.
Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.