Help Memory and task management

Brian706 said:

Just to be clear, with this device, you will lose your app data while rooting. Unless you are on an older hboot and can s-off only, you will have to do the HTC bootloader unlock. For security purposes, the bootloader unlock performs a factory reset. Without root there is no way to backup all of your app data.

Just didn't want you to be disappointed after root and wanted you to be prepared.

Click to expand...

+1

You can't back up everything with MyBackup Pro but it does a pretty good job with the essentials.

No worries there, I'm still 1.30 hboot, and can use revolutionary. Brian you helped me figure all that stuff out maybe a year ago; I got busy and never did get around to playing with it, but now looks like I have to.

My user storage is down to 10MB (over 250 should be free and is missing or lost), and I've moved most of the big stuff to SD. In case the 10MB disappears, I've been saving moving anymore apps to "hold" the space for when I need it. Sounds like it won't be an issue then.

In my reviewing of the root process I discovered that I might be able to access the stock recovery (boot into bootloader, select recovery then vol up+down to make the menu appear?) and wipe the cache partition. Might that help anything? Would I be wiping anything important?

K927A said:

No worries there, I'm still 1.30 hboot, and can use revolutionary. Brian you helped me figure all that stuff out maybe a year ago; I got busy and never did get around to playing with it, but now looks like I have to.

My user storage is down to 10MB (over 250 should be free and is missing or lost), and I've moved most of the big stuff to SD. In case the 10MB disappears, I've been saving moving anymore apps to "hold" the space for when I need it. Sounds like it won't be an issue then.

In my reviewing of the root process I discovered that I might be able to access the stock recovery (boot into bootloader, select recovery then vol up+down to make the menu appear?) and wipe the cache partition. Might that help anything? Would I be wiping anything important?

Click to expand...

That's the system cache, you can wipe that anytime, it simply gets rebuilt as you run.

Like the /system, the /cache partition is reserved for the operating system - it's not counted in any user storage reports that you've seen.

EarlyMon said:

That's the system cache, you can wipe that anytime, it simply gets rebuilt as you run.

Like the /system, the /cache partition is reserved for the operating system - it's not counted in any user storage reports that you've seen.

Click to expand...

That's right....I'm slowly starting to understand how the bits of android work.

I might try it anyhow, partly just out of curiousity, can't hurt!

K927A said:

That's right....I'm slowly starting to understand how the bits of android work.

I might try it anyhow, partly just out of curiousity, can't hurt!

Click to expand...

Exactly.

All a factory data reset does is erase the /cache, the Dalvik cache (table of association look ups) and the /data partition.

Its power is really in clearing out the cache and the Dalvik cache.

Absolutely a crime that that's so easy for rooted users and so hard for others.

I see. I thought a factory reset also refreshed/repaired the OS. So if the strange file dates were linked to a corrupted file in the /system partition a factory reset technically wouldn't fix that?

I'm guessing /system never gets written to on a stock locked device. I wonder if clearing /cache would clear up the corruption. Is there a good thread where I can read more about caches and what they do? Always wondered what dalvik was for exactly and why I should want to wipe it periodically

I'll look for some links for you - or you can trust me and go with the following:

Android = real-time embedded Linux operating system + Dalvik Virtual Machine + apps that run inside the Dalvik and call on system services from the underlying libraries with Linux

As soon as you open your dialer, it connects to your contacts. You can replace either and everything is still just magically connected - and in this example, also to your SMS, mail, even Skype if you installed that.

How do the apps know to do that?

They don't.

They let the Dalvik know what they are, if they're the default, and what intents are required to launch and connect them.

And the Dalvik is not going to stop and decide all of that every time you launch an app because that repetitious behavior would be a ridiculous waste of time.

So it builds and maintains a sort of database that shows how each of your apps are related and what their intents and defaults are - and uses a quick lookup from there.

Want to make a phone call? Not only is your dialer there, suddenly so is your preferred contacts and everything else.

And that "sort of database" is the Dalvik cache.

And it can and usually does get tangled up with OTA updates when the device suddenly gets a new set of system apps and no one told the Dalvik.

And yes, the /system partition is locked to read-only at all times.

The only ways to write there are 1) an OTA, or 2) root access along with a separate, specific user-permitted actions to (temporarily) set it to read-write.

NOTHING writes to the /system partition during normal ops besides the two cases I've mentioned. Not an opinion - fact.

As for the main cache - the Dalvik is what we call a stack engine. And it reduces its known repetitive actions to its cache, usually in the /cache partition.

Exactly in the manner that your desktop browser does and for the same reason - speed.

And usually each of the apps themselves have their own caches (not just the browser). And those are always in the proper place under the /data partition.

Again, factory data reset - erases /data and /cache partitions, and erases the Dalvik cache, whose location varies slightly from model to model.

The /system partition is never touched except for the two cases I've mentioned.

You clean out the cache, Dalvik and app caches on occasion exactly as you do for your desktop browser and in the same manner - things got tangled, you clear them.

As for the dates - that's not going to get fixed with cache cleaning. If it does, then it'll be the extraordinary exception.

A process screwed up time stamping.

The time stamps for your user storage (yet another set of partitions) have been changed and written that way.

Some may clear up through a reset but I wouldn't hold my breath. They'll likely clear up after a reset as they get used later.

Lol I only was asking for links so I didn't steer the thread way off topic. I appreciate the explanation. While I can't say I fully get it, I have a much better idea of the inner workings now, and certainly get what dalvik cache does.

So if i'm in astro, unrooted, and hit "up" until I get to the root folder, what am I looking at there? That's where I'm seeing the messed up dates. Is the /System folder I see the same as the /system partition or is it user storage? That's one of the folders with the error date, along with sbin, sdcard, and all the loose files in the root directory. There's also several applications (preinstalled system ones) showing a size of 0b (as viewed from the list of apps under settings); I imagine the two issues are related. I haven't checked thier sizes in astro. I suspect dialer storage may be a player as well, I noticed that the size fluctuates up and down though I haven't deleted any messages.

If a hard reset isn't going to clear up those issues then what is my next move? I'm curious to track down the root of the problem and properly fix it/avoid it in the future

Here's the filesystem layout for the Evo 4G LTE, a little different from your 3D but still, HTC's basic overall Android layout. (click to enlarge)



In this case, the system partition and the /system folder mean the same thing. Ignore any dates you see in there. Ignore zero sized files.

Ignore everything under /dev.

In fact, all you really might care about as far as deciding if things are corrupted would be under /data or /sdcard folders.

If timestamped dates have been corrupted, they've been corrupted. You're not going to fix them without backing up, reformatting, and restoring for the SD card and not at all for /data unless you're rooted, and then by going the same route.

Unless something time-sensitive is broken, I wouldn't sweat the timestamps a whole lot but that's just me.

Btw - your phone may never glitch again or it may too. Glitches cannot be prevented.

Not everything with a null timestamp was glitched.

I'm starting to get lost, to be honest.

Please tell me again what's broken when you use your phone, thanks.

/dev and /data both have normal timestamps. And with the exception of the files in the root directory, the error timestamps are only on folders, and the files and folders within show correct timestamps.

/sdcard or /mnt/sdcard both have the error, but sd card doesn't show any issues when plugged into the pc.

This all came about when 100mb of free space on internal storage disappeared, and I mean in an instant: My battery died, and when I finally got powered up again it said I was out of space. I managed to free up an additional 150mb, and a few days later I got the error again. I freed up 10 more mb not wanting to free up too much in case it disappeared agian, and so far that's been holding. I thought the corrupt dates might have to do with the vanishing memory.

More likely that the storage didn't disappear in an instant.

More likely that what you were seeing was an error condition clearing, or a stuck process completing, and the real storage being reported correctly. By the same token, your battery readout before dying - that could have been wrong as well.

Ignore the dates on the folders the way you're seeing them and - again - ignore /dev, nothing there for you to be concerned about, really.

I don't follow. What are you referring to when you say what I was seeing, the free space or lack thereof?

I'd had that ~100mb free for months, and it was definitely there before this happened, i.e. it wasn't slowly dwindling leading up to this. Everything was stable until that incident. Are you saying that free space was being incorrectly reported all that time?

My battery has been acting weird past few months, dying faster than usual, with the percentage dropping at unreal rates. Sometimes it'll go to 0% shut down then after only being plugged in a few minutes jump from a just few to 20 or 30%.

K927A said:

I don't follow. What are you referring to when you say what I was seeing, the free space or lack thereof?

I'd had that ~100mb free for months, and it was definitely there before this happened, i.e. it wasn't slowly dwindling leading up to this. Everything was stable until that incident. Are you saying that free space was being incorrectly reported all that time?

My battery has been acting weird past few months, dying faster than usual, with the percentage dropping at unreal rates. Sometimes it'll go to 0% shut down then after only being plugged in a few minutes jump from a just few to 20 or 30%.

Click to expand...

Where was the free space?

If it was associated with any cache storage, that could have been floating up and down, and that 100 MB meant nothing other than what it was when you were looking.

I can't go back in time and help verify or check that.

How old is your battery? How often has it gotten very warm and run down? It could be at its end.

The free space is internal app storage. I wouldn't know with what or if that gets shared . I've might have seen it fluctuate by maybe a few MB (or mayeb I was looking at processes/RAM) but 100mb fluctuations seems extreme for a device that only has 1gb of internal storage.

Looking at app storage using DiskUsage shows apps taking up 405mb and system data is 767mb now and seems to be growing. Any idea what that chunk repersents and if I can flush it?

Battery is original, I figured it was in its death throes. But I also think the phone is getting less efficient and that's running the battery down more. I'll find out when I do a reset and clean things up

The 3D has about 1 GB set aside for apps and app data, including their caches.

The radio firmware, bootloader and system use are elsewhere on the eMMC storage chip, and that all takes up about 3 GB - 2 GB of which is reserved for the /system, going by memory, I'll have to verify the final amount on /system.

The /system partition is reserved in size and is larger than any image planned during development.

It's read-only and doesn't change during use.

I'm sorry, I'll have to fire up my 3D to see what the Disk Utility and what it's trying to say.

EDIT -

1175 MB is available for /data

So, 405 + 767 = 1173 MB -> storage full, so that makes sense.

System data - data used by apps.

Saving a lot of Gmail by any chance?

Any new games? App updates?

Yes I understand all of that. The 767 part was smaller a few hours ago, giving me more room. Past few days my 10mb has dwindled down to 4. Occasionally it was dropping below two but a restart usually frees it up.

And nope gmail has had its data cleared and gone unused for a while now since it stops syncing under 120mb free. The exact value for that I'm not sure of. No new games or apps. No easy explanation that I can think of for the sudden loss of free space.

K927A said:

Yes I understand all of that. The 767 part was smaller a few hours ago, giving me more room. Past few days my 10mb has dwindled down to 4. Occasionally it was dropping below two but a restart usually frees it up.

And nope gmail has had its data cleared and gone unused for a while now since it stops syncing under 120mb free. The exact value for that I'm not sure of. No new games or apps. No easy explanation that I can think of for the sudden loss of free space.

Click to expand...

You asked what chunk system data represented and could you ditch it.

I did my best to answer. With a few examples.

But you understand all of that.

Your data is stored app by app under the /data/data folder for the most part, separated by each app and under each app, a hierarchy defined by each app developer.

The answer to whatever point you're trying to make or question you're trying to ask is there.

I'm sorry, for me this is going in circles and I just don't know what else to tell you.

Sorry, I meant understood generally how the 3d's onboard memory is divvied up. I guess there's no obvious answer, I'll just have to do a hard reset and go from there. I can't say I know how android works exactly but I can say I have an idea, and the loss of memory I experienced is definitely out of the ordinary, some sort of glitch. I was hoping to track down what exactly caused it because by nature I like to know what makes things tick. And it's nice when you can zap a problem right at its source rather than having to apply a blanket solution like a hard reset.
Based on what I've learned here, I hope that a hard reset will fix it even. I'm sorry I'm not able to explain the issue better, I understand at first glance it sounds like many a common problem. I did google before posting but as far as I can tell no one else has had a similar situation.

As a side note my /data folder is empty. I'm guessing I can't see anything without root. Lol it's like learning a new language, so frustrating at first when nothing makes sense then one day it all clicks and you get it.

Sorry for the frustration. I'll try what I can try and post back what happens.
And I thought smartphones were supposed to make life LESS complicated! Lol

Ok I think I've found where my memory is disappearing to. When viewing app storage via the DiskUsage app, I can see that my apps are consistently taking up 404mb, while the green portion labeled "system data" is growing, proportional to the space I'm losing.

Can anyone tell me what is represented by that "system data" block? I'd venture to guess it has to be something in the /data partition since it's part of the 1.1gb of appstorage.

Hope that makes sense

Just wanted to post that I may have found an answer. Apparently this is a somewhat common issue with some "application not responding" log file growing out of control and not being purged. This makes sense in my case, I have an app that becomes unresponsive several times a day for over a year now. I usually just hit wait, and it responds normally within a few seconds, but if it's been logging this every time that would explain the situation. I have yet to root and test this out. It's apparently only solveable with either root and targeted delete of the log, or a full on hard reset. Haven't found any info yet on future prevention.

Another update, hope it helps someone. Finally rooted and had a look at /data. I'm still watching it but anr_history.txt is at ~340mb and steadily growing by the day. So that seems to explain part of the disappearing memory.
I also dug into mms storage on a lead from another thread and found that all the pic attachments from mms are taking up a bit of space.

The other big chunk is data/radio/kernelinfo.txt
Can anyone tell me what that is/what it does? Google gave me nothing