I may tack this on to the OP and for now it's a work in progress:
Troubleshooting phone slowness.
Based on the OP I can assure you that memory is the least likely cause of slowness for your phone. The huge letters reminding you that you are not running a Windows Mobile device should say enough. The OP goes through some of the technical details of Linux, and therefore Android, memory management but it doesn't talk about the things that can cause your phone to slow down. Maybe I was a little myopic in the OP so here is an addendum which I may add to the OP later that talks about why your may have slowness on your Hero and how to troubleshoot the issue. I have tried to aim this towards the non-rooted phone but the same steps apply to rooted phones as well.
Things to remember:
1. The phone is always slow when booted because it's processing every single thing in startup. You can't edit this so you have to wait until everything has started, updated via the intertubes and so on. You can mitigate this by having fewer widgets that access the intertubes for information because when you boot all of them want updates.
2. The number of apps you have showing up in your task manager doesn't matter one single bit. You may think it does but it doesn't. Get this out of your head before you start troubleshooting.
Troubleshooting slowness on your phone:
First we have to understand why the phone could be slow. Since we aren't in the Windows world then memory is not the first place we look. It could be memory but chances are good that it is not so we won't even look at a task manager during troubleshooting.
I assume you have had your Hero for a bit and installed a bunch of cool crap from the Marketplace mainly because that's what I did and everyone I know did. If you are reading this before you have installed anything then you are in luck because I can give you the best advice in the world. Don't install more than one thing at a time! If you already have a ton of apps and can bring yourself to do it then factory restore your phone. Most of you won't like this idea so we'll work through troubleshooting without it. But if you fall into the category of not having apps coming out of your ears already jsut install one thing at a time, let that one thing run for a bit, and see how it affects your phone. If your phone was fine and last night you installed Panty Raid for Android (a funny name for a fake app) then this morning your phone is slow don't go get a task manager! Just uninstall Panty Raid and reboot your phone and the problem SHOULD be gone. If you install all of your apps like this you will end up with a core set of apps you know don't screw with your phone. Do updates the same way and you should never have to troubleshoot much of anything.
So what actually makes an Android phone slow? I have found two major culprits, well actually one but it makes sense to address them as two, and those are CPU and Data. If an application is hogging the CPU then no other process can get to the CPU and they all sit there in the queue waiting to be processed and making the phone slow. Imagine a line for a roller coaster, a one seat roller coaster, and on the coaster is a bully who won't let anyone else on until he feels sick then one or two people can slip on but as soon as he feels better he's right back on the coaster. The line would move slower than molasses in January! That's pretty much how the system processes things. Normally apps are nice (inside joke for the geeks) and there aren't any issues because they all share processor time back and forth. But if an app gets out of control then it's eating CPU cycles and not leaving any for anyone else.
Data is sort of the same way. The pathway across which the system transfers data is not very big. I won't go into technical details here because they are boring but transferring data involves system calls across a small pathway and having the CPU process the bits. Normally this isn't a big deal since apps don't deal with large data chunks on a phone however apps that manage things like mp3 libraries, hi-res wallpaper and things like that can cause slowness. This is especially true if they scan the SD Card for changes which, at that point, is utilizing two different data pathways and the CPU. Toss in networking with this app and you have an app that can make your phone a sloth. You add in the networking stack, regardless of whether or not it's REV1, REV0, or WiFi and your just adding more CPU time and other pathways into the mix. So even with a well coded app anytime you are moving around data, regardless of the source of the data (SD, WiFi, and so on) your phone will slow down noticeably but this should be a temporary slow down. Notably if you get a lot of email and check your email every 15 minutes and have 3 accounts you could almost cripple your phone.
So with those explanations out of the way, if you are still with me, let's start troubleshooting phone slowness.
Go download spare parts from the Market. Spare parts is your friend. It won't help you right away as you need to let it gather data but it will help find bad apps. If you are the patient type install spare parts, reboot your phone, and come back in a couple of hours.
- - Check your awake time: Main screen -> Menu -> Settings -> About Phone -> Status: If your awake time is high then you likely have an app that's not behaving properly. Now let's figure out what app that is.
- - Load up spare parts and see if any apps are keeping your phone awake. (If you aren't the patient type skip to step two). Check the following:
- - Battery History -> CPU Usage: If you see an app you aren't "using" a lot with high CPU usage you may have found your culprit.
- - Battery History -> Partial Wake Usage: Nothing should be high here at all. Partial wake is like when the weather app updates and then goes back to sleep. If you have high partial wake from an app it's doing a lot while you are not using it.
- - Widgets: Turn off any widgets you don't need. Widgets update and keep caches. If you have a dozen widgets that all update on a regular basis then you will see slowness if more than a couple of them try to update at once. See the explanation above on data being the cause of a slowdown. Uninstall any widgets you have downloaded and are not using.
- - Applications: If none of the above has led you to a conclusion and your phone is still slow then it's time to start looking at apps. The only real way to do this is one at a time. Don't update your apps before you start this because you may introduce new issues. This is a painful but easy process. Every app you installed needs to come off your phone one at a time. Uninstall the app, reboot the phone, let it run for a while and if the slowness comes back move on to the next app. Do not re-install apps between these tests. When you get to a point that your phone isn't slowing down anymore then you can start adding apps back in. Do this one at a time and give it enough time to actually show problems before installing the next application or you will never find the problem you are looking for. This is a tedious process but your patience will pay off.
- - Task Manager: As a last resort you can grab a task manager and kill processes to see what's slowing up your phone. Don't kill everything or you'll not find the source of the problem. Don't pay attention to the amount of free RAM because it's truly not relevant. Kill your running apps one at a time and see if your phone gets faster. You may notice some pop back into the list. Don't kill them again but do give them a few a seconds to actually load up and not be using the resources it takes to start an app (see the reason above regarding your phone being slow on startup).
Without root that's about all the steps I can think of to track down slowness issues. Remember in your own troubleshooting that RAM isn't slowing down your phone or affecting your battery. It costs the same amount of battery to write to a memory space with data already in it as it does to one that's empty. In fact erasing stuff from memory takes battery power so it would drain your battery more to "empty" memory space when it's not necessary. What you are looking for is what is hogging your CPU, Network, GPS and so on. Ignore the free memory number altogether because it doesn't have the meaning that it does on Windows systems.
Please feel free to ask me questions. I want to tweak this before I add it to the OP. So hit me up. I can't promise a fast response but I will respond! And thanks to Tisnatch for asking. it made me realize how much tunnel vision I had in writing the initial diatribe.