Some apps can be closed by pressing back, although some needs to be forced via task mannager before fully closing.

Sent From My HTC Evo 3D Using TapaTalk 2

 

Here's a great explanation from "mrgibbage" on another forum:

"There is a misconception here that I will try to clear up. The notion that a program is running in the background, or at the same time, like windows allows, simply does not exist in Android. Yes, there are apps that can load themselves as services, but for all practical purposes, only one program is running at any time in Android, and that's the program in the foreground. Yes, Android keeps track of the last six apps that you ran so it can start them back up again, but all it does is store a very small stub in its memory. This memory was already set aside for this purpose, so even if you could clear it out, it wouldn't free up anything for you to use.
Like I said, there are programs that can run as services, and as such, they can run in the background and therefore they are running all the time. Android is very careful to not allow such processes take up very much of the cpu total cycle. If you are truly having a problem where your phone is getting sluggish (although I highly suspect this is your imagination and misconception after using windows for so many years where this truly is a reality), then I would systematically try to figure out what app is causing the problem and uninstall it."

So to specifically answer your question, I don't believe that the app will remove itself from the task manager unless you swipe it away via the task manager itself.

 

download application monitor app its just what your looking for.

 

Hold down the home button then swipe the open apps to the left, that seems to do the trick.