An idle task will eventually be removed from memory by the OS... more quickly if another active app needs the resources. While it is sitting there idle, it does no harm. This seems to be something that people with history in DOS and Windows don't easily grasp.
However, it is bizarre that there is no way to cleanly exit the app. I would suggest finding another browser that acts the way that you want it to act.