Yes, pressing pause will stop it.
If you want to actual close the application you will need to download a task manager from Market (personally I use TaskKillerPro)
From the task manager you can close apps.
The newest Android update, Donut 1.6, will feature a task manager of sorts built in to it. Don't ask "when will Donut come out on Hero as [you've] seen it on other Android phones" as no one knows that million dollar question yet. Best to follow HTC on Twitter for more info twitter.com/HTC