Loads of folk keep their phone stock and have no issues, so if you decide to not bother with any 3rd Party apps, I'm sure you'll be fine.
If you're in to taking pictures, the stock HTC Camera is probably about the best you can get. It takes a decent picture and it's pretty fast (burst mode is second to none).
The stock text app is fine, it's pretty basic, but does what it needs to. I'd steer clear of GoSMS as some people report ads in their notification bar when using Go Products, but look at Handcent, Pansi or Chomp as alternatives.
I just use gmail for mail, so can't really comment on the stock mail app, sorry. I have used K9 Mail on previous handsets and found it excellent so you could have a look at that if HTC's offering doesn't do what you need it to.
As for screwing stuff up by installing the wrong apps, just limit your installs to stuff on Google Play and check out app ratings and feedback and you should be ok. Be sure to check out app permissions to, make sure there's nothing untoward there. If you fancy a bit of lengthy reading on app permissions, take a look here: Android permissions explained, security tips, and avoiding malware
If you get stuck, by all means, don't be afraid to ask us questions.