I think I can safely say it was my Exchange mail. I removed the account, added back all of my widgets, and I've been texting and emailing (through Gmail) for a solid hour and my battery is still registering 100%. Before, it would have already been at 80% with it just laying idle. Much worse if I'd actually been using it.
What would cause exchange to do that? It never behaved that way until I rooted because I've always had this exchange account on every phone I've had and the only time I've had a usage problem is after I rooted.
Update - So now I'm even more confused. I restored back to the stock ROM (but still rooted) and my exchange runs just fine on there. It's not continuously downloading like it is on a custom ROM. What causes that?