I'd definitely calibrate the battery after flashing a new ROM/Updating to a newer version of an existing ROM. I'd also recommend calibrating when you've restored a backup of a previous ROM.
Flashing a kernel, I'm not entirely sure. I updated a new one on Saturday and my battery life appears to be what I'd expect. You're probably best to play it by ear, if you think you're not getting the battery life you should be, then there's no harm in recalibrating.
Themes etc, nah, no point. Again though, if you notice any difference to what you were getting before, then there's no harm in going for it.
To calibrate properly, you'll need to download apps called
Current Widget and
Battery Calibration. Stick the Current Widget app on one of your homescreens and charge until it reads 0mAh (or as close as you can get it, I've waited ages on occasion and it won't go below 2mAh). Once done, open the battery calibration app, accept the Super User prompt (this may or may not appear) then hit Calibrate, then unplug your charger. Let your battery die and don't charge your phone again until it switches itself off. Charge fully again. Once done, your phone/battery is calibrated.
It's called Battery Calibration, but it's slightly misleading; it's nothing to do with the battery at all, more so setting a precedent for your phone. There's a file on your phones internal memory called Batterystats.bin, that holds information letting your phone know what a full charge is and what empty is. When you hit calibrate in the battery calibration app, this deletes the file and a new one is written when the phone dies/or is switched on again, I can't remember which
.
Does that all make sense?