The carriers all put their own UIs and extras on top of Vanilla Android, plus each phone needs its own drivers(supplied by the manufacturers). This all need extensive integration testing to emsure that nothing is broken. Then there's the statutory certifications in some contries (probably different in each case)
So, you're pretty much correct in your suppositions. All of which is why nexus kit always gets updated forst.