The sort by size only takes into account the size of the application itself, not the cache, but in my experience the largest applications tend to have the largest cache. Also, take a look at any social media apps that you use (Facebook, twitter, etc.), particularly ones that you share photos / video with - those always have a lot of cache.
I used to use a cache cleaner app, but it gave me too many problems on the ICS leak builds before ICS was released for our phones so I stopped using it. I'll look around for a cache cleaning app now and see if I can find anything good.