When Brock ended up in the hospital Jesse realized that the ricin cigarette was missing. His first assumption was that Walter had Saul take it from him. Walter convinced Jesse that it wasn't him and that Gus must have had someone at the laundry take it from Jesse (stating "Who do we know that is OK with hurting children? Gus."). When yule is able to take Jesse's weed without him realizing it, it puts all the pieces of the puzzle together for Jesse. He realizes that Walter tricked him into helping kill Gus, that Walter must have had Saul and yule take the ricin cigarette, that Walter was there when they found the missing 'ricin' cigarette (it was actually table salt in a cigarette that Walter brought to Jesse's house and planted inside the vacuum so Jesse would 'find it' and relax about the whole situation). Jesse finally realizes that Walter doesn't care about him at all, and that when it comes right down to it, Walter cares about himself and only himself, and his family, and Jesse's just a piece he can manipulate like everyone else.
The irony is that Walter won't even entertain the idea of getting rid of Jesse, showing that even after everything, Walter really does care about Jesse.