Wow! That manager sounds like a total jerk. We have one of those at my AT&T too, unfortunately.
I have a white Note, and no issues of that sort at all.
I am sorry that you are having an issue with yours.
I agree that it doesn't sound fair that you would be charged a restocking fee for this issue, but unfortunately, that is the same kind of treatment I recieved from the AT&T where I purchased an Atrix that had issues, and there was NOTHING that I could do about it.
I wish you luck in resolving this issue.