Fleming was reported to be unhappy with the casting of Connery, and had actually wanted Roger Moore in the first place. But I understand that this was largely because Fleming was a dreadful snob and thought Connery not enough of a "gentleman" for the role.
Personally I generally preferred Connery's films over Moore's for precisely the reasons that Bobby prefers Moore's: fewer gadgets, though they became more prevelant with time, and less funny. No disrespect to Roger, the series was evolving that way before he took over the role, and he did what the producers were looking for.
As Bond, I find Craig most convincing, and least "fantastic". In part because they do allow him some weakness in places, and to show that there is more behind the facade, in part because his physicality and taciturn manner make him more convincing as a cold-hearted killer, which if you are honest is what Bond is. Because I'm not a fan of camp I also prefer early Connery to later.
As for who is the most authentic Bond, in the sense of truest to the character in the books, I'd say probably Craig in the first act of Casino Royale, before he meets Vesper Lynd. The Bond of the books may be an expensively-educated "gentleman hero", but he is also a bastard with a sizeable streak of cruelty, and I always felt that any screen presentation of him had to be sanitised since a straight portrayal would be too unsympathetic.
I've actually never seen either of Dalton's films, and though I've seen parts of some of Brosnan's films I'm not sure I've ever watched one all the way through (too reliant on set-pieces and gadgets for my tastes).
I'm going to be controversial and say that I think that the consensus is too harsh on Lazenby. Coming straight after Connery, who was not only a popular and charismatic Bond but the first one, was always going to be a tough gig, and while he was a bit unthreatening for my tastes (as noted above, Bond is really quite an unpleasant character, and I prefer at least some of that to be present, even if only emerging at times) I do sort of wish he'd had a second chance, just to see whether he could develop the role. Of course I like the fact that they played that film quite "by the book" with no gadgets, and the ending, rather than some tacked-on "Bond saves the world then snuggles down with the girl as we cut to the theme music", was straight out of the book and IMO the most powerful ending of any Bond film (even if it may not have gone down well with audiences who were expecting an adventure romp). The fact that it faded to Louis Armstrong is also a massive point in its favour.
So overall, I'm inclined to Craig over Connery, then Moore, then Lazenby (with the caveat that there was only one film to judge on, and I preferred it in some ways to many of the others), then Brosnan, and no opinion on Dalton.