my husband tries to be good to me, but has told me that he "never learned how to be thoughtful and considerate". after being together for 20 years i thought he might have learned by now.
this is pretty significant in your relationship with your husband. has he tried to "learn" thoughtful and considerate now? to say that he never learned it and to know that he needs to would seem that he would seek help in figuring it out. for example, to say that i'm an alcoholic would be the first step, but if i never go to AA or seek help, then my words are nothing. i'm still an alcholic and nobody would accept it as just that.
20 years is a long time to be with someone, so i totally get why leaving would be such a mountain. i agree with what darkrainclouds said. everyone will get through this - even the children - and in the end, things will seem right. i think what holds you back is the idea of "affair" as your reason for leaving.
think of it this way. pretend there is no other man - no affair. he's gone...he didn't exist. ok...now would you consider a divorce? if the answer is still yes, then you know you are thinking with the reality of your marriage. if your answer is no, then you are thinking in terms of someone else "rescuing" you, just as your husband did in the first place.
you have to be willing to be alone - no lover and no husband - to make sure that you are making the right decision. it is great that you are so considerate of all involved. it shows that you truly respect and love your husband in spite of his lack of affections.
good luck with it. no use beating yourself up, but stay focused