My New BF wants anal, but I think he's to big

So, little background. I married my Jr High Sweet heart and we got married while I was in the military. I lived overseas with a room mate while she went to school. The night before my room mate got out of the military we got drunk ended up giving each other oral. I really enjoyed it and thought about it alot. Move forward couple years, I had a child with my wife and started having desires with a guy. I came out to my wife and in the end, we divorced but are best friends. My first real BF was great for over a year. We had sex all the time, I was the bottom in the relationship. Well, I got him cheating and we broke up. I recently started dating again and became really attracted to this really nice, sweet black man. My ex wife actually helped us get togather. We hit if off really well and been dating a month now. Last night we were going to do the deed for the first time and I Really wanted to do it. As we took off our clothes and started messing around.. I became scarred that it would really hurt. My last BF was smaller and that took a while to get used to. He knew that I wanted it, but I made a lame excuse and ended up only doing oral. Finally the question, how do I prepare myself for him? I don't think I can take him. I'm nervous. I really like him, but if I can't make love to him it wouldn't be fair to be for either of us. Any help would be appreciated. Sorry for such a long question.