There's no need to be mad at the guy (or his ex) about this, but I wouldn't recommend sticking it out. This is going to negatively affect your life in a number of ways.
First of all, get ready to be his second priority, at best. This is an inherent imbalance in the relationship, and it's permanent. Worse yet, the relationship didn't start out this way, so you're really going to feel the difference. He will presumably be your first priority, and you'll always be a distant second to him. I've been there, and it is not a good feeling. And it's not as though you can get upset with him for prioritizing his son or daughter.
Secondly, that ex is in his life forever, which means she's in your life forever. Even if she's sweet and unintrusive, she is going to get under your skin in one way or another. You can't imagine what it's like to watch your partner and his ex have conversations about schools, or doctor's appointments, or parenting decisions right in front of you like you're not there. And get ready for a lot of uncomfortable talks with your partner about the level of involvement you're allowed or not allowed to have with a child that will occasionally live in your house.
Child support is a whole other discussion. It's going to happen, and even in the best of circumstances, it sucks. It's going to hurt him financially, and will hurt both of you if the two of you move in together. You'll find yourself scrutinizing every purchase the ex makes, and looking over the kid with a microscope to try to judge how much of that child support has actually gone toward his/her wellbeing. Spoiler alert: it won't be all of it, probably by a long shot.
God help you if the two of you later have children together. You'll quickly find that many of the parenting decisions have already been made for you, because that's how the last kid was raised, and therefore your husband has more experience, or things need to be fair, or whatever. And as big as your heart might be, you will of course love one of your husband's children--yours--far more than the other. And since that child will live with the two of you and the other won't (until the "I want to live with Dad" phase, anyway), chances are good that he'll love one of them more also. Yeah. That doesn't get talked about, but it's true.
You didn't sign up for this, and you don't have anything to prove. Don't waste your life by trying to show the world what a cool, understanding partner you are. Your boyfriend probably isn't a bad guy, but he is definitely not worth the level of stress that a baby is going to bring into the relationship. Find someone better matched to your lifestyle, and don't get sucked into somebody else's family drama. Going your separate ways is the right decision for both of you: you don't want to be dragged down by baby drama, and he doesn't need to tiptoe around relationship issues while he's trying to figure out fatherhood.