I agree with the pp that suggested that you need a long talk with your husband. If you are wanting to lose weight to feel better and destress (through exercise for example) that is one thing, but is it just because he is telling you so?
If you want to work out on your own, can't your husband watch LO for a half hour at least while you walk around the block or something? He can come home at 7:30 instead every other day and cut short his own workout. Or you both can come up with a suitable arrangement for that. If he is not willing to give you that time, then you can tell him there's nothing you can do more than you are already doing and he needs to accept it instead of pressuring you.
Alternatively, if you are not ready yet to devote a considerable amount of personal will power and energy to losing weight, you could tell him you will do so, but when the baby has established a more regular routine--prob. around 6 months of age or more.
I am biased here. My experience was that my dh was incredibly inconsiderate, selfish and distant to me and my LO for the first 3 months of her life and he didn't cut it out until I sat down and told him he needed to change and roll with our new life. He had to give things up too and things couldn't be just what he wanted. It was very difficult, but we got through it and now he is a wonderful considerate father. good luck to you