If he’s still showing you affections then the better would be why are you annoyed by it? You want the affections yet you push them away. What you need to do is talk to him about it. Postpartum can affects sex drive because we may feel that our body has changed and the birth could’ve been traumatic for you. If you love him then You need to show him your love and affections too. Don’t think that “why does it have to be me first” - when you love someone tell and show, how else would they know? Maybe he’s thinking you’re not ready to have sex as normal yet. Our body actually takes up to 2 years to go back to normal”normal” after birth. And it’s normal to not feel sexy or want sex as much because you’re both parents now. The relationship is going to be different compared to when it was only just 2 of you. Start talking and ask him about all these worries. If you love then you trust and don’t jump to ungrounded conclusions.
Honestly once the trust is broken it’s really hard to feel safe to do it. Those messages changed the dynamics. And if he did it once he will do it again when he feels things are not where he expects it to go. So what you are feeling is normal. And it’s not about you it’s about him- what can he do to rebuild the trust he broke. Hugs and kisses isn’t how you regain trust that’s just him trying to get what he wants.