Am I being unreasonable?!

I don’t know If I am letting my emotions get the best of me in this situation so I would really appreciate some thoughts/advice. My in-laws live 1 hour 30mins away, the last time we drove up to them, my LG who was 10weeks at the time. Was sooo bad especially on the way back, she cried every time I put her in the car seat. We had to keep stopping and ended up getting home at 10pm. Also while we were at my in-laws, no one was letting her go down for a nap, kept getting told to put her down. Essentially she was overstimulated. I was so angry at myself because I knew it was a bad idea taking her and I was dreading it leading up, but I did it because my husband wanted to go. Now it is my husbands birthday next week and he suggested going, my heart sank because I am so traumatised by last time. Yesterday I heard his mum say to come on the 2nd and he said - yeah I’ll see. Immediately I got angry and said no we can’t go, she can’t cope with the journey, so he said, what if I tell them to come here. BUT he won’t be home til after 5pm. So I was like you’re not even going to be here. I asked why he didn’t explain to them that she struggles, so they are aware instead of them thinking that it’s me who doesn’t want to take her. My husband thinks I’m being selfish because I don’t want to do either plan. Tbh I hate being around them, I just feel awkward and uncomfortable. My husband and I had an argument about this and I ended up saying, I wish she never had to go there but that can’t happen. Which he thought was hurtful. I can’t apologise because I am so angry, why should we put our baby through that journey but also why is he inviting them on a day when he’s not even here. Why is he calling me selfish when he can’t see that his family is awkward and not nice for me to be around. He thinks I am just trying to keep my daughter away from his family because of how I feel about them. Now we are not speaking. Am I just being a bitch and selfish? How do I resolve this? Thank you xx