It's up to you, I just wouldnt ignore that advice personally. It's just a fact that things they visualize on a SIS can prevent an embryo from attaching. My clinics protocol is to do a hysteroscopy after a live birth regardless of whether they see something or not. They did visualize what they thought could be scar tissue after my csection and it ended up not being that but was just "fuzzy stuff" made of polyp like material but I'm glad I had it removed to give my embryo the best chance implanting.
I had one prior to my first FET and the embryo didn’t implant, it was advised that I do it so I have the best outcome. Just in my situation it didn’t work, still I would trust the doctors opinion
You should ask your provider about the risks of not doing the procedure. There's a chance they might want you to get it done to understand your risks. Maybe they are worried that the scarring/damage could make a subsequent pregnancy dangerous. In that case you might want to do it. BUT if thr provider isn't worried about something like that, then that's a different story.