It only matters when someone wants to make their dislike of you your problem.

I competed in roller derby with a guy who told me to my face he didn’t like me, but he was respectful and never withheld coaching, scrimmaged with me and was a good teammate, and we otherwise just stayed out of each others’ space. We just didn’t have personalities that meshed.

It was no big deal.