Maybe inasmuch as you fear someone and/or what they think of you (including society/humanity generally), theresomuch is it impossible to love them.
If that is so, then for your own mental health it is probably pragmatic to preemptively and unilaterally cease to fear anyone and anything.
Perhaps: If I have a self I can know and get in touch with to love people from, it is no memory or idea or conviction–only the feeling that I am or can be fundamentally okay, without problem, or even happy. Nothing else is truly me.
If fear drives out love, then to some extent maybe it is even irresponsible (to yourself and others) to let yourself become afraid of what another thinks of you.
Maybe that is our woe–trying to graft a system of control where others reserve the right to declare that another ought to hate or fear themself or feel shame and guilt about what they are . . . onto the fact that there is no love inasmuch as a person feels that way about themself, or fears punishment if they seem not to feel that way about themself.