There’s no morality involved in your visceral reaction. We evolved under very exigent circumstances in which poor friendship and mating decisions could easily cost us everything. Hence we have quite well-developed hardwiring for determining fitness, and the face is the most cue-laden aspect of ourselves that we present to the world. A deformed (e.g. not standard) face screams either environmental or genetic problems (or both) and hence our reaction to it is one of aversion, just as we find certain smells unpleasant so as to ensure we don’t consume things that would harm us.
As is so very often the case, once we understand the biological underpinnings of our behaviors we can gain not only far greater insight but we can also stop meandering around intellectual dead-ends and performing unnecessary acts of emotional self-flagellation.