Human brains have definitely evolved to allow face recognition and distinction from infancy. There are people with "face blindness," prosopagnosia and they have difficulty distinguishing characteristics of their own face. I have heard someone with face blindness describe distinguishing between two faces similar to distinguishing between two similar river rocks.

Interestingly, and perhaps as clue to your first suggestion, experimentation implies crows can recognize and remember human faces over long durations of time. http://www.sciencedirect.com/science/article/pii/S0003347209...