Sex-contingent face aftereffects depend on perceptual category rather than structural encoding

Many studies have used visual adaptation paradigms to investigate how recent visual experience with faces shapes subsequent perception. While faces similar to those seen during adaptation phases are typically perceived as more 'normal' after adaptation, aftereffects are most pronounced when adapting and test faces are matched on dimensions such as sex, race or identity. Furthermore, it is possible to induce aftereffects in one direction for one category (e.g. female) and
simultaneously induce aftereffects in the opposite direction for another category (e.g. male). Such aftereffects could reflect 'category-contingent' adaptation of neurons selective for semantic class (e.g. male or female) or 'structure-contingent' adaptation of lower-level neurons coding the physical characteristics of different face patterns (e.g. including features related to sex discrimination).
We compared these explanations by testing for simultaneous opposite aftereffects following adaptation to (a) two groups of faces from distinct sex categories (male and female) or (b) two groups of faces from the same sex category (female and hyperfemale) where the structural differences between the female and hyperfemale groups were mathematically identical to those between male and female groups. We were able to induce opposite aftereffects following adaptation between sex categories but not after adaptation within a sex category. These findings indicate the involvement of neurons coding high-level aspects
of faces in sex-contingent face aftereffects and cannot be explained by neurons coding only the physical aspects of face patterns.