This study examined the latent structure of spontaneous social attention in 11- to 26-month-olds with autism spectrum disorder (ASD, n = 90) and typically developing (n = 79) controls. Application of the joint and individual variance explained decomposition technique revealed that attention was driven by a condition-independent tuning into the dynamic social scenes construct and context-specific constructs capturing selection of the most relevant social features for processing. Gaze behavior in ASD is characterized by a limited tuning into the social scenes and by a selection of atypical targets for processing. While the former may be due to early disruption of the reward circuitry leading to limited appreciation of the behavioral relevance of social information, the latter may represent secondary deficits reflecting limited knowledge about social partners.