Background: Despite the well-known role of parents as caregivers, few studies have addressed their health outcomes related to the Zika virus epidemic. Methods: A cross-sectional study was carried out with 146 primary caregivers of children 15–26 months of age, with laboratory and/or clinical evidence of Zika infection between August and October 2017 in three Brazilian municipalities: João Pessoa and Campina Grande in the state of Paraíba and Fortaleza in the state of Ceará. Caregivers reported on their child's life and health, family circumstances and underwent screening for stress using the Parenting Stress Index-Short Form. Children were evaluated for developmental delays and clinical outcomes. Differences in the prevalence of risk factors between caregivers with high or clinically relevant stress and those with normal stress were evaluated. Results: Of the 146 participants, 13% (n = 19) were classified as having high or clinically relevant stress, all of them mothers. The two risk factors significantly and independently associated with high levels of stress, compared with individuals with normal stress levels, were "reporting difficulty in covering basic expenses" (adjusted OR 3.6 (95% CI 1.1–11.8; p = 0.034)) and "having a child with sleep problems" (adjusted OR 10.4 (95% CI 1.3–81.7; p = 0.026)). Conclusions: Some factors seem to contribute significantly more than others to the level of stress experienced by caregivers of children with evidence of Zika virus congenital infection. Interventions and preventive strategies should also target caregivers, who in turn will be able to respond to the unique characteristics of their child.