<bold>Objectives: </bold>Family caregivers (FCs) face a variety of demands while caring for persons with Alzheimer disease (AD). Longitudinal studies identifying the specific AD-related neuropsychiatric symptoms (NPS) that contribute to FC distress are rare. We analyzed which NPS in association with care recipient and caregiver demographic factors are associated with FC psychological distress over a 36-month follow-up period.<bold>Design: </bold>This is a longitudinal study with annual follow-up. Participants were people with AD (n = 226) and their FCs (n = 226).<bold>Measurements: </bold>The Neuropsychiatric Inventory was used to assess NPS, and The General Health Questionnaire was used as a measure of caregiver distress. The effect of NPS on FC psychological distress over time was analyzed using a linear-mixed effect model.<bold>Results: </bold>Delusions (P = .003), agitation (P < .001), and sleep disturbance (P = .005) are associated with FC psychological distress. One of four people with AD developed delusions and agitation during the early stages of the disease. Sleep disturbances increased over the follow-up time. A marital relationship was associated with FC distress, while some prevalent symptoms, such as depression, did not affect distress.<bold>Conclusions: </bold>Delusions, agitation, and sleep disturbances may cause distress to the FCs of persons with AD, especially if they live together. Clinicians should meet with FCs regularly and recognize those FCs at risk for a decline in psychosocial health.