It is unclear whether caregiving has an impact on the physical, mental and functional health of older caregivers. This study aimed to describe physical, mental and functional health in relation to family caregiving in old age (60+) over a six-year period. The study comprised 2,294 randomly selected individuals (60-96 years) from the Swedish National Study on Aging and Care, who answered the question on whether they were caregivers and who were followed up six years later. The prevalence of family caregivers was 13.1% and the incidence was 12.4%. Four tracks (T) were identified; T1) Family caregiver both at baseline and follow-up (n = 74), T2) Family caregiver at baseline but not at follow-up (n = 226), T3) non-caregiver at baseline but family caregiver at follow-up (n = 218), T4) non-caregiver both at baseline and follow-up (1,776). Only non-caregivers (T4) reported a decline in mental health, p < .036. Worries about health increased significantly in T2 and T4. The prevalence of caregivers was 13.1% with a high turnover. There are differences between family caregivers and non-caregivers in deterioration in physical and mental health as well as physical function over a six-year period.