Objectives: Novel coronavirus disease (COVID-19) pandemic could increase the mental health burden of family caregivers of older adults, but related reports are limited. We examined the association between family caregiving and changes in the depressive symptom status during the pandemic. Methods: This cross-sectional study included 957 (mean age [standard deviation] = 80.8 [4.8] years; 53.5% females) community-dwelling older adults aged ≥ 65 years from a semi-urban area of Japan, who completed a mailed questionnaire. Based on the depressive symptom status assessed with the Two-Question Screen between March and October 2020, participants were classified into four groups: “non-depressive symptoms,” “incidence of depressive symptoms,” “remission from depressive symptoms,” or “persistence of depressive symptoms.” Participants were assessed in October 2020 for the family caregiving status, caregiving role, the severity of care recipients’ needs, and increased caregiver burden during the pandemic, each with the simple question. Multinomial logistic regression analysis was applied to obtain the odds ratios (ORs) and 95% confidence intervals (CIs) for changes in depressive symptom status. Results: Compared to non-caregivers, family caregivers were associated with the incidence (OR [95% CI] = 3.17 [1.55–6.51], p < 0.01) and persistence of depressive symptoms (OR [95% CI] = 2.39 [1.30–4.38], p < 0.01). Primary caregivers, caregivers for individuals with severe care needs, and caregivers with increased burden during the pandemic had a high risk of depressive symptoms. Conclusions: Family caregivers had a high risk of depressive symptoms during the pandemic. Our findings highlight the need for a support system for family caregivers.