Background: The objective of this study was to assess the complex relationship between the multiple determinants of the caregiving process, the caregiver burden, and depression during the COVID-19 pandemic in Serbia. Methods: A cross-sectional study was conducted on a nationally representative sample (n = 798) during the COVID-19 pandemic in Serbia from March to September 2020. A nine-section questionnaire designed for this study included the characteristics of caregivers, characteristics of care and care recipients, COVID-19 related questions, and the following standardized instruments: 12-Item Short-Form Health Survey, Fatigue Severity Scale, Activities of Daily Living Scale and Instrumental Activities of Daily Living Scale, Zarit Caregiver Burden Scale, and Beck Depression Inventory. Path analysis was used for the simultaneous assessment of the direct and indirect relationships of all determinants. Results: More than two thirds (71.9%) of informal caregivers experienced a burden, and more than one quarter (27.1%) had depression symptomatology. Self-rated physical health, need for psychosocial support, and caregiver burden were the main direct predictors of depression. Multiple determinants of the caregiving process had indirect effects on depressive symptomatology via the caregiver burden as a mediating factor. Conclusions: The subjective burden presented a significant risk factor for depressive symptoms in caregivers during the COVID-19 pandemic. The provision of psychosocial support was identified as an important opportunity to reduce depressive risk in informal caregivers.