Background: Families are the backbone of caregiving for older adults living in communities. This is a tradition common to Thailand and many low- and middle-income countries where formal long-term care services are not so available or accessible. Therefore, population aging demands more and more young people engaging as family caregivers. Informal caregiving can become an unexpected duty for anyone anytime. However, studies measuring the burden of informal caregivers are limited. We aimed to determine the caregiver burden, both from the perspective of the caregivers as well as that of their care recipients. Method: We used the baseline survey data from a cluster randomized controlled trial providing a community integrated intermediary care (CIIC) service for seniors in Chiang Mai, Thailand, TCTR20190412004. Study participants were 867 pairs of older adults and their primary family caregivers. Descriptive analysis explored the characteristics of the caregivers and binary logistic regression identified factors influencing the caregivers’ burden. Results: The mean age of family caregivers was 55.27 ± 13.7 years and 5.5% indicated the need for respite care with Caregiver Burden Inventory (CBI) scores ≥24. The highest burden was noted in the time-dependence burden domain (25.7%). The significant associated factors affecting CBI ≥24 were as follows: caregivers older than 60 years, being female, current smokers, having diabetes, and caring for seniors with probable depression and moderate to severe dependency. Conclusions: A quarter of caregivers can have their careers disturbed because of the time consumed with caregiving. Policies to assist families and interventions, such as respite service, care capacity building, official leave for caregiving, etc., may reduce the burden of families struggling with informal care chores. Furthermore, caregiver burden measurements can be applied as a screening tool to assess long-term care needs, complementing the dependency assessment. Finally, implementation research is required to determine the effectiveness of respite care services for older people in Thailand.