Aim: Early‐onset dementia (EOD) (defined as dementia onset before age 65) presents specific challenges and issues, adding to the negative impact of dementia on the health‐related quality of life (HRQOL) of both patients and their caregivers. However, very few published studies have specifically compared the HRQOL of caregivers of people with EOD and late‐onset dementia (LOD). This information is critical in allocating and prioritizing scarce health‐care resources. We aimed to assess the HRQOL of primary informal caregivers of community‐dwelling individuals with EOD in Singapore and compare it with that of caregivers of individuals with LOD. Methods: This was a cross‐sectional study of consecutive patient–caregiver dyads from a tertiary dementia clinic. Results: No significant differences in disease severity were found between the 111 EOD and 235 LOD patient–caregiver dyads. The mean Mental Component Summary score of the 36‐item Short‐Form Health Survey version 2 was significantly worse in caregivers of EOD patients than in LOD caregivers (mean: 41.42 vs 45.12, P = 0.001), although the mean Physical Component Summary scores were comparable (49.71 vs 49.53, P = 0.934). However, the impact of dementia early onset on caregivers' mental health diminished immediately after adjustment for the disease severity indicators, of which the Neuropsychiatric Inventory Questionnaire distress score was the only significant clinical factor (regression coefficient β = −0.29, P < 0.001). The amount of variability in the HRQOL of the caregivers explained by patient and caregiver factors across all the models was rather small (adjusted R2 = 19.3% for the Mental Composite Score, 5.2% for Physical Composite Score). Conclusion: Caregivers of EOD patients had worse mental health than LOD caregivers probably because individual with EOD have more behavioural disturbances. This reinforces the indispensable role of managing behavioural problems when caring for a family member with dementia, especially for EOD. HRQOL ideally needs to be assessed based on self‐report rather than inferences from indirect data such as the subjective caregiver burden.