Purpose: Cancer and its treatment can affect quality of life (QOL) in cancer patient and family caregiver dyads. However, the factors influencing dyad QOL remain inconclusive. Our study was designed to (i) assess dyads’ QOL, and examine the relationship between the QOL of cancer patients and that of their family caregivers, and (ii) investigate factors that may modify this relationship. Methods: Participants comprised 641 cancer patient-family caregiver dyads. Four types of variables were collected as potential influencing factors, including cancer patient–related variables, family caregiver–related variables, family-related variables, and symptom distress–related variables. Results: Generally, family caregivers reported better QOL than cancer patients did. The effect sizes of the correlation (r) between cancer patients’ QOL and those of their family caregivers ranged from 0.08 to 0.27. Various variables influencing the QOL correlations between cancer patients and family caregivers were identified, including cancer patient–related variables (e.g., age, gender, marital status, understanding of the disease, cancer type and treatment); family caregiver–related variables (e.g., being the spouse or offspring of a patient, duration in their role as a family caregiver, understanding of the disease, and amount of time spent on caregiving each day); family-related variables (e.g., cancer patient enjoyed a good relationship with family pre-cancer diagnosis, family was experiencing serious or mild financial burden due to cancer treatment); and symptom stress–related variables (anxiety and depression). Conclusions: Study findings draw attention to QOL and its related factors in cancer patient-family caregiver dyads. This will benefit the development of interventions to improve dyad QOL.