Background: When family caregivers are involved in patient care, both patients and caregivers experience better clinical outcomes. However, caregivers experience communication difficulties as they navigate a complex health care system and interact with health care providers. Research indicates that caregivers experience a communication burden that can result in topic avoidance and distress; however, little is known about how burden stemming from communication difficulties with health care providers relates to caregiving outcomes. Objectives: To investigate how family caregiver communication difficulties with health care providers influence caregiver quality of life and anxiety. Methods: Data were collected in a cross-sectional online survey of 220 caregivers with communication difficulties resulting from caregiver avoidance of caregiving-related topics, inadequate reading and question-asking health literacy, and low communication self-efficacy. Results: Caregiver outcomes were not affected by reading health literacy level but did differ based on question-asking health literacy level. Adequate question-asking health literacy was associated with lower anxiety and a higher quality of life. Caregivers who avoided discussing caregiving topics reported higher anxiety and lower quality of life and caregivers with increased communication self-efficacy reported a higher quality of life. Conclusion: Involvement of family caregivers in care is likely to require tailored approaches that address caregiver communication and health literacy skills. Findings from this study suggest that hospice and palliative care providers should identify and provide support for caregiver communication difficulties in order to positively influence caregiver quality of life and anxiety.