Developing economic measures for Aboriginal and Torres Strait Islander families on out-of-pocket healthcare expenditure

Keywords: Aboriginal and Torres Strait Islander health, assessment, families, First Nations health, out-of-pocket healthcare expenditure, reliability, survey, validity.

Our services are tired of seeing patients go without medicines and get really ill because they physically can’t get to a chemist shop, or because they can’t afford their medicines. (Dr Puggy Hunter¹)

Out-of-pocket healthcare expenditure (OOPHE) is a global health concern, significantly impacting families and patients who experience the greatest health inequities.^2–8 OOPHE are any expenses not covered through universal taxpayer-funded health insurance or private health insurances and relate to the illness or condition. In high-income countries, OOPHE is enabled through increasing reliance on healthcare co-payment by patients and the expectation of patients to fund other healthcare needs. This includes transportation to access health-related appointments, additional pharmaceutical costs, or equipment purchase. In addition to OOPHE, patients and families also face indirect costs related to loss of income from being sick or caring for family who are sick.

In Australia, the health service is structured as a taxpayer-funded health insurance known as Medicare.^2,9 Medicare includes the Medical Benefits Scheme and Pharmaceutical Benefits Scheme, which are applied across a range of healthcare settings.⁹ Households can purchase private healthcare insurance for coverage at private hospitals and subsidies of other services. This does not protect all Australians, with households reporting lower confidence in affording health care for serious illness when compared with households in other high-income countries, such as England, New Zealand and Sweden.¹⁰ The impact that OOPHE has on Australian households includes: sacrificing general necessities (e.g. food, clothing, accommodation), medication compliance from inability to afford prescriptions, delaying or not seeking medical treatment, through to significant economic hardship, which induces bankruptcy.^{1–3,8,11–13}

In Australia, the majority of OOPHE research has focussed on the general population, or on older people suffering from chronic conditions.^3–5 There is limited research exploring the substantial burden, financial hardship, or specific cultural and social nuances of OOPHE on Aboriginal and Torres Strait Islander families. This is despite Aboriginal and Torres Strait Islander patients having a higher burden of chronic disease, reporting their health as poorer, higher admissions to tertiary health services, and greater risk of economic hardship from management of chronic conditions, such as chronic heart failure, as compared with other Australians.^14,15 In Australia, Aboriginal and Torres Strait Islander children have a higher burden of burn injury as compared with other Australian children. The Coolamon Project is a large study exploring this impact, including the financial implications of burn injury to Aboriginal and Torres Strait Islander families.¹⁶ In this study, OOPHE explorative work has been conducted with Aboriginal families, revealing OOPHE impacts and financial strain on households (currently under publication review).

The aim of this study was to modify an existing Australian OOPHE survey and psychometrically assess this survey for Aboriginal and Torres Strait Islander households. Currently in the health and medical research setting, it is commonplace to use survey tools that are only psychometrically assessed in the dominant Australian population. This is not optimal for Aboriginal and Torres Strait Islander communities, as it reinforces dominant colonial knowledge constructs.^17,18 To address this gap, this study aimed to:

1. Conduct a scoping review of OOPHE measures for Australian households; and

2. Assess a modified version of an OOPHE measure with Aboriginal and Torres Strait Islander households.

Knowledge interface methodology was employed in this study, bringing together psychometric assessment survey methods (Fig. 1) with Indigenous knowledges (Supplementary Figure S1).^19,20 Indigenous knowledges were important for the overall study conceptualisation (item creation, participant focus, outcome contextualisation, interpretation, and resetting dominant norms). In this process, the first author drew on their own lived experience as an Aboriginal woman, decolonisation methodologies, Indigenous data sovereignty processes, and Yarning with their Aboriginal supervisor and members of the Coolamon Study Aboriginal Steering Committee.^21–23 For example, outcome contextualisation was important at the interface (Supplementary Figure S1), as psychometric assessment of quantitative measures is firmly embedded in Western knowledge systems (i.e. quantitative research methods (validity, reliability)). This is not to dismiss the imperative process of psychometric assessment, but to ensure decolonisation is employed to leverage and critically analyse these processes and understand whether they were enabling, assumptive or disempowering. These processes are to ensure outcome contextualisation. The first author participated in multiple Yarning sessions with senior biostatisticians and their Aboriginal supervisor.²³ This approach was significant in ensuring an appreciation of both knowledge systems, one where the required psychometric assessment methods for the OOPHE survey could be conducted but enquired and informed upon in the context of Indigenous knowledges.

Fig. 1.  Process of psychometric assessment undertaken for modification of the out-of-pocket health expenditure (OOPHE) questionnaire.

A scoping review was conducted in May 2017, to identify OOPHE tools specific to Australian households, following methods outlined by Munn et al.²⁴ Four key databases were searched (Medline, Scopus, Web of Science, and Google scholar), using a combination of key search terms ‘out-of-pocket’ and ‘Australian’ in the title or abstract. A total of 10 papers met inclusion requirements; studies on OOPHE focussed on Australian patients/carers/families/households, over the last 10 years. Various methods (interviews, surveys, mixed, and general estimations) were used in these studies to determine OOPHE impacts on Australian households, with the majority (80%) using surveys. In these eight studies, four different OOPHE surveys featured a:

1. Financial stress scale;²⁵

2. OOPHE questionnaire;^3,7,9

3. Economic survey;²⁶

4. Questionnaire (no details).²⁷

No psychometric assessment had been conducted on any of these surveys. The OOPHE questionnaire (2) was used in 40% of studies, making it the most commonly used survey.^3,7,9,28 Although psychometric assessment had not been conducted on the OOPHE questionnaire (2), it was selected as being the most appropriate for modification as it contained focussed and targeted items on OOPHE.

Judgement quantification for face and content validity, with content experts, followed similar processes to Ryder et al. (2017) and Lynn (1986).^29,30 The content experts consisted of five professionals:

• burns specialist with clinical experience in remote and Aboriginal health settings in the Northern Territory;

• health economist specialised in OOPHE measures in Australia;

• Aboriginal health researcher specialised in chronic conditions;

• Two Injury researchers specialised in housing, injury and women’s health in Aboriginal and Torres Strait Islander Health.

Content validity index (CVI) was used for assessment, where content experts rated the relevance of each item in the OOPHE survey on a four-point Likert scale to determine item-level CVI (I-CVI).^30,31 I-CVI was calculated from content expert responses; items in the OOPHE survey with I-CVI >0.80 remained unchanged, items with I-CVI of 0.50–0.80 were modified on content expert input, items >0.50 were removed. Inter-rater reliability was calculated through an interclass correlation coefficient (ICC) through a two-way mixed effects model where ICC values <0.50 indicated poor reliability, 0.50–0.75 moderate reliability, 0.75–0.90 good reliability and values >0.9 excellent reliability.^32–34 All calculations were completed using Stata version 15.1 (StataCorp).

Participants for this study were parents or carers of Aboriginal and Torres Strait Islander children. Participants were included in the study if they identified as an Aboriginal and/or Torres Strait Islander family where by one parent/career identified as an Aboriginal and/or Torres Strait Islander person, their children were under the age of 16 years, the parent/career had Internet and accessed social media and sales platforms (i.e. Facebook or Gumtree) and email. A target sample size of 30 participants was established for the OOPHE survey assessment; this target size is recommended for pilot studies and has been used in other studies testing surveys for psychometric assessment.^35–38 Participants were recruited through social media (i.e. Facebook, Twitter, Newsletters), Online Marketplaces (i.e. Gumtree) and email. A range of Aboriginal and Torres Strait Islander organisations assisted in distributing advertisements through their networks. Participants who completed both the OOPHE surveys were provided with payment for their time. The OOPHE survey was administered through REDCaps, where participants were reminded through email and/or text message to complete the survey at the 2-week interval period.

Exploratory factor analysis (EFA) was used to discover the number of distinct factors (latent constructs) and examine the pattern of inter-relatability between items in the OOPHE survey.^39,40 Prior to EFA, the OOPHE survey Test 1 outcomes data were cleaned and reviewed. This review required only including OOPHE survey items specific to capturing OOPHE and items that met polychloric correlation assumptions (Supplementary Figure S2). OOPHE survey items were then constructed into a polychloric correlation matrix for continuous, ordinal and binary items. EFA suitability was examined for common variance through Kaiser–Meyer–Olkin (KMO) measure of sampling where a value >0.5 was considered suitable, and equal variance through Bartlett’s Test of Sphericity, where a P value <0.05 was considered appropriate.^29,41,42 Scree plots and parallel analyses were used to determine the number of factors to retain and promax rotation was used to simplify the factor loading matrix structure for EFA and data interpretation.^39,43,44 Internal reliability was calculated by Cronbach’s α on factor discovery; high values are desirable, so we decided values >0.70 were considered good, and values of 0.50–0.70 were adequate.^29,45,46 For items that did not have a clear concise loading, or cross-loaded to factors, a value of 0.15 was considered the lowest acceptable cut-off for loading.⁴⁷

Reproducibility of the OOPHE survey using test–retest methods over an interval of 2 weeks were used to assess reproducibility and item internal consistency. A 2-week interval period was deemed a suitable period for this assessment, and to decrease the impacts of response shift by participants, this time frame has also been used in previous studies.^29,48–52 An ICC one-way random effects model was used for continuous items, weighted kappa for polytomous items with quadratic weighting for items with ordinal scales and kappa for dichotomous items to assess reliability.^{32–34,51,53} Outputs of 1.00–0.81 represented perfect agreement, 0.80–0.61 substantial agreement, 0.60–0.41 moderate agreement, 0.40–0.21 fair agreement, 0.20–0.00 slight agreement and <0.00 poor agreement, based on the scale from Landis and Koch.⁵⁴ Internal consistency of each item scale was calculated using Cronbach’s α, where an α score of ≤0 demonstrates no agreement, ≤0.5 unacceptable agreement, and ≥0.7 acceptable agreement.⁵⁵ All calculations were conducted in Stata version 15.1 (StataCorp).

Ethics approval was acquired from various jurisdictions (Table 1).

Table 1.  Human research ethics approval for scoping study by jurisdiction
NSW, New South Wales; Qld, Queensland; SA, South Australia; NT, Northern Territory

A total of four out of five expert panel members participated in survey content review. ICC inter-rater reliability between individual items was calculated at 0.23 (indicating poor reliability), but the absolute ICC was calculated at 0.55, indicating moderate reliability.³² Of the 192 items in the modified Essue^3,7,9,28 OOPHE questionnaire, 25 items were removed, 33 items modified, and one item added, based on feedback from content experts. This led to the final iteration of the OOPHE survey that was piloted, containing 168 items.

A total of 67 parents participated in Test 1 of the OOPHE survey, where 47 parents went on to complete Test 2. Most parents who participated were male, between the ages of 34 and 44 years and identified as Aboriginal and Torres Strait Islander and were employed (Table 2). Most households were in metropolitan locations and reported middle socioeconomic status.

Table 2.  Parent demographics from Test 1 (n = 67)

Cleaning and review of Test 1 for the OOPHE survey (Supplementary Figure S2) resulted in construction of a 62-item correlation matrix for EFA (Supplementary Table S1). Suitability for EFA provided a KMO value of 0.00 (undesirable) and Bartlett’s Test of Sphericity P < 0.00 (appropriate). Scree plot (Fig. 2) and parallel analyses with the application of the Kaiser’s criteria (eigenvalues >1.0) suggested a three-factor model for the OOPHE survey.^56,57

Fig. 2.  Scree plot of eigenvalues after exploratory factor analysis (EFA).

Over half of the items (n = 34) loaded to Factor 1, which on item review was named Financial Strain/Impacts (Table 3). In Factor 1, correlations ranged from 0.21 to 0.81, with 32% (n = 11) of items loaded with low correlations (<0.40), and 12% (n = 4) loading with high correlations (>0.70). This was similar for Factor 2 (Family Income and Support), with 33% (n = 5) loading with low correlations (0.15–0.39) and Factor 3 (Injury/Condition Impacts) with 40% (n = 4) loading with low correlations (0.22–0.40). Over half of the items (60%, n = 6) did load with high correlations (0.70–0.90) to Factor 3, whereas only 13% (n = 2) loaded with high correlations (0.70–0.75) to Factor 2.

Table 3.  Exploratory factor analyses and internal consistency of the OOPHE survey (n = 67, Items = 62)
Items – Injury33, Finances44, and Finances47, did not meet loading cut-offs (≥ 0.15)

Test–retest reliability results (Table 4) indicated that 75% of OOPHE survey items (n = 43) reached slight to fair agreement, with an ICC range from 0.03 to 0.22 for continuous items, and a 0.00–0.60 Kappa range for ordinal and binary items. The remaining 14 items reached a poor level of agreement with ICC and Kappa ranges below 0.00. Internal reliability reached a good level (>0.70) for Factor 1 – Financial Strain/Impacts at 0.71 but was inadequate for Factor 2 – Family Income and Support at 0.46 and Factor 3 – Injury/Condition Impacts at 0.38 (Table 3). The internal consistency for all items reached a good level (0.72); factor removal improved internal consistency to 0.83 for removal of Factor 1, but did not improve for removal of Factor 2 (0.63) or Factor 3 (0.55).

Table 4.  Test–re-test reliability of the OOPHE survey (n = 47)
CI, confidence interval; NA, not applicable

In this study, we modified an existing OOPHE survey, using face and content validity, and psychometric pilot testing. Construct validity identified a three-factor model through EFA. Approximately 20% of items (n = 12) produced high loadings, representing good interconnectedness with their factors.⁵⁷ Although 32% (n = 20) of items produced low correlations with their factors, the remaining 48% (n = 40) of items produced medium level (0.4–0.7) loadings; this suggests interrelatedness between factors, which transpires through factor cross loadings.^57,58 Given this is the first iteration of the OOPHE survey for Aboriginal and Torres Strait Islander households, it is likely items are not measuring one concise latent construct, but in actual fact, items are exhibiting interconnectedness across the three-factor model.^58,59

The majority of OOPHE survey items met a reliability level of slight to fair agreement (75%, n = 43). For the 14 items that yielded a poor level of reliability, half (n = 7) loaded to Factor 1 and one-third (n = 5) related to Factor 2. In addition, most of these items (n = 8) had an ordinal response scale. The poor kappa produced in these items could be due to chance, but is more likely caused from systemic differences. These systematic differences could be due to factors unrelated to the OOPHE survey, such as the homogeneity of the parent cohort (Table 1). In our cohort, most parents where employed, aged 35–44 years, with only one child at home. Homogeneity in study cohorts has been found to decrease reliability estimates of surveys.^48,60 High study attrition can also contribute; in our study, we had a 30% attrition rate between test 1 and test 2; this can create a situation where the test 2 homogeneity is greater than test 1, further impacting reliability measures.⁴⁸ Additional systemic differences impacting on item reliability have been suggested in dominant populations, such as parent rehearsal or recall where parent reflection alters item responses the second time, and transition errors where variations in mood or feelings from parents impacts on item responses for the retest.⁴⁸ These systematic differences could be related to item and scale clarity, expression and interconnectedness, along with the overall length and time needed to complete the OOPHE survey.^29,45,59 Poor reliability results have also been associated with surveys that take longer periods of time to complete; this would be especially true for parents whom are time-poor or mentally fatigued from full-time work, children’s extracurricular activities, managing households or additional extended family obligations.⁶¹ A range of these systematic differences may not be applicable in this context, as they are grounded in concepts and understandings from the unacknowledged but dominant population.²² It is unclear how impacts of marginalisation from ongoing colonisation, transgenerational trauma and grief, holistic concepts of health and wellbeing, culturally unsafe settings and the multiple determinants of Indigenous health, manifest in this area as they remain unexplored, but are important for First Nations households and must be considered in future work.

This pilot study has demonstrated pertinent points for future work in this area, such as the complexities in developing robust, culturally safe and specific surveys, which reach ideal psychometric levels of validity and reliability for Aboriginal and Torres Strait Islander communities. Certainly, it is our recommendation that in the absence of purpose-built surveys, generic surveys must be adapted for use with Aboriginal and Torres Strait Islander communities. This requires a gold standard approach to psychometric assessment, which further engages with Indigenous knowledges, research methodologies and methods.^17,22,62 Such processes are decolonising in their actions, and act to create data sovereignty in this space for Aboriginal and Torres Strait Islander households.^22,62 Recommendations for future work in this area, would be to move back to face and content validity. In doing this, researchers must work closely with Aboriginal and Torres Strait Islander families to obtain an understanding of: the unique nature and context of OOPHE on households, appropriate number of items, suitable survey length, ideal scales and measures including the use of visual representations. These factors will ensure the construction of a robust and relevant OOPHE survey for Aboriginal and Torres Strait Islander households. Only after these actions occur can additional psychometric assessment be undertaken with a larger diverse cohort of Aboriginal and Torres Strait Islander households.

The major strength of this paper was the use of knowledge interface methodology, in which psychometric assessment methods were used with Indigenous research methods, to test a modified OOPHE survey for use with Aboriginal and Torres Strait Islander households. This study faced limitations; KMO assessment did not reach sampling adequacy, the lowest acceptable cut-off (0.15) for items that cross-loaded or did not load concisely may have been too low.⁴⁷ We continued as this is a pilot study, but note this limitation maybe reflective in our sample-to-variable ratio.⁴¹ Also, participant bias may be present through participant homogeneity.

Aboriginal and Torres Strait Islander families with children who are hospitalised for an injury or condition experience OOPHE impacts. We found that a modified OOPHE survey for Aboriginal and Torres Strait Islander households contained promising items; however, the introduction of contextual factors to the pre-existing survey was impracticable. We recommend that OOPHE surveys should be developed with Aboriginal and Torres Strait Islander families from the outset, so they can include important contextual factors for Aboriginal and Torres Strait Islander households.

The authors declare no competing interests.