Community-based COVID-19 vaccination services improve user satisfaction: findings from a large household survey in Bali Province, Indonesia

Study design, settings, and sampling

The current study was part of the COVID-19 vaccination coverage survey. In 2022, we conducted a large-scale household survey to investigate COVID-19 vaccination coverage across nine districts in Bali Province, Indonesia. In this survey, we also collected data regarding user satisfaction. We followed the World Health Organization’s (WHO’s) Vaccination Coverage Cluster Survey reference manual (WHO 2018). One of the objectives of the survey was to assess satisfaction levels among vaccine recipients and to investigate factors influencing community satisfaction levels.

The study population comprised all residents aged ≥12 years in these nine districts who had received at least one dose of COVID-19 vaccination. This study captured the participants’ overall satisfaction with the COVID-19 vaccination, regardless of their vaccination status (partial, complete, or booster). The inclusion criteria for the study were: (1) individuals who had resided in the selected cluster for at least 3 months; (2) those who had received at least one dose of COVID-19 vaccination; (3) individuals aged ≥12 years old; and (4) those willing to be interviewed, demonstrated by signing an informed consent form. Exclusions were made for individuals who were seriously ill, had mental health disorders, or had trouble communicating.

We calculated the sample size using the Excel software provided by the WHO (WHO 2018) with the following parameters: the estimated immunisation coverage (99%); number of strata (nine districts); Delta (5%); Alpha (5%); target number of respondents per cluster (15); ICC for immunisation (0.16); average number to visit to find one eligible subject (1); and response rate (98%). Based on the calculation, we should visit 315 clusters, 16 households per cluster, resulting in 5040 households to be visited. Samples were selected using multi-stage systematic random sampling techniques (detailed in Fig. 1). Initially, 15% of all villages in each district were randomly chosen. In the second stage, a minimum of 30 clusters (i.e. sub-village level) were randomly selected from all available clusters across the chosen villages. The total number of selected clusters in each district was adjusted based on the area’s population size. Finally, 16 households were randomly selected from the lists of household heads in each cluster. A total of 12,120 recipients of COVID-19 vaccination (partial or one dose, complete or two doses, and booster or three doses) were involved in the current study.

Fig. 1.

Multi-stage systematic random sampling processes.

Study instrument

Data were gathered through a structured questionnaire that delved into socio-demographic characteristics, vaccination locations, and user satisfaction (see Sawitri et al. (2022) for the full questionnaires or survey instruments). The vaccination locations served as a proxy to classify the vaccination strategies. For instance, the health facility-based approach encompassed individuals vaccinated at primary healthcare facilities, clinics, or hospitals. The community-based approach pertained to those vaccinated at public spaces, schools, government agencies, workplaces, and places of worship. The home visit or outreach approach applied to individuals receiving vaccination at their residence (see Supplementary File S2 for further description of these approaches). Sixteen statements were used to gauge user satisfaction levels, adapted from Parasuraman’s SERVQUAL approach (Parasuraman et al. 1988) and the findings of a COVID-19 vaccine service evaluation in Surabaya City (Fadhilah et al. 2021). These statements were modified to suit the context of COVID-19 vaccination services in both health facilities and communities. They encompassed four statements on tangibility (visible and physical aspects of vaccination services), four on reliability, three on responsiveness, three on assurance, and two on empathy. Respondents rated these statements on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Satisfaction levels were categorised as either satisfied or unsatisfied, with the satisfied category defined for those scoring ≥ (number of statements x 4) across all dimensions and within each SERVQUAL dimension.

Data collection procedure

Household surveys were carried out from March to July 2022. Trained enumerators, totalling 45, conducted structured interviews under the supervision of five survey coordinators. Enumerators engaged respondents in interviews, initially recording responses on paper-based questionnaires while in the field, before subsequently inputting the data into the KoboToolbox application (The Kobo Organization 2022). Enumerators double-checked for data completeness. In cases of unclear or missing data, enumerators reached out to respondents using the contact information provided in the questionnaire. Survey coordinators reviewed incoming data and verified respondents’ vaccination statuses. One data manager was responsible for managing data within the KoboToolbox ensuring completeness, accuracy, and regular backups.

Data analysis

The data stored on the KoboToolbox server were initially exported into Microsoft Excel (Microsoft Office Professional Plus 2016) format and was then transferred to Stata ver. 14 (Stata/IC 14.2 for Windows) for further analysis. Descriptive analysis was performed to describe socio-demographic characteristics of respondents and research variables. Results from this descriptive analysis were presented in the form of a single distribution table. A difference of proportions test was conducted using Chi Square to see a difference of satisfaction based on some variables. To see the correlation of independent variables to satisfaction level, the simple logistic regression was implemented to overall and each SERVQUAL dimension. All independent variables with a significance level below 0.250 were included in the multiple logistic regression to evaluate relationships of vaccination approaches and socio-demographic characteristics on user satisfaction levels. A multiple logistic regression analysis was not conducted for the responsiveness dimension because no variables were found to have significant effects on satisfaction in this dimension.

Due to the complex sampling designs (see Fig. 1), the selected samples may not represent the population, arising from issues such as unequal probabilities of selection, non-coverage of the population, and non-response. All analyses in this study were conducted using weighting factors. These weighting factors were implemented to mitigate any potential biases arising from the unequal probabilities of sample selection. The weighting factors were based on the probabilities of selection at each stage and were calculated based on the number of units for each stage of the sampling selection (i.e. villages, sub-villages, head of households, and eligible respondents) (see Supplementary File S1). Decisions were made by researchers in consultation with the WHO Indonesia guided by the United Nations Household Survey Manual (United Nations 2005).

Ethics approval

The Survey of COVID-19 Vaccination Coverage in Bali Province 2022 obtained ethical clearance (No. 537/UN14.2.2.VII.14/LT/2022) from the Ethics Commission of the Faculty of Medicine, Udayana University/RSUP Sanglah, dated 16 March 2022. Additionally, survey permission was granted by the Bali Provincial Government (No. B.30.070/719.E/IZIN-C/DPMPTSP), dated 9 March 2022 and a letter of recommendation was issued by the Bali Province COVID-19 Countermeasures Task Force (No. 540/SatgasCovid19/III/2022), dated 28 March 2022. Permission to utilise the survey data for this publication has been obtained from the World Health Organization Indonesia Country Office, the funder of the Survey of COVID-19 Vaccination Coverage in Bali Province 2022.

Socio-demographic characteristics

Table 1 presents a descriptive analysis of socio-demographic of respondents and the proportion difference of satisfaction levels based on socio-demographic characteristics and vaccination locations. The majority of the recipients were adults (42.03%), with a median age of 43 years (interquartile range, IQR = 26). The study identified that 16.35% of respondents belonged to the elderly population, which is considered a vulnerable group and a priority for vaccination. The gender distribution among respondents was nearly equal, with males accounting for 50.8% and females 49.2%. The majority of respondents had completed senior high school (34.88%) and worked in occupations such as farming or labour (25.35%). Geographically, over half of the respondents resided in urban villages (52.72%). Approximately half of the respondents received their COVID-19 vaccine in public spaces (46.23%), including venues such as village halls, open fields, malls, multipurpose buildings, and other public areas.

User satisfaction regarding COVID-19 vaccination services

Satisfaction levels based on socio-demographic characteristics are presented in Table 1. Satisfaction appears to be higher among adolescents, males, housewives, and rural residents. Satisfaction levels based on different vaccination locations are presented in Table 2. The overall satisfaction rate was 84.31%, with small variation (<3%) across the three vaccination approaches. However, it is crucial to note that the number of respondents reached through the home visit approach was very limited, comprising only 30 respondents. Satisfaction levels against the 16 statements across five SERVQUAL dimensions are presented in Table 3. Multiple areas that require further improvement included: implementation of protocols for maintaining seat distance, supervision and management of adverse events following immunisation (AEFI), providing a sense of comfort during vaccination, and a lack of attention from officers during service.

Effects of socio-demographic characteristics and vaccination approaches on user satisfaction

The multiple logistic regression analysis of the overall score across all five dimensions showed that no variables were found to significantly influence satisfaction (P-value > 0.05) (see Table 4). There was also no significant difference of satisfaction level and overall SERVQUAL score between health facility and community-based approaches. During the pandemic, both approaches resulted in high satisfaction rates among vaccine recipients. However, upon closer examination of each SERVQUAL dimension, respondents who received vaccinations through a community-based approach reported slightly higher satisfaction compared to those who received vaccinations through a health facility-based approach.

Age appears to influence satisfaction levels in the assurance and empathy dimensions, with older respondents (pre-elderly and elderly) exhibiting lower satisfaction compared to those in the adolescent group within these dimensions. The statistical values support this observation: for assurance, the pre-elderly (adjusted odds ratio, AOR = 0.62, P-value = 0.020, 95% confidence interval (CI) = 0.42–0.93) and elderly (AOR = 0.65, P-value = 0.036, 95%CI = 0.44–0.97) age groups showed lower satisfaction, while in the empathy dimension, both the pre-elderly (AOR = 0.66, P-value = 0.023, 95%CI = 0.46–0.94) and elderly (AOR = 0.62, P-value = 0.010, 95%CI = 0.43–0.89) groups demonstrated decreased satisfaction.

Regarding education, its impact on satisfaction was observed solely in the tangible dimension. Respondents with higher levels of education were found to be 0.76 times less satisfied with vaccination services compared to those with lower levels of education (P-value = 0.032, 95%CI = 0.59–0.98).

Occupation also plays a role in influencing satisfaction levels, particularly in the assurance and empathy dimensions. Housewives and individuals with other occupations tended to express higher satisfaction compared to civil servants. Specifically, housewives (AOR = 3.31, P-value = 0.004, 95%CI = 1.48–7.39) and those with other occupations (AOR = 2.25, P-value = 0.032, 95%CI = 1.07–4.74) were more satisfied in the assurance dimension. Similarly, in the empathy dimension, housewives exhibited significantly higher satisfaction levels, being 5.24 times more satisfied with vaccination services than civil servants (P-value = <0.001, 95%CI = 2.44–11.25).

Study limitations

In this study, satisfaction data were obtained from all participants regardless of their vaccination status (partial/first dose, complete/two-doses, and booster/three-doses). We recognise that the satisfaction and experiences of first-time vaccine recipients could be different from second-time or booster recipients. The main limitation of this study is that we do not separate the satisfaction levels between these three groups. This approach may influence the satisfaction perceptions of individuals who have had more than one dose, potentially altering their views on the initial vaccination service received. The study solely evaluated performance indicators of service providers, overlooking the expectations of vaccine recipients regarding these aspects. To enhance the accuracy of satisfaction measurements, future research should incorporate assessment of service recipients’ expectations concerning various service dimensions.

There is a significant disparity in respondent numbers among different vaccination strategies. For instance, there were 4689 respondents vaccinated through health facility-based, 7401 through community-based, and only 30 through outreach or door to door approaches. This disparity leads to a wide confidence interval in the analysis results for the outreach or door to door approach, necessitating caution when drawing conclusions in this category.