The impact of Chatbot-Assisted Self Assessment (CASA) on intentions for sexual health screening in people from minoritised ethnic groups at risk of sexually transmitted infections

Chatbot-Assisted Self Assessment (CASA)

Stigma and discrimination can create barriers to accessing necessary healthcare services, which is where conversational AI can be particularly useful. Chatbots can make it easier for people to assess their sexual health needs and encourage more open conversations around sensitive topics, such as STIs.²⁰ This could lead to a higher frequency of clinical consultations and uptake of STI screening, compared to traditional face-to-face settings. Chatbots can also facilitate numerous behaviour change techniques to promote healthy habits, while presenting content in multiple translations, thus overcoming language barriers.²¹ Chatbots can help with goal setting, self-monitoring and provide personalised feedback, empowering users to better understand and manage their health. By offering emotionally intelligent support and tailored messaging, chatbots can help overcome barriers to STI screening that are experienced by ethnic minorities such as stigma, fear of discrimination and challenges related to language and culture.²² Additionally, disclosing information to chatbots can motivate some individuals from linguistically diverse groups to access appropriate healthcare services in their language. Meta-analyses indicate a ‘question-behaviour effect’, where personal and health-related enquiries from chatbots significantly impact behaviour change.^23,24 This suggests that AI systems incorporating conversational techniques could lead to better user reflections and positive health outcomes.

Our previous mixed-methods research identified design principles for a chatbot-based intervention aimed at behaviour change in ethnically diverse communities.¹⁸ The current study aimed to apply CASA principles in the context of ethnic disparities in sexual health outcomes. Our objective was to examine the impact of chatbot conversation on motivation for STI screening as well as understand the attitudes and level of information disclosure to the chatbot.

Design

A within-subject design was used to assess the impact of CASA on intentions for STIs/HIV screening, concern about STIs and a general attitude towards STI screening. While the dependent variable was screening intentions, the independent variables were demographic and behavioural factors related to STI/HIV susceptibility. The study was approved by the University of Westminster Research Ethics Committee [ref: ETH2223-0608].

Participants and recruitment

The study focused on ethnically diverse populations that are likely to be at an increased risk of STIs. Eligible participants were above the age of 18 years, residing in the UK and able to provide informed consent. Those who indicated their residence outside the UK were excluded from the analysis to make the study relevant to the UK healthcare context.

Between December 2022 and July 2023, we used social media platforms, including Twitter, Facebook, Instagram, and Reddit, and employed various handles and hashtags relevant to our study aims, such as ‘#BlackSexualHealth’. We also advertised our study on Prolific, a platform specifically designed for research participation and data collection. We reached out to student organisations and unions in UK-based universities to circulate our study advert. We distributed paper leaflets and posters across multiple university campuses and halls of residence in London and used an in-person leafleting strategy in public spaces across London with a high representation of people from ethnically diverse communities. We also engaged with third sector organisations such as HIV and sexual health charities to circulate our study advert within their networks. Due to the multi-channel approach to recruitment, we could not record participation rates via each recruitment strategy.

Procedure and measurement

Potential participants could access the study, hosted on Qualtrics, via a URL or QR code. The information page outlined the study aims, including the CASA simulation for sexual health, eligibility criteria and information about optional entry to a prize draw worth GBP100 for those who completed it. Consent was gathered by ticking a button indicating that participants had understood the nature of the study and their rights. The survey began with demographic questions, i.e. country of residence, age, ethnicity, gender identity and relationship status. Next, participants were asked to indicate their agreement to three statements related to sexual health screening: (1) their intentions to undergo a sexual health check-up in the next 3 weeks; (2) their concern about STIs; and (3) their attitude towards regular sexual health screening. For all survey attitudinal items, seven optional responses ranged from ‘−3 Strongly disagree’ to ‘0 Neither agree nor disagree’ and ‘+3 Strongly agree’, with higher scores indicating stronger intentions, more concern and positive attitudes.

Participants were then asked for a nickname combining four letters and two numbers to link their responses with chatbot conversations. They were given information about CASA and its purpose, including a confidentiality statement. Participants were asked to exit the Qualtrics survey using a link to a platform where the CASA was hosted. There, the chatbot introduced itself and asked users to insert their nickname. Using a conversational approach, it asked users about their age, gender identity, racial identity, ethnic origin and the language they speak at home. Next, it asked behavioural questions such as the last time they had sexual contact (if ever), any current STI symptoms, past STI diagnosis, the gender of their sexual partners, the consistency of condom use, the last time they tested for STIs, and the number of sexual partners since the last STI test. Afterwards, the chatbot informed the users about their eligibility for a free sexual health check-up and asked if they wished to learn more. The last question of the CASA asked whether users wanted to order an STI screening kit, book a check-up appointment, talk to a human operator or visit the Positive East website for more information. Throughout the conversation, the chatbot language used empathic language and explained the purpose of asking these specific questions to promote self-reflection about personal susceptibility to STIs. Users were also given a ‘prefer not to say’ option to indicate non-disclosure of information.

The chatbot then asked users to complete the Qualtrics survey, which again asked about intentions, STI concerns, and attitudes towards sexual health screening. There were also seven attitudinal questions about CASA related to the ease of use, satisfaction, perceived relevance and usefulness. The participants were then presented with a link to another chatbot called Pat, hosted by leading HIV charity – Positive East (www.chattopat.com).²⁵ This chatbot, which used natural language processing was able to understand user questions and offer further information concerning sexual health and HIV as well as it allowed users to book an STI/HIV check-up appointment directly. Users were able to follow pre-selected questions such as ‘Where should I test for HIV or STI?’ or type their questions using a free text box.

Data analysis

Descriptive statistics were first conducted to identify trends in data and check for assumptions. Paired-sample t-tests were conducted to determine the impact of the CASA on intentions and attitudes towards STI screening and STI concerns. ANCOVA tests were then conducted, with baseline STI screening intentions as a covariate and post-CASA STI screening intentions as the dependent variable. Each demographic and behavioural variable was entered as an independent variable to determine its association with STI screening intentions. Afterwards, the attitude variable ‘chatbot is a bad idea’ was reversed and seven attitudes towards CASA variables were summed, ranging from −21 to 21. ANOVA tests were performed to identify demographic and behavioural variables associated with positive attitudes towards the CASA. Finally, a phrase count analysis of the Pat chatbot script was used to explore the type and frequency of information about sexual health and HIV that was requested from users and the number of subsequent STI/HIV screening appointments booked.

Public and patient involvement

Throughout the research process, a Public and Patient Involvement (PPI) group played an integral role, being involved at key stages, from the study’s co-design to data collection to finalising the research outputs.²⁶ This engagement ensured that the study reflected real-world perspectives and addressed relevant concerns. The PPI group comprised eight public members from MEGs, representing a diverse cohort. They actively reviewed and validated the study’s findings, providing valuable insights into the application of the CASA for sexual health services.

Sample characteristics

A total of 548 participants took part in the study (Table 1). The majority were women, native English speakers, identified as Black, with an average age of 30 years. There were equal proportions of those who were single and in a monogamous relationship. While 10% reported not having had sexual intercourse before, 79% of those sexually active reported having sexual partners of the opposite sex, 45% had sex within the past 2 weeks, 41% had one sexual partner since their last STI check-up, 16% had a previous STI diagnosis, and 32% did not use condoms consistently. Just under half (47%) had never been tested for STIs. In response to using CASA, just over half (54%) wanted to learn more about STI testing, 44% reported the willingness to order an STI testing kit and 9% to book a sexual health screening appointment. In general, the non-disclosure of information to the chatbot was low across all variables, ranging between 0.5% and 3%, with one exception for the question about condom use (9%).

Impact of CASA on screening motivations

Before the CASA, most participants had not had strong intentions to screen for STIs (mean, −0.96; s.d., 1.97), had low concerns about STIs (mean, −0.32; s.d., 2.15) and had positive attitudes towards regular sexual health screening (mean, 2.07; s.d., 1.21). After the CASA, there was a significant positive increase in intentions towards STI screening (mean, 0.33; s.d., 2.12) [t(547), −10.3; P < 0.001; Cohen’s d, 0.63], a significant increase in concerns about STIs (mean, 0.32; s.d., 2.17) [t(544), −4.96; P < 0.001; Cohen’s d, 0.30], and a significant increase in positive attitudes towards sexual health screening (mean, 2.36; s.d., 1.02) [t(543), −4.36; P < 0.001; Cohen’s d, 0.26] (Fig. 1).

Fig. 1.

Changes in attitudes and intentions for STI screening, and concern about STIs after chatbot conversation.

The analysis revealed statistically significant differences in intentions to screen for STI (Table 2). These increases in intentions were most pronounced amongst those who identified as Asian/Pacific Islander from south/south-eastern Asia, had sex in the past 2 weeks, reported having an STI symptom or being unsure, had a history of STI diagnosis, had sex with both men and women, had an STI test within 3 months, had four to five sexual partners since the last STI screening, and/or were a non-native English speaker. There were no differences in intentions by age, gender identity and relationship status (Fig. 2).

Fig. 2.

Average difference in intention for STI screening after chatbot conversation by demographic and behavioural variables.

Attitudes towards CASA

There were positive attitudes towards the CASA intervention (mean, 13.30; s.d., 6.73; range, −17 to 21; Table 3). Most thought that the chatbot helped them to understand their STI risk and decide whether they needed to undergo STI screening. The majority thought it was easy to use, were satisfied with the intervention and would recommend the chatbot to others like them. Attitudes differed by ethnic identity [F(6, 537), 2.78; P = 0.01]; post hoc comparisons found that those who identified as Asian had significantly more positive attitude scores than black participants. Attitudes did not differ by age group [F(4,542), 2.01; P = 0.09], gender [F(3,543), 1.64; P = 0.179], or relationship status [F(2,538), 1.66; P = 0.190]. Non-native English speakers had significantly more positive attitudes than native English speakers [t(545), −3.08; P = 0.001]. Attitudes differed significantly by current STI symptoms [F(2,479), 3.13; P = 0.04], those who were ‘not sure’ had more positive attitudes than those who had no symptoms. Attitude did not differ by previous STI test [F(3,475), 0.617; P = 0.604], but differed by condom use [F(3,484), 13.66, P = <0.001], with those who used condoms some of the time having a more positive attitude than those who never used condoms and those who preferred not to say. Those who reported having had two to three partners had more positive attitudes than those who had no partners or one sexual partner.

Chatbot script

The analysis of the Pat chatbot showed that out of 1066 HIV/STI test appointments booked with Positive East, 11% (N = 124) were made directly via the chatbot, with 72% of users identifying with MEGs. Overall, 42% of chatbot users searched for information about STI/HIV testing, 15% wanted to understand ‘when to test for STI/HIV’, 12% suspected having an STI and needed more information, and less than 5% searched for information about Mpox, contraception, HIV pre-and post-exposure prophylaxis, specific STI symptoms, test results, STI treatment and condoms, respectively.

Limitations and further implications

The CASA intervention was created with input from diverse groups, which may explain why it was well received in this study and has the potential for broader use. One of the study’s strengths was the application of two chatbot services: the anonymous CASA and the confidential Pat, which was able to answer additional questions about STI testing and book appointments automatically, making sexual health services more accessible. Our previous research indicated that users were more willing to discuss their sexual behaviours if chatbots were anonymous. As a result, we implemented a two-chatbot design; however, due to technical and ethical issues, it was not feasible to trace an individual user’s journey from the automated sexual history taking to the STI screening test results. Thus, we were unable to report the impact of chatbots on appointment booking and uptake of sexual health screening, that goes beyond motivations and intentions. Nevertheless, our simulation study provides insights into the ecological validity of this intervention. Users from a wide spectrum of STI susceptibility can discuss their sexual health with chatbots, making the findings more generalisable in a wider context. Future randomised controlled trials are needed to understand the impact of the CASA on uptake of screening and overall STI/HIV rates in MEGs.

We used a within-subject design to examine the effect of CASA on individuals’ motivation to undergo STI screening. However, carryover effects may confound the results, as participants’ responses may be influenced by their previous experiences of the three measurements.³⁵ Despite this, the results showed that those who were not sexually active reported reduced STI screening intentions, indicating minimal impact of such effects. Although a wide range of recruitment methods, e.g. online and community-based, were used in the study to reach ‘seldom heard’ populations suspectable to STIs and HV, the recruitment process raises concerns about potential selection bias as the participants were mainly sourced from social media channels and locations, which may not represent the broader UK population. However, the rate of those who had never tested for STIs in our sample is comparable to rates seen in the UK, indicating the potential representativeness of the findings.³⁷ Also, using chatbot interventions may make it difficult to reach the most underserved groups that lack access to technology or the Internet. Future research should address this digital divide to ensure all populations can access AI-powered interventions. Additionally, self-reported data may be subject to social desirability bias, which may affect the accuracy of the results.³⁸ Nevertheless, this bias may be less relevant in digital interventions where concerns about human judgement are eliminated. Further, the study was not designed to track actual behaviour post-intervention, which limits the ability to draw causal inferences regarding the impact of CASA. Therefore, future research should focus on providing evidence of the impact of CASA on individual motivation, behaviour, and STI screening uptake.

The evidence that CASA can enhance motivation for STI screening highlights the importance of leveraging technological advancements to address health inequalities. The chatbot’s ability to mimic sexual history-taking and articulate the rationale behind each query appears to have the effect of increasing concern about STIs and fostering positive attitudes, which are essential precursors to action in health-seeking behaviours. Notably, the increased intention to engage in sexual health screening among those susceptible to STIs/HIV suggests that such digital interventions could be particularly effective in targeting at-risk populations. The study’s finding of high acceptability and the preference for booking appointments through conversational AI suggests that such technologies can play a crucial role in facilitating easier access to healthcare services. This aligns with the wider digital transformation of health care and points towards a future where AI augments provision of health care to meet diverse patient needs.